متدولوژی تصمیم گیری چند معیاره منعطف برای حمایت از مدیریت استراتژیک برنامه های تحقیقاتی علوم، فناوری و نوآوری A flexible multicriteria decision-making methodology to support the strategic management of Science, Technology and Innovation research funding programs

1. Introduction Policies for Science, Technology and Innovation can complement and reinforce the long-term strategies for social and economic development of a country (OECD, 2012). Research funding agencies contribute to the implementation of the government policies by promoting and supporting research activities. As part of their efforts, agencies usually launch research funding programs (hereinafter, simply referred to as programs) (European Science Foundation, 2011; Guthrie et al., 2013). These programs can pursue one or more objectives, which need to be aligned with the public policies, as well as with the current strategy and mission of the agency (Edquist & ZabalaIturriagagoitia, 2012; Joyce, 2015). In order to evaluate research projects, agencies often rely on the peer review process (PRP). It can be implemented in many different ways (August & Muraskin, 1999; Abdoul et al., 2012). However, it is commonly implemented in such a way that every application is evaluated by one or more ad hoc reviewers and, subsequently, by the members of a committee. Despite being the current dominant practice, PRPs have been widely criticized (Marsh et al., 2008; Abramo et al., 2009; Abramo & D’Angelo, 2011; Ebadi & Schiffauerova, 2016). The criticisms are mainly related to the dependency of the PRP outcomes on the selection of the individuals to be involved in the review process. It is a well known fact that the decision quality may degrade because of conflicts of interest in the committee or due to knowledge and experience limitations of the reviewers. Another typical weakness in most PRPs is associated with the fact that each reviewer often handles only a few submissions. This leads each of them to use different references for evaluating the applications (Marsh et al., 2008), which may introduce more bias into the process, instead of the promised impartiality. Among the possible measures to overcome such problems, it is a common practice to consider multiple reviews for each application and to rely on the final consensual recommendation provided by the committee. However, these measures may not completely prevent the influence of a biased committee or of the cognitive biases, which are inherent in human mind (Kahneman, 2011). In particular, they are not suitable to deal with some collateral effects of the use of highly subjective criteria to evaluate the applications. The excessive subjectivity allows interferences of individual or group interests in the judgments. It also may hinder the ability of the agency staff to discern whether it is necessary to intervene in the review process to prevent deviations from the program objectives. Such occurrences have negative impact for the researchers whose projects are evaluated unfairly, as well as for the reputation of the agency itself. Beyond that, these occurrences can jeopardize an effective implementation of the program (Durlak & DuPre, 2008). In this paper, we propose a methodology, named DEMUCTI (which stands for Decis˜ao Multicrit´erio em Ciˆencia, Tecnologia e Inova¸c˜ao), to support the funding decision. It aids the assessment, evaluation, prioritization and selection of the most adequate eligible applications, under a multi-step decision-making process, which fits into a strategic management process (SMP) within the agency (Costa, 2014). DEMUCTI was developed to meet a demand of the Brazilian agency — the National Council for Scientific and Technological Development (CNPq). The main requirements underlying the methodology scope were that it should be flexible enough to be applied in the context of diverse programs; fit the needs of different PRP implementations; incorporate multiple criteria of a quantitative or a qualitative nature; and enable the agency-wide institutionalization.