With the support of the Consultation and Evaluation Committee of the Academic Divisions of the Chinese Academy of Sciences (CAS), a task force headed by CAS Member FANG Rongxiang recently conducted a project entitled Science Value Assessment in an Increasingly Globalized World. Based on the study, the researchers presented a report on the improvement of China’s science assessment system. The following paragraphs present the major ideas of this report.
The central authorities have placed great importance on science and technology (S&T) in recent years, leading to the dramatic growth of S&T investment and the rapid advancement of S&T in China. However, generally speaking, China still lags far behind advanced countries in S&T development, with fewer original discoveries, a lack of direction in research and even occasional instances of scientific misconduct. The improvement of quality in scientific research presents many challenges. The level and competency of China’s science enterprise have not met the national strategic requirement of building a country via innovation-driven development; moreover, they do not accord with China’s international status.
A key reason for this state of affairs is an inadequate understanding of the value of science, thus leading to a science evaluation system far from meeting the needs of science development. While visiting the CAS in 2013, Chinese President XI Jinping said that in order to implement an innovation-driven development strategy it is urgent to remove systematic and institutional barriers. He called for a determined effort to overcome systematic barriers hindering the upgrade of S&T innovation capacity through the development of optimized S&T policies and an improved S&T assessment system. Science assessment, a seemingly specific part of the S&T system, has an overall impact on S&T development. This is because science assessment is not only a direct reflection of the science values in society, but an important tool for S&T management. The process and result of the assessment have a direct bearing on the allocation of S&T resources. In addition, the evaluation criteria act like a “conducting baton,” affecting the behaviors of scientists.
The way we understand the value of science and perform science assessment has a profound influence on the image of Chinese scientists and China’s international S&T cooperation. In recent years, the world academic community has voiced mixed regard for, even skepticism of, China’s scientific undertakings, largely out of concern for our mechanism of, and the criteria adopted for, science assessment. Without a good understanding of science values and a sound system of science assessment, an enormous waste of science resources and vicious competition among scientists might occur, which will damage the academic atmosphere, encourage fraud and seriously impair the drive and creativity of S&T workers. This will not only have a negative impact on China’s science development, the image of Chinese scientists, and China’s national image, but it will also slow down or even halt the efforts to build an innovative country and to sustain the strategy of innovation-driven development.
I. Main Problems with China’s Science Assessment System
1. Wide Influence and Grave Consequences of Utilitarian Values of Science
Some evaluation policies place too much emphasis on the utility of scientific research and make undue linkage between the results of evaluation and the benefits of the people or institution subject to assessment. The practice often leads to the denial of sustainable support to some important scientific researches requiring long-term commitment and efforts but with a higher risk of failure and the basic researches lacking obvious short-term application prospects. In addition, it could encourage scientific misconduct to a certain extent. In order to make S&T resources best serve their interests, some research institutes strive for a variety of research projects without considering if the project serves their missions, leading to the overlap of functions, unclear obligations of S&T planning, redundant investment, and waste and inefficiency of S&T resources. It also discourages institutions from sustainably pursuing knowledge accumulation and core competency buildup in specific fields. As a result, some S&T workers pay excessive attention to material gain and social status brought about by research projects; they seek short-term successes and quick profits without internal dynamics and persistence to painstakingly carry out studies and seek the truth. In recent years, the S&T community has been shamed and society shocked by instances of academic fraud and corruption.
2. Non-academic Interference in Science Assessment Process
First, the evaluation process is extensively influenced by administrative power. Although most assessment processes adopt the mechanism of expert review, the experts’ views and the assessment results are largely subject to the influence of administrative agencies. Second, the lack of transparency and openness in the science assessment process encourages tampering by interested parties, leading to conflicts of interest and to various kinds of power abuse. Third, the assessment process is designed largely to meet the requirement of administrative management and convenience rather than to follow the established rules and to serve scientific research management. Overly detailed, too frequent and complicated reviews have seriously impaired scientific research work; scientists are presented with a great many assessment actions, including project applications, annual reviews and completion appraisals. Under the pressure to submit various reports, researchers are unable to devote themselves entirely to qualified scientific research.
3. Ineffective Social Supervision and Institutional Constraint
Generally, there is a lack of well-designed and effective social supervision and institutional constraint in China’s S&T evaluation practice. The shortage of necessary transparency and openness makes it difficult to enforce external supervision of the evaluation process and to eliminate abuses of power. Without clear and efficient supervision and punishment measures, infractions of this kind can hardly be contained in an effective way. Because the benefits of violating the rules of science evaluation are much greater than the cost, infractions continue to increase.
4. Existing Assessment Criteria Detrimental to Promoting Excellence in Research
Overemphasizing short-term quantitative indicators makes it hard to encourage transformative research. When determining S&T awards, key academic disciplines and major research centers or institutions, for example, emphasis is often placed on the number of papers, projects, prizes, and academicians. Also, excessively linking assessment incentives with personal or institutional benefits has a detrimental effect on the cultivation of excellent research institutions and scientists. For instance, some research institutions are keen to carry out “public relations” activities, even promoting cheating in activities such as research assessment, science prize selection and academician election. Some incentive policies even lead to results contrary to the original intention. For example, the great difference in remuneration between winners of some talent programs and large number of young S&T workers with innovation potential is likely to decrease the attraction of pursuing science excellence. This can have a negative effect on a merit-based resource allocation system; its negative effect could offset even exceed the positive one.
II. Major Root Causes of the Problems
1. Inadequate Understanding of the Value of Science and the Characteristics of Scientific Research
To promote a healthy and sustainable development of science through the improvement of science assessment, it is important to have a correct understanding of the value of science and to precisely follow the rules of scientific research. The role of science in promoting technical progress and socioeconomic growth is entirely based on its continuous discovery and creation of new knowledge. “Excellent” science, which promotes continuous progress in new knowledge, is the source of miracles of applied science. An incomplete understanding of the values of science leads to the neglect of the primary role of science assessment in promoting “excellent science” and, instead, to a pursuit of the utility value of science. This seriously distorts the objectives of science assessment, which not only makes it difficult to effectively improve the quality of scientific research, but also harms the scientific spirit, which will have a negative bearing on the sustainable development of science. Also, as an effort to explore the unknown, scientific research strives to find new knowledge in areas full of uncertainty, where there is no guarantee of success. Science assessment should stimulate the innovation dynamics of scientists, encourage them to conduct explorations, promote enthusiasm, and tolerate their failures. Because of an inadequate understanding of the features and rules of scientific research, our science assessment is often conducted in an administrative way with excessively detailed and rigid regulations. This not only disturbs the process of scientific research but also leads to seriously misguided behavior among scientists.
2. Inappropriate Management and Allocation Modes of S&T Resources
Administration plays a leading role in laying down the evaluation criteria for allocating S&T resources, and therefore science assessment has a tendency to show the record of administrators. In order to be direct in their management, administrators often adopt the simple evaluation methods of quantitative comparison. In addition, the commonly-used practice of S&T resource allocation through competition leads to a serious imbalance between competitive investment in projects and steady investment into institutions. This not only result in an unhealthy competition for S&T resources, forcing scientists to spend a huge amount of their time and energy on applying for research projects, but also deny steady support to important research activities requiring long-term efforts. In addition, S&T resources are often managed separately by different administrative departments, which roll out numerous S&T programs. The unsettled position of the programs often gives uniformity to evaluation indicators and research activities. The overly frequent and overly complicated management measures of research projects, with their excessive details and rigid application requirements and guidelines, cause scientists who could not get steady support for their research to search for resources from different channels and make desperate efforts to adapt to various requirements of different programs.
3. Woefully Inadequate Self-governance and Self-regulation of the Scientific Community
First, China’s scientific community lacks adequate autonomy, still relying heavily on administrative power. It is unable to effectively resist administrative interference in the science evaluation process. Some researchers even obtain personal or institutional benefits through personal relations with S&T administrators. Second, scientists lack sufficient self-regulation. Some researchers turn a blind eye to established evaluation rules and secrecy and even go around the evaluation process and evaluation experts with intensive lobbying. Some evaluation experts do not read the assessment documents seriously before making a judgment; they even allow their students to conduct the evaluation. Third, having inadequate sense of responsibility, the scientific community lacks an adequate self-governance. For instance, many scientists are reluctant to openly challenge or criticize flawed evaluation conclusions. Out of concern for personal relations or embarrassment when they are subject to assessment, many evaluators are reluctant to offend people being evaluated; “going astray collectively” appears occasionally. As the misconduct of some researchers is not duly punished, they serve as a bad example for students and young researchers, making a negative impact on the young generation of researchers.
III. Major Recommendations for Improving Science Assessment System
The following is the basic idea for the recommendation to improve China’s science evaluation system. The objective is to further excellence in science and dramatically upgrade China’s original innovation capacity; accordingly, efforts should be made to deepen the reform of S&T resource management and allocation system and to stimulate the creativity and enthusiasm of researchers. In order to promote a healthy and sustainable development of China’s scientific undertakings, we must understand the requirements of scientific research, follow the rules of science development, and respect the dominant role of researchers. We should set up a science assessment system featuring rigor, transparency and fairness and cultivate a harmonious and dynamic academic environment.
1. Strengthen Education of Science Values and Uphold a Correct Value Orientation
Efforts should be made to guide the scientific community and society in general to attain a correct understanding of science values, taking the discovery of new knowledge as the core mission of science activities and making the exploration of truth an important basis for the comprehensive realization of science values. We should rethink and re-examine the impact of utilitarianism on China’s science policy, adjust the approach to science evaluation, and encourage scientific researchers to accept a scientific attitude featuring preciseness, thoroughness and innovativeness. Evaluation-related incentives should not induce researchers to carry out rushed research for sub-optimally quick results and short-term or individual benefits. We should advocate painstaking research, collaboration, and encourage governmental officials to conduct science management by observing the rules and characteristics of scientific research.
2. Rebuild the Current S&T Management System and Set Up a Resource Allocation Mechanism in Light of the Characteristics of Scientific Research
First, it is advisable to conduct feasibility studies for establishing a National Research Council in China, exploring a new mechanism for S&T resource allocation that is jointly managed by administration and the scientific community. Second, it is advisable to adjust the structure of S&T resource allocation and strengthen the steady support to institutions engaged in basic and public welfare research. Third, overly linking science evaluation to personal and institutional gain should be avoided, thereby preventing a policy orientation of hasty work; instead, full play should be given to the diagnostic role of science evaluation in improving scientific research and upgrading its quality.
3. Standardize Science Evaluation Process and Set up a Strict, Transparent Evaluation System
First, a code of conduct for science assessment should be formulated as soon as possible to address problems in China’s science evaluation. It should be regarded as the guide to action for science evaluation. Second, efforts should be made to strictly standardize the process of science assessment, designing the procedures and mechanisms in a systematic way and making clear the power and responsibility of different stakeholders. It is important to strictly follow the regulations on conflicts of interest and set up a strict challenge system. We should upgrade the transparency and openness of the evaluation process, establish a responsibility and information openness system for evaluators, a supervision mechanism featuring authority, independence and fairness, and a system that protects the lawful rights and interests of whistleblowers. Third, it is important to strengthen international cooperation in science evaluation by giving full play to international experts in that evaluation.
4. Set up Reasonable Evaluation Criteria and Encourage Quality and Original Research
There is a need to formulate reasonable evaluation criteria relevant to the range of research activities and their differences in nature and objectives. The most important criteria of science assessment should be the encouragement of original and transformative research while giving due consideration to academic value and social impact. Special attention should be paid to problem-oriented research. It is important to design a reasonable assessment cycle and operation procedures so as to prevent over-frequent and complicated assessment activities from interfering with regular activities of scientific research; and it is important to cultivate a sound environment for scientific research institutions in which they continuously upgrade their research capacity and scientific researchers devote themselves to key scientific issues.
5. Facilitate Self-governance and Self-regulation of the Scientific Community and Improve Peer Review System
First, there is a need to foster a sense of mission and responsibility in the scientific community in terms of promoting excellence in scientific research. Free academic exchanges and criticism on an equal footing should be encouraged within the community. An effective quality control mechanism for error correction and the selection of the superior should be established by giving full play to the collective error correcting mechanism and the collaboration of scientists. Second, it is important to encourage scientists to guard against undue non-academic interference in science evaluation. The lawful rights and interests of scientists in science assessment should be guaranteed institutionally, and the peer review system should be improved. Third, scientific integrity should be strengthened and the ethics review system should be improved.