RESEARCH ARTICLE
A comprehensive analysis of the journal
evaluation system in China
Ying Huang1,2
, Ruinan Li1
, Lin Zhang1,2
, and Gunnar Sivertsen3
1School of Information Management, Wuhan University, China
2Centre for R&D Monitoring (ECOOM) and Dept. MSI, KU Leuven, Belgien
3Nordic Institute for Studies in Innovation, Research and Education, Tøyen, Oslo, Norwegen
Schlüsselwörter: bibliometrics, journal evaluation, peer review, science journal
ABSTRAKT
Journal evaluation systems reflect how new insights are critically reviewed and published, Und
the prestige and impact of a discipline’s journals is a key metric in many research assessment,
performance evaluation, and funding systems. With the expansion of China’s research and
innovation systems and its rise as a major contributor to global innovation, journal evaluation has
become an especially important issue. In diesem Papier, we first describe the history and background
of journal evaluation in China and then systematically introduce and compare the most currently
influential journal lists and indexing services. These are the Chinese Science Citation Database
(CSCD), the Journal Partition Table ( JPT), the AMI Comprehensive Evaluation Report (AMI), Die
Chinese S&T Journal Citation Report (CJCR), “A Guide to the Core Journals of China” (GCJC),
the Chinese Social Sciences Citation Index (CSSCI), and the World Academic Journal Clout
Index ( WAJCI). Some other influential lists produced by government agencies, professional
associations, and universities are also briefly introduced. Through the lens of these systems, Wir
provide comprehensive coverage of the tradition and landscape of the journal evaluation system
in China and the methods and practices of journal evaluation in China with some comparisons
to how other countries assess and rank journals.
1.
EINFÜHRUNG
China is among the many countries where the career prospects of researchers, in part, depend on
the journals in which they publish. Knowledge of which journals are considered prestigious and
which are of dubious quality is critical to the scientific community for assessing the standing of a
research institution, tenure decisions, grant funding, performance evaluations, usw.
The process of journal evaluation dates back to Gross and Gross (1927), who postulated that
the number of citations one journal receives over another similar journal suggests something
about its importance to the field. Shortly after, the British mathematician, librarian, and documen-
talist Samuel C. Bradford published his study on publications in geophysics and lubrication. Der
paper presented the concept of “core-area journals” and an empirical law that would, von 1948,
become Bradford’s well-known law of scattering (Bradford, 1934, 1984). Im Gegenzug, Bradford influ-
enced Eugene Garfield of the United States, who subsequently published a groundbreaking paper
on citation indexing called “Citation Indexes for Science: A New Dimension in Documentation
through Association of Ideas.” According to Garfield (1955), “the citation index … may help a
historian to measure the influence of an article—that is, its ‘impact factor’.” In the 1960s, Garfield
Keine offenen Zugänge
Tagebuch
Zitat: Huang, Y., Li, R., Zhang, L., &
Sivertsen, G. (2020). A comprehensive
analysis of the journal evaluation
system in China. Quantitative Science
Studien, 2(1), 300–326. https://doi.org
/10.1162/qss_a_00103
DOI:
https://doi.org/10.1162/qss_a_00103
Erhalten: 30 April 2020
Akzeptiert: 18 September 2020
Korrespondierender Autor:
Lin Zhang
linzhang1117@whu.edu.cn
Handling-Editor:
Liying Yang
Urheberrechte ©: © 2020 Ying Huang, Ruinan
Li, Lin Zhang, and Gunnar Sivertsen.
Veröffentlicht unter Creative Commons
Namensnennung 4.0 International (CC BY 4.0)
Lizenz.
Die MIT-Presse
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
conducted a large-scale statistical analysis of citations in the literature, reaching the conclusion
that many citations were concentrated in just a few journals and the many remaining journals
only accounted for a few citations (Garfield, 1963, 1964). Garfield went on to create the
Institute for Scientific Information (ISI), then successively published the Science Citation Index
(SCI), the Social Science Citation Index (SSCI), and the Art and Humanities Citation Index
(A&HCI) databases.
Assessing the quality of published research output is important in all contexts where research
assessment takes place—for example, when evaluating the success of research projects or when
distributing research funding (Su, Shang et al., 2017). As part of the assessment, evaluating and
ranking the quality of the journals where the output was published has become increasingly
important (Mingers & Yang, 2017). Journal evaluation and rankings are used by governments,
Organisationen, universities, Schulen, and departments to evaluate the quality and quantity of fac-
ulty research productivity, ranging from promotion and tenure to monetary rewards (Black,
Stainbank et al., 2017). Even though the merit of using such a system is not universally agreed
upon (Dobson, 2014), and is sometimes even contested (Zhang, Rousseau, & Sivertsen, 2017), Es
Ist, Jedoch, widely believed that the rank or citation impact of a journal is supposed to reflect its
prestige, influence, and even difficulty of having a paper accepted for publication (Su et al.,
2017).
Over the past few years, the number of papers published in international journals by Chinese
researchers has seen a dramatic increase, to the point that, Heute, China is the largest contributor
to international journals covered by Web of Science ( WoS) and Scopus. In tandem, government
policies and guidance, especially the call to “publish your best work in your motherland to
benefit local society,” proposed by President Xi in 20161, are seeing more and more papers
published in China’s domestic journals. daher, with these increases in the number of papers
and journals, it will be an important task to explore the strengths and weaknesses of various
methods for evaluating journals as well as the types of ranking systems that may be suitable
for China’s national conditions.
The journal evaluation system in China was established gradually, beginning with the intro-
duction of Western journal evaluation theories about 60 years ago. Over the last 30 Jahre, In
besondere, these foreign theories have been adopted, adapted, researched, and vigorously rede-
veloped. In the past, journal evaluation and selection results were mainly used to help librarians
develop their collections and to help readers better identify a discipline’s core journals. Jedoch,
in recent years, the results of journal evaluation and ranking have increasingly been applied to
scientific research evaluation and management (d.h., in tenure decisions, grant funding, and per-
formance evaluations). (Shu, Quan et al., 2020). Many institutions are increasingly relying on
journal impact factors ( JIFs) to evaluate papers and researchers. This is commonly referred to
in China as “evaluating a paper based on the journal and ranking list” (以刊评文，Yi Kan Ping
Wen). The higher the journal’s rank and JIF, the higher the expected outcome of evaluations.
In the ever-changing environment of scientific research evaluation, the research and practice
of journal evaluation in China is also evolving to meet different needs. Many influential journal
evaluations and indexing systems have been established since the 1990s, with their evaluation
methods and standards becoming increasingly mature. These activities have played a positive
role in promoting the development of scientific research and have also been helpful for improving
the quality of academic journals.
1 http://www.xinhuanet.com//politics/2016-05/31/c_1118965169.htm
Quantitative Science Studies
301
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
The aim of this study is to review the progress of journal evaluation in China and present a
comprehensive analysis of the current state of the art. Somit, the main body of this article is a
comparative analysis of the journal lists that are most influential in China’s academic landscape.
The results not only offer a deeper understanding of China’s journal evaluation methods and
practices but also reveal some insights into the journal evaluation activities of other countries.
Gesamt, our aim is to make a valuable contribution to improving the theory and practice of
journal evaluation and to promote the sustainable and healthy development of journal manage-
ment and evaluation systems, both in China and abroad.
2.
JOURNAL EVALUATION IN CHINA
2.1. A Brief History
Journal evaluation in China dates back to the 1960s, with some fairly distinct stages during its
Entwicklung. Qiyu Zhang and Enguang Wang first introduced the Science Citation Index
(SCI) to Chinese readers in 1964 (Zhang, 2015). In 1973, Erzhong Wu introduced a core journal
list for chemistry. This was the first mention of the concept of a “core journal” (Wu, 1973). In
1982, Liansheng Meng finished his Master’s thesis entitled “Chinese science citation analysis”
(Meng, 1982), and then, In 1989, he built the Chinese Science Citation Index (CSCI), now called
the Chinese Science Citation Database (CSCD), with the support of the Documentation and
Information Center of the Chinese Academy of Sciences. At this stage of development, interna-
tional journal evaluation practices were simply applied to the Chinese context almost without
making any changes to the underlying methodologies. Gleichzeitig, exploring bibliometric
laws and potential applications became an important topic for researchers in library and infor-
mation science.
In 1988, Jing and Xian used the “citation method” to identify a list of “Chinese Natural Science
Core Journals,” which included 104 core Chinese journals in the natural sciences. This is now
typically recognized as the first Chinese journal list (Jing & Xian, 1988). Around that same time,
some institutions began to undertake journal evaluation activities. Zum Beispiel, In 1987, Die
Institute of Scientific and Technical Information of China (ISTIC) (commissioned by the Ministry
of Science and Technology—formerly the National Scientific and Technological Commission)
began to analyze publications in the SCI, the Index to Scientific Reviews (ISR), and the Index to
Scientific&Technical Proceeding (ISTP), und in 1989 it began selecting domestic scientific journals
for analysis. During this process, 1,189 journals were selected from 3,025 scientific journals
nationwide as statistical sources of journal selection, which have been adjusted annually ever
seit (Qian, 2006). Somit, this second stage of development saw the beginnings of adapting
international evaluation systems and approaches to local journals, and some institutions building
their own citation and bibliographic indexes.
From the 1990s onwards, journal evaluation activities moved on to rapid development with
equal emphasis on theoretical research and practical applications. On the theoretical side,
bibliometric researchers and information scientists were engaged in developing more advanced
evaluation methods and better indicators. The theories and methods of journal evaluation spread
from the natural sciences to the social sciences and humanities (SSH). In terms of practical appli-
Kationen, more and more researchers in the library and information science fields began to depart
from individual research agendas and move into joint working groups and professional evalua-
tion institutions to promote journal evaluation practices. The number of journal lists burgeoned as
well. Some combined “quantitative” methods and “qualitative” approaches, wie zum Beispiel “A Guide to
the Core Journals of China” (GCJC) by the Peking University Library and the Chinese Social
Sciences Citation Index (CSSCI) from the China Social Sciences Research Center of Nanjing
Quantitative Science Studies
302
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
Universität. Others were proposed by joint working groups and research institutions, and these
lists began to be used to support scientific research evaluation and management.
Im Großen und Ganzen, these advances in methods and standards played a positive role in promoting
the quality of academic journals. Jedoch, im Laufe der Zeit, JIFs have tended to become a proxy for the
quality of the papers and authors published within their pages (d.h., “evaluating a paper based on
the journal and ranking list” [以刊评文，Yi Kan Ping Wen]). This phenomenon has been causing
a wide debate nationwide, with many calling for papers to be judged by their content, not by
their wrapping (Zhang et al., 2017). In this regard, the number of solutions proposed to improve
the standards of journal selection and to avoid improper or even misleading use continues to
multiply.
2.2. Motivations for Performing Journal Evaluation in China
Advances in science and technology and the rapid growth of scientific research have brought a
change in the way that journals are evaluated. Anfänglich, the assessments were reader oriented,
serving as a guide for journal audiences to understand research trends and developments in the
various disciplines. Later, greater focus was placed on the needs of libraries and other organiza-
tionen. English core journals were translated into Chinese and introduced to China to ensure better
use of the most valuable journals with limited funds and to optimize the journal collections of
China’s libraries.
Jedoch, with the rapid development of information network technology and the populari-
zation of reading on screen, electronic journal databases are having an unprecedented impact on
journal subscriptions. Weiter, early use of journal evaluation systems by pioneering institutions
has spread beyond the library and information science community. Heute, journal evaluations
are inextricably tied to many aspects of assessing research performance.
The Journal Citation Reports ( JCR), an annual publication by Clarivate, contains a relatively
transparent data set for calculating JIFs and citation-based performance metrics at the article and
the journal level. Weiter, JCR clearly outlines a network of references that represent the journal’s
voice in the global scholarly dialog, highlighting the institutional and international players who
are part of the journal’s community. Jedoch, many journals selected for inclusion in JCR are
from English-speaking countries, as shown in Table 1. To fulfill a growing demand to extend the
universe of journals in JCR, the WoS platform launched the Emerging Sources Citation Index
(ESCI) in November 2015. Jedoch, ESCI has also done very little to promote journals from
non-English-speaking countries and regions (Huang, Zhu et al., 2017). Although English is the
Tisch 1.
The top 10 countries with the highest journal numbers in JCR 2019
Country
USA
Vereinigtes Königreich
Niederlande
Deutschland
Schweiz
Number of journals
in JCR 2019
4135
Number of
publications
470,953
2864
929
721
278
146,734
49,135
127,881
47,005
Country
Japan
China
Frankreich
Australia
Russland
Data Source: Web of Science Group (2019).
Notiz: Some journals in the portfolio of international publishers have no genuine national affiliation.
Quantitative Science Studies
Number of journals
in JCR 2019
248
Number of
publications
88,507
221
192
159
151
497,802
83,152
82,132
44,165
303
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
Figur 1. Der 10 countries with the largest number of publications in WoS (1975–2019).
Notiz: Indexes = SCIE, SSCI, A&HCI; Document types = article, Rezension.
working language of the international scientific community, for many reasons, it is not a wise
choice for researchers to only publish their scholarly contributions in English. Building domestic
evaluation systems turns out to be very necessary for fostering domestic collaborations, appropri-
ately evaluating research performance, and keeping up with research trends close to home.
Darüber hinaus, China’s economy is growing rapidly and, along with it, the country’s scientific
activity is also flourishing. As Figure 1 zeigt an, China’s scientific research inputs and outputs have
consistently increased over the past few decades, exceeding that of the United States in 2019 Zu
become the most productive country in the world. With such a large number of papers, die Arbeit
of researchers cannot be assessed without shortcuts. Daher, for want of a better system, the quality
of the journals in which a researcher’s papers are published has become a proxy for evaluating the
quality of the researcher themselves, and ways to define “core journals” and how to select those
indexed journals have attracted wide attention, especially from the Chinese government.
Außerdem, national policies, such as those listed in Table 2, are now playing a vital role in
these evaluation activities. Early in China’s history of journal evaluation, the policies implemented
were designed to support the development of some influential journals across the natural and social
Wissenschaften. More recently, Jedoch, the government’s policies have sought to reverse the excessive
emphasis that has come to be placed on the volume of a researcher’s output and the JIF of their
venues (Zhang & Sivertsen, 2020). Research institutions and universities are now being encouraged
to adopt a more comprehensive evaluation method that combines qualitative and quantitative
methods and pays more attention to the quality, contribution, and impact of a researcher’s master-
Stück. Somit, more indicators are being taken into account and, upon these, a culture more con-
ducive to exploration is being established that does not prioritize SCI/SSCI/A&HCI journals to the
exclusion of all else.
3. THE LEADING JOURNAL EVALUATION SYSTEMS OF ACADEMIC JOURNALS IN CHINA
Through these three stages of development, multiple institutions in China have established
comprehensive journal evaluation systems that combine quantitative and qualitative methods
and a variety of different indicators, many of which have had a significant influence on scien-
tific research activities. Somit, what follows is a comparison of the current journal indexes in
China. These are the CSCD and the Journal Partition Table ( JPT) from the National Science
Library, Chinesische Akademie der Wissenschaft (NSLC); the AMI journal list from the Chinese
Academy of Social Sciences Evaluation Studies (CASSES); the Chinese S&T Journal Citation
Quantitative Science Studies
304
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
NEIN.
1
2
3
4
5
6
Tisch 2.
The related policies about journal evaluation in China (selected)
Policy document name
Authority
Implementation plan of the project of the famous journals of
philosophy and social sciences in colleges and universities
MOE
Some opinions on the role of S&T journals in the academic
evaluation
Several opinions on further strengthening scientific integrity
The opinions on deepening reform in review and evaluation of
Projekte, talent and institutions
CAST; MOE; SAPPRFT
(Jetzt, NRTA); CAS; CAE
General Office of the CPC
Central Committee; GOSC
General Office of the CPC
Central Committee; GOSC
Some measures to eliminate the bad orientation of “papers only”
MOST
in science and technology evaluation (trial)
Date
2003.11.06
2015.11.03
2018.05.30
2018.07.03
2020.02.17
Some suggestions on standardizing the use of sci paper indexes
MOE; MOST
2020.02.18
in higher educational institutes and establishing correct
evaluation orientation
Notiz: Ministry of Education of the People’s Republic of China (MOE); China Association for Science and Technology (CAST); State Administration of Press,
Publication, Radio, Film and Television of the People’s Republic of China (SAPPRFT), and it was renamed as National Radio and Television Administration
of the People’s Republic of China (NRTA) In 2018; Chinesische Akademie der Wissenschaft (CAS); Chinese Academy of Engineering (CAE); General Office of the
State Council of the People’s Republic of China (GOSC); Ministry of Science and Technology of the People(cid:1)s Republic of China (MOST); Communist Party
of China (CPC).
Bericht (CJCR) from ISTIC; “A Guide to the Core Journals of China” (GCJC) from Peking
University Library; CSSCI from the Institute for Chinese Social Science Research and
Assessment (ICSSRA) of Nanjing University, and the World Academic Journal Clout Index
( WAJCI) from the China National Knowledge Infrastructure (CNKI). Zusätzlich, some other
influential lists produced by government agencies, professional associations, and universities
are also briefly discussed.
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
3.1. NSLC: CSCD Journal List
3.1.1. Hintergrund
The CSCD was established in 1989 by the National Science Library of CAS, with the aim of
disseminating excellent scientific research achievements in China and helping scientists to dis-
cover information. This database covers more than 1,000 of the top academic journals in the
areas of engineering, medicine, Mathematik, Physik, Chemie, the life and earth sciences,
agricultural science, industrial technology, the environmental sciences, und so weiter (National
Science Library of CAS, 2019B). Since its inception, the CSCD has amassed 5.5 million articles
Und 80.5 million citation records. As the first Chinese citation database, the CSCD published
the first printed book of journals in 1995 and the first retrieval CD-ROM in 1998, followed by
an online version in 2003. In 1999, it launched the “CSCD ESI Annual Report” and in 2005 Die
“CSCD JCR Annual Report”, which are similar to the ESI and JCR and very well known across
China. Jedoch, perhaps the most notable feature of CSCD is its cooperation with Clarivate
Analytics (formerly, Thomson-Reuters) In 2007 to offer a cross-database search with the WoS,
giving rise to the first-ever database of non-English-language journals.
CSCD provides information discovery services for analyzing China from the perspective of the
world and analyzing the world from the perspective of China. daher, it is widely used by
research institutes and universities for subject searches, funding support, project evaluations,
declaring achievements, talent selection, literature measurement, and evaluation research. Es ist
Quantitative Science Studies
305
A comprehensive analysis of the journal evaluation system in China
also an authoritative document retrieval tool (Jin & Wang, 1999). Jin, Zhang et al. (2002) Und
Rousseau, Jin, and Yang (2001) both provide relatively thorough explorations and discussions
of this journal list.
3.1.2.
Journal selection criteria
The CSCD journal list is updated every 2 Jahre, using both quantitative and qualitative
Methoden. The most recent report (2019–2020) was released in April 2019 and listed 1,229
source journals in total: 228 English journals published in China and 1,001 Chinese journals.
The selection criteria are summarized below (National Science Library of CAS, 2019A).
Journal scope The journal must be published in either Chinese or English in China, mit
3.1.2.1.
both an International Standard Serial Number (ISSN) and a China Domestic Uniform Serial
Publication Number (CN). The subject coverage includes mathematics, Physik, Chemie, earth
Wissenschaft, biological science, agricultural science, medicine and health, engineering technology,
environmental science, interdisciplinary disciplines, and some other similar subject areas.
3.1.2.2. Research fields The research fields are mainly derived from the Level 1 Und 2 classes of
the 5th Chinese Library Classification (CLC). Jedoch, the Level 2 classes might be further sub-
divided based on the coupling strength between the citations and semantic similarity of articles
published in the corresponding journal set. In the most recent edition, es gibt 61 research fields.
To avoid the possible bias of subjectively allocating journals to fields, classifications are based on
cross-citation relationships, and any journal can be classified into more than one field.
3.1.2.3. Evaluation indicators To ensure fairness to all candidate journals, journal self-citations
are excluded. The qualitative indicators used to measure different aspects of a journal’s quality
sind in der Tabelle aufgeführt 3.
Tisch 3. Quantitative indicators of CSCD journal list
Indicators
JIF (excluding self-citations)
Definition
Calculated by dividing the number of citations to the journal from other journals in the 6th year
by the number of articles published in that journal in the previous 5 Jahre.
Hot index
Taking 10 years as the statistical window, the hot index is defined as the ratio of the number of
highly cited papers published by the journal to the total number of highly cited papers in the
given discipline the journal belongs to.
Eigenfactor score
Calculated based on the number of times articles from the journal published in the past 5 Jahre
have been cited in the JCR year (exclude self-citations).
Article influence score
Calculated by multiplying the eigenfactor score by 0.01 and dividing by the number of articles in
the given journal in the past 5 Jahre, normalized as a fraction of all articles in all publications.
It is used to track the influence of a journal at the single article level.
Diffusion index
Excellent index
The ratio of the number of citing journals to the total number of journals in the CSCD.
The ratio of the average received citations of a journal to the average received citations of the
given discipline the journal belongs to.
Utilization index
The ratio of the number of noncited articles to the total number of published articles in the journal.
Cross-citation index
The coefficient of skewness of the distribution of the citation journals received from other journals.
The index measures the skewness of the distribution of the journal’s citation data. The larger the
value, the more frequent the cross-citation behavior.
Quantitative Science Studies
306
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
3.2. NSLC: JPT Journal List
3.2.1. Hintergrund
The JPT was built and is maintained by the Centre of Scientometrics, NSLC. The idea behind the
partitioned design of this list began in 2000 with the goal of helping Chinese researchers distin-
guish between the JIFs of journals across different disciplines. The list was first released in 2004 In
Excel format and only included 13 broad research areas. In 2007, these research areas were
expanded to include the JCR categories, Und, seit 2012, the entire list has been published online
to meet the growing number of retrieval requests.
This list provides reference data for administrators and researchers to evaluate the influence of
international academic journals, and is widely recognized by many research institutions as a
metric in their cash reward policies (Quan, Chen, & Shu, 2017).
In 2019, the NSLC released a trial variation of the list while continuing to publish the official
Ausführung. The upgraded JPT (trial) includes 11930 journals that are classified into 18 major
disciplines. The journals cover most of the journals indexed in SCI and SSCI, as well as the
ESCI journals published in China.
3.2.2.
Journal selection criteria
Journals on the list are assessed using a rich array of citation metrics, including 3-year average JIFs.
The list is divided into four partitions according to the 3-year average JIFs by research areas/fields.
Using averages somewhat reduces any instability caused by significant annual fluctuations in
JIFs. The partitions follow a pyramidal distribution. The top partition contains the top 5% of jour-
nals with the highest 3-year average JIFs in their discipline. Partition 2 covers 6%–20% and par-
tition 3 covers 21%–50%, with the remaining journals in the fourth partition. Zusätzlich, all the
journals in the first partition and the top 10% of the journals in the second partition with the
highest total citations (TC) are marked as “Top Journals.”
Im 2019 edition, multidisciplinary journals, such as Nature and Science, were ranked
according to the average impact of each paper in an assigned discipline as determined by the
majority of references given in the paper (Research Services Group at Clarivate, 2019). That said,
the papers in these journals are counted as multidisciplinary, despite the fact that many of them
are highly specialized and represent research in specific fields, such as immunology, Physik,
neuroscience, usw.
Compared to the official version, the trial version has incorporated several essential updates
(Centre of Scientometrics of NSLC, 2020). Erste, the journals are classified based on the average
impact of each paper published in the journal, and the papers are assigned to specific topics
according to both citation relationship and text similarity (Waltman & van Eck, 2012). Zweite,
this version introduces a citation success index (Franceschini, Galetto et al., 2012; Kosmulski,
2011) to replace JIFs as a measure of a journal’s impact. The citation success index of the target
journal compared with the reference journal is defined as the probability of the citation count of a
randomly selected paper from the target journal being larger than that of a random paper from a
reference journal (Milojevic(cid:1), Radicchi, & Bar-Ilan, 2017). Dritte, it extends the coverage of disci-
plines from the natural sciences into the social sciences to support the internationalization pro-
cess of domestic titles. Genauer, coverage is extended to some local journals that are not
listed in the JCR but are listed in the ESCI.
The initial purpose of the list was to evaluate the academic influence of SCIE journals, to pro-
vide academic submission references for scientific researchers, and to support macro analysis
for research management departments. Although the Centre of Scientometrics, NSLC, hat
Quantitative Science Studies
307
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
consistently stated that the list should not be used to make judgments at the micro level (z.B., Zu
evaluate the performance of an individual), many institutions still use the JCR as a tool to evaluate
the research of their employees. The list’s prominent position and strong influence in China’s
scientific research evaluation has caused extensive debate, especially in the field of nuclear phys-
ics in 2018 (Wang, 2018).
3.3. CASSES: AMI Journal List
3.3.1. Hintergrund
The AMI journal list is managed by the Chinese Academy of Social Sciences Evaluation Studies
(CASSES), which was established in July 2017 out of the Centre of Social Sciences Evaluation,
Chinese Academy of Social Sciences (CASS). CASSES has conducted a series of journal evalua-
tion systems for Chinese journals based on the characteristics of disciplines and journals to form a
comprehensive evaluation report of Chinese journals in the SSH. CASSES’ mandate is to optimize
the utilization of scientific research journals and literature resources, as well as to provide refer-
ences for journal evaluation, scientific research performance evaluation, scientific research man-
agement, talent selection, usw. (Ma, 2016). The purpose of AMI is to focus on formative evaluation
“to help and improve” rather than perform a summative evaluation “to judge” a journal’s quality.
Another goal is to increase recognition of journals in the SSH by collaborating nationally across
institutions, rather than competing to support good journals. The basic principle of AMI is to
provide well-informed judgments about journals, not simple indicators, that translates to reliable
advice on where to publish.
CASSES also provides evaluations on both new journals and English-language journals pub-
lished in China to promote their development. New journals are defined as less than 5 Jahre alt.
At present, no other domestic evaluation scheme has undertaken a similar expansion, which turns
out to be one of the innovations of this index.
3.3.2.
Journal selection criteria
The AMI journal list is updated every 4 Jahre, and its comprehensive evaluation method
combines quantitative evaluation with expert qualitative evaluation. According to the latest
report of 2018, 1,291 academic journals in the field of SSH founded in 2012 or before are
published in mainland China, Und 164 new journals and 68 English journals were targets of
particular evaluation. The reports divide the journals into categories: the Top Journals (5),
Authoritative Journals (56), Core Journals (546), Extended Journals (711), and Indexed Journals
(179) (CASSES, 2018). And 26 English journals without CN have not evaluated.
The selection criteria for inclusion in the list are summarized below (CASSES, 2018;
Su, 2019):
Journal scope The journals in the AMI list include some 2000 SSH journals listed by the
3.3.2.1.
former State Administration of Press, Publication, Radio, Film and Television of the People’s
Republic of China in 2014 Und 2017 (SAPPRFT, 2014, 2017). The lists include English-language
journals and new journals that were founded in 2013–2017, and the final scope of journal
evaluation is 1,291 Chinese academic journals, 164 new journals, Und 68 English-language
journals.
3.3.2.2. Research fields The journals are divided into three broad subject categories, 23 Thema
categories, Und 33 subject subcategories based on the university degree and academic training
Quantitative Science Studies
308
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
directory published by the Ministry of Education of the People’s Republic of China, Classification
and the disciplines codes GB/T 13745-2009 in the Chinese Library Classification (fifth edition).
Evaluation indicators There are three evaluation metrics: attraction, management
3.3.2.3.
power, and influence. Attraction gauges the journal’s external environment, its reputation among
readers and researchers, and its ability to acquire external resources. Management power refers
to the ability of the editorial team to promote the journal’s development. Influence represents the
journal’s academic, sozial, and international influence, which is affected by the other two
Befugnisse (attraction and management).
In addition to these three indicators, there are a further 10 second-level indicators and 24 dritte-
level indicators, as shown in Table 4. Looking closely at the list, one can see that most of the
quantitative indicators can be obtained from different data sources (z.B., the journal’s website,
academic news sources, citation platforms). Data to inform the remaining qualitative indicators
is drawn from a broad survey and follow-up interviews. Note that the weights of the first-level
indicators for pure humanities journals (H) versus the social sciences (SS) and multidisciplinary
journals (MJ) are different, as indicated in the table by H/SS/MJ.
3.4.
ISTIC: CJCR Journal List
3.4.1. Hintergrund
As late as 1987, few Chinese knew how many papers were published by Chinese scientists in the
Welt, and no one knew how many papers were published domestically. Infolge, the Institute
of Science and Technical Information of China (ISTIC) was commissioned to conduct a paper
“census.” Thus, initiated by ISTIC and sponsored by the Ministry of Science and Technology (Dann
the State Science and Technology Commission), the China Scientific and Technical Papers and
Citations Database (CSTPCD) was born as a database dedicated to the partial evaluation of the
research performance of China’s scientists and engineers (Wu, Pan et al., 2004).
ISTIC took advantage of the CSTPCD data to conduct statistical analyses on various categories
of China’s scientific output each year. The results were then published in the form of an annual
report and an accompanying press conference to inform society of China’s academic progress.
The report includes the Chinese S&T Papers Statistics and Analysis: Annual Research Report and
the Chinese S&T Journal Citation Reports (Core Edition), which provides a wealth of information
and decision support for government administration departments, colleges and universities,
research institutions, and researchers (ISTIC, 2020A).
3.4.2.
Journal selection criteria
The list of journals selected by CSTPCD is called the “Statistical Source Journal of Chinese
Science and Technology.” These journals are selected from a rigorous peer review and quanti-
tative evaluation, and so are regarded as important scientific and technical journals in various
disciplines in China. Currently, the list includes 2049 journals (1933 Chinese-language journals
Und 116 English-language journals) in the fields of natural sciences, engineering and technology,
Und 395 journals in the social sciences (ISTIC, 2020B). More details on the selection criteria are
provided below (ISTIC, 2020A).
Journal scope The catalog of China’s core S&T journals is adjusted once a year. Der
3.4.2.1.
candidate journals to be evaluated include the core S&T journals selected in the previous year,
along with applications to be considered for the current year that have held a CN number for more
als 2 Jahre. Weiter, the journal’s impact indicators must be ranked at the forefront of their
Quantitative Science Studies
309
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
Tisch 4. Quantitative indicators of AMI journal list
2nd level indicator
Name
Typ
S
3rd level indicator
Awards received by journals and editors
Awards received by publications
1st level indicator
Attraction 0.45/0.35/0.40
Award status
Publication
Peer review
Management 0.20/0.20/0.20
Academic misconductb
Editorial team and author group
Institutional regulation
Informalization construction
Influence 0.35/0.45/0.40
Academic influence
S
S
Ö
N
S
S
S
Funded paper ratioa
Open access
Download frequency
Expert committee members
Suggested experts
Scientific researchers and administrators
Editorial team
Author group
System construction
Editor specification
Website construction
The online manuscript processing system
WeChat public account
Journal immediacy index
2-year JIF
5-year JIF
Reprint ratio
The ratio of cited journals from the given
categories or external categories
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
Social influence
N
Circulation
International influence
N/S
Overseas distribution
Cited by foreign journals
Online propagation capability
Notiz: The indicator type S means measurements add to the total score; O means measurements will reduce the total score; N means measurements do not affect the
score in the current edition.
a This refers to the proportion of papers that are funded by national funds in a journal.
b This indicator is a point deduction indicator. If there is no academic misconduct, the score is “0”; eher, if there is such behavior, points will be deducted.
discipline; should operate in line with national regulations and academic publishing norms; Und
must meet publishing integrity and ethical requirements. If a journal fails to meet these criteria or
its peer assessment, its application is rejected or, if a journal is already listed in the catalog, es ist
withdrawn and can be reevaluated 1 Jahr später.
Quantitative Science Studies
310
A comprehensive analysis of the journal evaluation system in China
3.4.2.2. Research fields The journals are distributed across 112 subject classifications in the
natural sciences and 40 in den Sozialwissenschaften.
3.4.2.3. Evaluation indicators The evaluation system is based on multiple indexes, mostly bib-
liometric, and a combination of quantitative and qualitative methods. Specific indexes include
citation frequency, JIF, important database collection, and overall evaluation score (Ma, 2019).
3.5. Peking University Library: GCJC Journal List
3.5.1. Hintergrund
The GCJC is a research project conducted by researchers at the Peking University Library in con-
junction with a dozen other university libraries and experts from other institutions. The guide is
regularly updated to reflect the changing dynamics of journal development and has been pub-
lished every 4 years since 1992 and every 3 years since 2011. It is only published in a printed
Buch, and to date eight editions have been published by Peking University Press.
Whether and how the guide is used is up to the institutions that make use of it. It is worth noting
that the guide is not an evaluation criterion for academic research and has no legal or adminis-
trative effectiveness, but some institutions do use it this way, which can create conflict. Der
selection principles emphasize that core journals are a relative concept to specific disciplines
and periods. Größtenteils, the guide is used by library intelligence departments as an infor-
mational reference to purchase and reserve books, and to help tutors develop reading lists
(Committee for A Guide to the Core Journals of China, 2018).
3.5.2.
Journal selection criteria
Der 2017 edition of GCJC contains 1983 core journals assigned to seven categories and 78
disciplines. The selection criteria are provided below (Chen, Zhu et al., 2018).
3.5.2.1.
Journal scope Any Chinese journal published in mainland China can be a candidate.
3.5.2.2. Research fields Fields are based on the CLC categories of Philosophy, Sociology, Politik,
and Law (Teil I); Economy (Teil 2); Culture, Education and History (Teil 3); Natural Science (Teil 4);
Medicine and Health (Teil 5); Agricultural Science (Teil 6); Industrial Technology (Teil 7).
3.5.2.3. Evaluation indicators Comprehensive quantitative and qualitative analysis of 16 evalu-
ation indicators, combined with the opinions of experts and scholars, are the basis of selection,
as shown in Table 5.
Tisch 5. Quantitative indicators of GCJC journal list
ID
1
2
3
4
Indicators
Reprint count
(full-text, abstract)
Reprint ratio
(full-text, abstract)
Times cited
Times cited
ID
5
6
7
8
Indicators
2-year JIF
ID
9
Indicators
Eigenfactor index
ID
13 Awards or indexed by
Indicators
important retrieval systems
2-year JIF
10 Article influence scores
14
Funded paper ratio (National
(exclude self-citations)
and provincial level)
5-year JIF
5-year JIF
11
12
Paper cited index
15 Web downloads amount
Inner-citation index
16 Web downloads ratio
(exclude self-citations)
(exclude self-citations)
Quantitative Science Studies
311
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
3.6.
ICSSRA: CSSCI Journal List
3.6.1. Hintergrund
CSSCI was developed by Nanjing University in 1997 and launched in 2000. CSSCI collects all
source and citation information from all papers in source journals and source collections
(published as one volume). The index records in CSSCI contain most of the bibliographic
information in the papers. The content is normative, and the reference data are searchable
(Qiu & Lou, 2014).
The focus is on the social sciences in China and is gathered for the purposes of providing an
efficient repository of information about Chinese knowledge innovation and cutting-edge
research in the SSH, coupled with a comprehensive evaluation of China’s academic influence
in these areas (Su, Deng, & Shen, 2012; Su, Han, & Han, 2001). The journal data in CSSCI
provides a wealth of raw information and statistics for researchers and institutions to study or
to conduct evaluations based on authentic records of research output and citations.
3.6.2.
Journal selection criteria
Through quantitative and qualitative evaluation methods, the 2019–2020 edition of CSSCI
contains 568 core source journals and 214 extended source journals assigned among 25 disci-
plines (ICSSRA, 2019). Extended source journals are evaluated and those that qualify are trans-
ferred to the core source journal list. The selection criteria are summarized below (CSSCI
Editorial Department, 2018; ICSSRA, 2019).
Journal scope In the latest edition of CSSCI (2019–2020), all selected journals/collections
3.6.2.1.
must meet the following basic requirements (CSSCI Editorial Department, 2018):
(cid:129) The journals must be Chinese and publish mainly original academic articles and reviews
in den Sozialwissenschaften
(cid:129) Journals published in mainland China must have a CN number. Journals published in
Hongkong, Macao, or Taiwan must have an ISSN, and academic collections must have
an ISBN.
(cid:129) Journals must be published regularly according to an established publishing cycle and
must conform to the standards of journal editing and publication with complete and
standardized bibliographic information.
3.6.2.2. Research fields Each article in the CSSCI database is categorized according to the
Classification and Code of Discipline (GBT 13745-2009) with reference to the Catalogue of
Degree Awarding and Personnel Training (2011) (Grad [2011] NEIN. 11) and the Subject
Classification Catalogue of National Social Science Foundation in China. At present, es gibt
23 journal categories based on subject classification, and two general journal categories: multi-
disciplinary university journals and multidisciplinary social science journals.
3.6.2.3. Evaluation indicators The source journals of CSSCI are determined according to their
2-year JIF (excluding self-citations), total times cited, other quantitative indicators, und das
opinions of experts from various disciplines.
3.7. CNKI: WAJCI Journal List
3.7.1. Hintergrund
The China National Knowledge Infrastructure (CNKI) is the largest comprehensive database in
China. It is a key national information project led by Tsinghua University, first launched in
Quantitative Science Studies
312
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
1996 in conjunction with the Tsinghua Tongfang Company. In 1999, CNKI began to develop
online databases. In October 2009, it unveiled the construction of an international digital library
together with world-famous foreign partners. At present, CNKI contains literature published since
1915 in over 7,000 academic journals published in China, including nearly 2,700 core and other
significant journals. The database is divided into 10 Serie, 168 Fächer, Und 3,600 subsubjects
(CNKI, 2020). When a Chinese scholar wants to read a paper, they typically go to CNKI as the first
port of call.
Seit 2009, CNKI has invested and managed the “International and Domestic Influence of
Chinese Academic Journals Statistics and Analysis Database.” This database publishes interna-
tional and domestic evaluation indicators for nearly 6,000 academic journals officially published
in China across four reports: the “Annual Report for Chinese Academic Journal Impact Factors,”
the “Annual Report for International Citation of Chinese Academic Journals,” the “Journal
Influence Statistical and Analysis Database,” and the “Statistical Report for Journal Network
Dissemination” (CNKI, 2018B).
Seit 2018, CNKI has also released the “Annual Report for World Academic Journal Impact
Index (WAJCI).” This report aims to explore a scientific and comprehensive method for evaluating
the academic influence of journals and provides objective statistics and comprehensive ranking for
the academic impact of the world’s journals. This idea is not only conducive to building an open,
diversified, and fairer evaluation system for journals; it is also helpful for improving the representa-
tion of Chinese journals in Western-dominated international indexes (CNKI, 2018A).
3.7.2.
Journal selection criteria
The WAJCI journal list is updated annually; the most recent report was released in October 2019.
The statistics shown in this report were derived from 22,325 source journals from 113 Länder
and regions (21,165 from the WoS database, einschließlich 9,211 from SCIE, 3,409 from SSCI, 7,814
from ESCI, Und 1,827 from A&HCI, Plus 1,160 Chinese journals). The WoS database does not
provide JCR evaluation reports for some journals. In the case of new journals, this is because
the citation frequency is typically very low. Excluding these journals without a JCR report leaves
13,088 journals to be evaluated, umfassend 1,429 journals from mainland China and 11,659
from other countries and regions. Of these, 486 journals are in the field of SSH, 957 journals
are in the field of science, Technologie, engineering and medicine (STEM), Und 14 journals are
interdisciplinary (CNKI, 2019).
The selection criteria follow.
Journal scope The target journals are academic journals with wide dissemination and
3.7.2.1.
significant influence around the world. Candidate journals must meet four basic requirements:
(cid:129) Journals should be published continuously and publicly.
(cid:129) Journals must predominantly publish original academic achievements, which should be
peer reviewed.
(cid:129) Journals should comply with the requirements of international publishing and profes-
sional ethics.
(cid:129) Papers published in journals must conform to international editorial standards, welche
include editorial and publishing teams of high standing in their disciplines and a high
level of originality, scientific rigor, and excellent readability.
3.7.2.2. Research fields CNKI mainly follows a hybrid of the JCR classification system, Die
International Classification for Standards (ICS), and the CLC. Chinese journals that cannot be
Quantitative Science Studies
313
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
found in JCR are categorized into disciplines in one of the other lists. Darüber hinaus, the disci-
plines of JCR journals with serious duplication are appropriately merged. The final list spans
175 STEM disciplines and 62 SSH disciplines. Letzten Endes, alle 13,088 journals are assigned
into relatively accurate disciplines to ensure that the journals are ranked and compared within
a unified discipline system.
3.7.2.3. Evaluation indicators To comprehensively assess the international influence of journals,
CNKI developed a complex indicator, called the CI index (clout index), that combines JIF with
citation counts (Wu, Xiao et al., 2015). It is generally believed that the most influential journal in
a field should be the journal with the highest JIF and TC. The meaning of the CI value represents
the degree of closeness between the journal influence and the optimal state of journal influence
in the field. The smaller the gap, the closer the distance, which indicates that the influence of
journals is closer to the optimal state. Außerdem, to compare journals on an international
scale, CNKI publishes the indicators in WAJCI. The higher the value of the WAJCI, the higher
the journal’s influence. WAJCI reflects the relative position of academic influence of journals
within a discipline, so it can be used for interdisciplinary and even cross-year comparison,
which has practical value.
3.8. Other Lists
In addition to the above seven main journal lists in China, another influential list, called the
Research Center for Chinese Science Evaluation (RCCSE) core journal list, was developed by
the RCCSE of Wuhan University in the early 2000s to provide multidisciplinary and comprehen-
sive Chinese journal rankings (Zhang & Lun, 2019). The evaluated journals mainly include pure
academic journals and semiacademic journals from the natural sciences or humanities and social
Wissenschaften (Qiu, Li, & Shu, 2009). It adopts a mixture of quantitative and qualitative methods to
evaluate the target journals and mainly concerns the quality, Ebene, and academic influence of
das Tagebuch (Qiu, 2011). In the process of journal evaluation, the general principles are classified
evaluation and hierarchical management (Qiu, 2011). Zusätzlich, there are some other lists pub-
lished by government agencies, professional associations, and universities that warrant mention,
and they are briefly described below.
3.8.1. CDGDC: A-class journal list
In 2016, the fourth China University Subject Rankings (CUSR) was launched to evaluate the sub-
jects of universities and colleges in mainland China in line with the “Subject Catalogue of Degree
Awarding and Personnel Training” approved by the Ministry of Education. Organized by the
China Academic Degrees and Graduate Education Development Centre (CDGDC), the aim
was to acquaint participating universities and institutions with the merits and demerits of their
subject constructions and curriculums, and to provide relevant information on national graduate
Ausbildung (CDGDC, 2016). The instructions of the fourth CUSR specifically point out that the
number of papers published in A-class journals (both Chinese and international) is a critical
indicator of the quality of a subject offering (Ministry of Education, 2016A).
As described by the Ministry of Education (Ministry of Education, 2016A, 2016B), the process
for selecting which journals were “A-class” was as follows. Erste, the publishers and bibliometric
data providers (z.B., Thomson Reuters, Sonst, CNKI, CSSCI, CSCD) were invited to provide a
preliminary list of journals based on bibliometric indicators, such as JIF and reputation indices.
Dann, doctoral tutors were invited to participate in online voting for the candidates. Last, Die
voting results were submitted to the Academic Degrees Committee of the State Council for
Rezension, who finalized the journal list. The A-class journal list exercise was an attempt to combine
Quantitative Science Studies
314
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
bibliometric indicators and expert opinions. Jedoch, the list was abandoned only 2 weeks after
release as a fiery debate erupted among many scientific communities.
3.8.2. Chinese Computing Federation: CCF-recommended journal list
The China Computer Federation (CCF) is a national academic association in China, established
In 1956. Their publication ranking list, released in 2012, divides well-known international com-
puter science conferences and journals into 10 subfields. A rank of A indicates the top confer-
ences and journals, B is for journals and conferences with significant impact, and important
conferences and journals are placed in the C bracket.
Im April 2019, the CCF released the 5th edition List of International Academic Conferences and
Periodicals Recommended by the CCF. In the course of the review, the CCF Committee on
Academic Affairs brought experts together to thoroughly discuss and analyze these suggestions.
The candidates were reviewed and shortlisted by an initial assessment panel, then examined by a
final evaluation panel before announcing the final results. Factors such as the venue’s influence
and an approximate balance between different fields were considered when compiling the list
(China Computer Federation, 2019). Heute, this list is widely recognized in computer science
fields and has accelerated the process of publishing more papers in top conferences, sowie
improving the quality of those publications (Li, Rong et al., 2018).
3.8.3. School or departmental journal lists
With the rapidly increasing and burdensome number of scholarly outlets for academic assessment,
administrators and research managers are constantly looking to improve the speed and efficiency
of their assessment processes. Many construct their own school or departmental list as a guide for
evaluating faculty research (Beets, Kelton, & Lewis, 2015). Business schools particularly prefer
internal journal lists to inform their promotion and tenure decisions (Bales, Hubbard et al.,
2019). Tatsächlich, almost all of the 137 Chinese universities that receive government funding have
created their own internal journal lists as indicators of faculty performance (Li, Lu et al., 2019).
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
4. COMPARATIVE ANALYSIS OF JOURNAL EVALUATION SYSTEMS IN CHINA
What is clear from the descriptions of each of the major journal lists is that each was established to
fill specific objectives, and each has its own selection criteria, yet there may be as many similar-
ities between the seven systems as there are differences. daher, for a broader picture of the
evaluation system landscape, we undertook a comprehensive comparative analysis. Our findings
are presented in this section.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
4.1. Profiles of Journal List and Indexed Journals
CJCR was first established by ISTIC in 1987. GCJC, CSCD, CSSCI, and JPT followed shortly after.
AMI and WAJCI joined the club more recently. As indicated in Table 6, studies on journal selec-
tion have included a wide variety of participants, such as research institutes, universities, Und
private enterprises. Another observation is that the regularity with which journal lists are updated
is not the same. Currently, JPT, CJCR, and WAJCI are updated once a year; CSCD and CSSCI are
updated every 2 Jahre; GCJC is updated every 3 Jahre, and AMI every 4 Jahre.
Clearly, the number of journals, scope, languages, and research areas of each journal list are
anders. JPT and WAJCI count the most journals, both of which have a domestic and international
scope. All of the other lists only cover domestic journals, obviously making them smaller than the
previous two. Although most of the journal lists include English-language journals, these are few in
Quantitative Science Studies
315
Q
u
A
N
T
ich
T
A
ich
T
ich
v
e
S
C
e
N
C
e
S
u
D
e
S
T
ich
Tisch 6.
Profiles of the leading academic journal lists in China
Indexed journals
Journal list
CSCD
Provider
Number of
journals
Scopes
Language
Research areas
NSLC
1,229 (2019) Domestic
Chinese, English LB; PS; DER; SS
Identification
Zahlen
ISSN and CN
Release
Jahr
1995
Update frequency
Biennially
(partly)
JPT
NSLC
11,930 (2019) Domestic &
English
LB; PS; SS; DER
ISSN
2004
Yearly
International
AMI
CJCR
GCJC
CASSES
ISTIC
1,523 (2018) Domestic
Chinese, English AH; SS
CN
2014
Quadrennial
2,444 (2019) Domestic
Chinese, English AH; LB; PS; SS; TE CN
1997
Yearly
Peking University
1,983 (2017) Domestic
Chinese
AH; LB; PS; SS; TE CN
1992
Library
CSSCI
ICSSRA, Nanjing
Universität
782 (2019) Domestic
Chinese
AH; SS
ISSN or CN
1998
Triennial (2008–);
Quadrennial
(Vor 2008)
Biennially (2003–)
Yearly (Vor 2003)
WAJCI
CNKI
13,088 (2018) Domestic &
Chinese, English AH; LB; PS; SS; DER
ISSN or CN
2018
Yearly
International
Notiz: The research areas are classified into five broad categories: Arts & Humanities (AH); Life Sciences & Biomedicine (LB); Physical Sciences (PS); Social Sciences (SS); Technologie
Maschinenbau (DER).
3
1
6
A
C
Ö
M
P
R
e
H
e
N
S
ich
v
e
A
N
A
l
j
S
ich
S
Ö
F
T
H
e
J
Ö
u
R
N
A
l
e
v
A
l
u
A
T
ich
Ö
N
S
j
S
T
e
M
ich
N
C
H
ich
N
A
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
China. In terms of disciplines, JPT and CSCD focus more on the natural sciences; AMI and CSSCI
focus on the SSH field; and CJCR, GCJC, and WAJCI span all disciplines.
There are also different requirements for ISSNs and CNs. AMI, CJCR, and GCJC only accept
journals with CNs, and the JPT only accepts journals with an ISSN. CSSCI and WAJCI have the
least stringent requirements, requiring that the journal has one or the other. Im Gegensatz, CSCD
requires its journals to have both.
4.2. The Evaluation Characteristics of Journal lists
How journals are assessed is the most critical aspect of any evaluation system. Weiter, as indicated
in Table 7 und Tisch 8, these systems were designed with many different objectives in mind.
Although often used in scientific evaluation, the purpose of most is to provide readers, librarians,
and information agencies with reference material to help them purchase and manage journal
stocks. This is certainly the case with JPT, CJCR, GCJC, CSSCI. CSCD, CJCR, and CSSCI extend this
mission further by seeking to provide references for research management and academic evalua-
tion. Jedoch, the objectives of AMI and WAJCI are different. AMI’s goal is to increase the quality
and recognition of journals in the SSH, while CNKI built WAJCI to provide “apples with apples”
statistics on the world’s journals.
The methods of calculation and indicators that each system uses are different. Most rely on a
combination of quantitative and qualitative tools, while JPT and WAJCI are largely quantitative
Systeme. Both are heavily dependent on JIFs, but JPT relies on a 3-year average, while WAJCI
combines JIFs with TC to create its indicator. The other lists mainly evaluate the attraction and
management ability of journals through bibliometric indicators, such as JIFs and citations, sup-
plemented by peer review. AMI, Jedoch, adds extra indicators over and above the standard set.
Darüber hinaus, the weight of each indicator changes depending on the purposes of lists. For exam-
Bitte, AMI’s mission is to improve the quality and influence of both China’s SSH journals and the
evaluation systems that rank them. daher, AMI houses a comprehensive set of indicators that
cover processes, talent, management, the editorial team, usw., each of which is measured against
the three “powers” (d.h., attraction, management, and influence). Im Gegensatz, the fundamental
purpose of GCJC is to help librarians optimize their journal collections and provide readers with
guidance on the titles in their disciplines. Somit, GCJC rests more on bibliometric indicators and
quantitative analyses of the growth trends and scatter law patterns of journals in a field.
Data sources of indicators are another characteristic for comparison. JPT is mainly an interna-
tional database. WAJCI and GCJC combine international databases with local Chinese databases
to expand the type and volume of data provided. Although the indicators data of AMI draws from
a wide range of sources, such as the self-built and self-collected data of CASSES (z.B., CHSSCD),
the third-party data, the self-evaluated data of the journal editorial departments, and the data
included are mostly determined by the producers of the original indexes. The same is true for
CSCD, CSSCI, and GCJC.
The last criterion for comparison is the grading system. All divide their listed journals into
disciplines, and most calculate their rankings relative to that discipline. JPT and WAJCI each have
four tiers, but the JPT system is a pyramid, whereas the WAJCI scheme is equally divided, Die
same as JCR. AMI’s system is more complicated because the journals are divided into three
categories (A-journals, new journals and English-language journals), then further subdivided into
five levels according to quality. CSCD and CSSCI are divided into two levels—core and extended
journals—and CJCR and GCJC do not have grades. Bis zu einem gewissen Grad, these divisions are hierar-
chical and systematic, which is convenient for users. Jedoch, how many journals appear in
Quantitative Science Studies
317
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
Q
u
A
N
T
ich
T
A
ich
T
ich
v
e
S
C
e
N
C
e
S
u
D
e
S
T
ich
Tisch 7.
Evaluation purposes, Methoden, and results of the leading academic journal lists in China
Journal list
CSCD
Evaluation purposes
Provide a basis for selecting source journals
for CSCD and journal evaluation
Evaluation methods
Qualitative &
quantitative
JPT
Help Chinese researchers to distinguish the
JIF of journals across different disciplines
Quantitative
Evaluation results
Disciplines
Grades
61 discipline categories
Core source journals;
18 major disciplines
Und 229 minor
disciplines
Three broad subject categories,
23 subject categories, Und
33 subject subcategories
112 subject classifications in
the natural sciences and
40 in den Sozialwissenschaften
Extended source journals
T1 (top 5%); T2 (6%–20%);
T3 (21%–50%);
T4 (51%–100%)
Top journals; Authoritative
journals; Core journals;
Extended journals;
Indexed journals
Natural sciences volume;
Social sciences volume
Qualitative &
quantitative
Qualitative &
quantitative
Qualitative &
quantitative
Seven categories and
78 discipline categories
Core journals
Qualitative &
quantitative
25 disciplines
Core source journals;
Extended source journals
Quantitative
237 Fächer, among which
175 are science and
technology and 62 Sind
social sciences
Q1 (top 25%); Q2 (26%–50%);
Q3 (51%–75%);
Q4 (76%–100%)
A
C
Ö
M
P
R
e
H
e
N
S
ich
v
e
A
N
A
l
j
S
ich
S
Ö
F
T
H
e
J
Ö
u
R
N
A
l
e
v
A
l
u
A
T
ich
Ö
N
S
j
S
T
e
M
ich
N
C
H
ich
N
A
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
AMI
Help and improve journal quality, Und
increase the recognition of journals in
the SSH
CJCR
Be used for quantitative analysis and
GCJC
scientific evaluation of the characteristics
and discipline status of journals, Und
providing a basis for scientific research
management
Help library intelligence departments to
purchase and reserve books as an
informational reference, and help tutors
develop reading lists
CSSCI
Provide a reference for evaluation,
WAJCI
performance appraisal, management,
and research
To explore scientific and comprehensive
evaluation methods for measuring the
influence of journals, and to provide
objective statistics and comprehensive
rankings of that influence
3
1
8
Q
u
A
N
T
ich
T
A
ich
T
ich
v
e
S
C
e
N
C
e
S
u
D
e
S
T
ich
A
C
Ö
M
P
R
e
H
e
N
S
ich
v
e
A
N
A
l
j
S
ich
S
Ö
F
T
H
e
J
Ö
u
R
N
A
l
e
v
A
l
u
A
T
ich
Ö
N
S
j
S
T
e
M
ich
N
C
H
ich
N
A
Tisch 8.
Evaluation criteria, indicators, data sources of the leading academic journal lists in China
Journal list
CSCD
Selection criteria
Editorial standardization, focus on basic research,
academic and theoretical frontier journals, usw.
Evaluation indicators
Indicators data sources
JIF (excluding self-citations), Hot index,
CSCD
Eigenfactor score, Article Influence score,
Diffusion index, Excellent index,
Utilization index, Cross-citation index
JPT
AMI
the same as JCR
3-year average impact factor
WoS
Journal influence, academic quality, Bearbeitung
specifications, ethical norms, usw.
Attraction power; Management and influence
(award status, Veröffentlichung, peer review,
academic misconduct, editor team and author
Gruppe, institutional regulation, informalization
construction, academic influence, sozial
influence, international influence)
the self-built and self-collected
data of CASSES (z.B., CHSSCD),
the third-party data, und das
self-evaluated data of the
journal editorial department
CJCR
Editorial norms, authoritative editorial board,
Citation frequency, JIF, important database
CSTPCD
timeliness, the collection of important retrieval
systems at home and abroad, the social and
academic influence of journals, the balance
of regions and disciplines, usw.
GCJC
Based on the quantitative selection of the core
list, the use of the middle-map method to
divide the size of the subject; journals are
representative and practical
Sammlung, overall evaluation score
Reprint count (full-text, abstract), 2-year JIF,
Eigenfactor index, Awards or indexed by
important retrieval systems, Reprint ratio
(full-text, abstract), 2-year JIF (excluding
self-citations), Article influence scores, Funded
paper ratio (national and provincial level),
Times cited, 5-year JIF, Paper cited index,
Web downloads amount, Times cited
(excluding self-citations), 5-year JIF
(excluding self-citations), Inner-citation index,
Web downloads ratio
CSCD, CSTPCD, CSSCI, Wanfang
Data Knowledge Service Platform
( Wanfang), China Science and
Technology Journal Database
( VIP), Scopus, Social Sciences
Citation Index (SSCI), usw.
CSSCI
Journal quality, regional and discipline balance,
editorial norms, timeliness, original, innovative
academic literature, usw.
2-year JIF (exclude self-citations), Total times
The self-built and self-collected
cited, usw.
Daten (z.B., CSSCI)
WAJCI
Continuous publication, journal attributes,
Clout index (CI); WAJCI index (both are based
WoS, JCR report, CNKI
ethical norms, editorial norms, Veröffentlichung
Stärke, content quality, usw.
on the JIF and total times cited [TC])
Notiz: Partly refer to Ma (2019).
3
1
9
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
more than one index and how similar their rankings are across indexes needs further analysis
und Diskussion.
5. CONCLUSIONS AND DISCUSSION
There is no doubt that China’s journal evaluation and selection systems have achieved remark-
able growth and impact, resulting in some influential journal lists. Im 60 years since journal
evaluation was first introduced to China, the functions of these lists have grown, diversified, Und
generated their fair share of controversy. Some lists simply seek to provide decision-making
support for information consultants, journal managers, research managers and funders, editors,
and others. Others are designed to help optimize library collections, provide reading-list
guidance or support referencing and citation services. Weiter, and more controversially, journal
evaluations are increasingly becoming proxies for evaluating the academic achievements of
individual researchers. As an extension of the original purposes, the evaluation of core journals
has an important influence on a journal’s editorial procedures and strategies. To maintain the
continuous development of their academic journals, publishers and publishing houses must
conduct journal evaluations as well as supervision (Ren & Rousseau, 2004).
5.1. Greater Cooperation Among the Different Journal List Providers
Seven different journal lists is a lot, even for a country as large and diverse as China. Jedoch,
what is more notable is the number of different institutions that contribute to informing these lists,
and the fact that individual universities still feel the need to create their own internal lists to
complement the published systems. Everyone in this landscape is gathering their data, and most
are constructing their own data sets, classifying and ranking the journals and papers, calculating
their own rankings and metrics, usw. The result is simply an overlap of effort in many cases. Wir
know that if institutions want to build an influential and authoritative journal evaluation or selec-
tion system, it not only needs to be based on sound indicators but also a on comprehensive range
of triangulated data sources. An obvious solution is for the producers of these evaluation instru-
ments to collaborate on research and development. They could build a national platform for
coordination, influence, and collaboration on developing shared information resources and tools
and agreed definitions and protocols (Zhang & Sivertsen, 2020). Cooperation would be condu-
cive to establishing a unified and authoritative journal evaluation and selection system and, mehr
importantly, it could significantly increase the objectivity and fairness of the results.
5.2. More Compatibility Between Subject Classifications
Our analysis shows that each scheme adopts a different subject classification system. Jedoch,
many articles are interdisciplinary and, because papers are assessed relative to their discipline, A
publication can be evaluated with very different results in each category. daher, Wann
evaluating and selecting journals, institutions should pay attention to the subject classification
of journals to ensure the relative accuracy of the grades.
5.3. Exercise Caution when Using Journal Evaluations for Scientific Research Assessment
and Management
Although the practice of China’s journal evaluation and selection systems is scientific and some-
what accurate, it is worth noting that journal rankings (such as JIFs) are not suitable for assessing
the quality of individual research. The phenomenon of emphasizing JIFs and rankings in re-
search evaluation is prevalent and persistent in China, but at least awareness of its adverse effects
is growing (Ministry of Education, 2020). Journal evaluation systems can make a strong
Quantitative Science Studies
320
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
contribution to research evaluation at the macro level, but applying those rankings as measures
of impact and quality at the micro level (to individuals, institutions, subjects and the like) should
be done with extreme caution. We should focus on the macro information about whether a given
journal has been indexed in such systems, such as CSCD and CSSCI. A good example of wise
macrolevel use of these evaluation systems can be found in the National Science Fund for
Distinguished Young Scholars. At one time, the bibliometric data indexed in the CSCD was con-
sidered in the Fund’s document preparation and subsequent peer review.
Darüber hinaus, and in line with the new research evaluation policy in China as of 2020, the use
of information such as journal quartiles and JIFs for microlevel evaluations should be reduced.
Individual institutions need to establish their own guidelines on how to use journal ranking
lists in their decisions (Black et al., 2017), but when journal rankings are used they should
be combined with other indicators. Research managers are also beginning to notice that there
is no direct link between the influence of a journal and a single paper published in it. The use
of JIFs for measuring the performance of individual researchers and their publications is highly
contested and has been demonstrated to be based on wrong assumptions (Zhang et al., 2017).
Entsprechend, there have been some recent efforts to improve research evaluation at the ar-
ticle level as opposed to the level of journal, such as the F5000 project from ISTIC (http://
f5000.istic.ac.cn/). In diesem Projekt, 5,000 outstanding articles from the top journals are selected
each year to showcase the quality of Chinese STM journals (Ma, 2019). The excellent articles
project by the CAST (http://www.cast.org.cn/art/2020/4/30/art_458_120103.html) is another
similar initiative. Reform will not be achieved overnight; that is long-term and arduous work.
Jedoch, the many steps that need to be taken to get there begin with collaborative efforts
such as these.
5.4. Collaborate with the International Bibliographic Databases
At present, there is no interconnection between the international evaluation systems and China’s,
especially in the SSH. Journal list producers should try to cooperate with the international biblio-
graphic databases to promote the internationalization of China’s journal evaluation systems. Für
Beispiel, linkages between CSCD and SCI over citation data have been established, and other
joint systems, such as CSSCI and SSCI, might be promoted in the future. Jedoch, we should also
realize that SSCI and A&HCI only partly represent the world’s scholarly publishing in the SSH
(Aksnes & Sivertsen, 2019). daher, China’s journal evaluation institutions should try to coop-
erate with international evaluation systems on the basis of improving their own systems as much
as possible.
5.5. Accelerate the Establishment of an Authoritative Evaluation System
At present, there are seven main predominant journal lists in China, each established with its own
evaluation objectives. This is a dispersed system, but not an integrated system. Although diversity
allowed for the exploration of evaluation methods and data sources, there is no unified, author-
itative standard. With the new research evaluation policy as of 2020, China is moving away from
indicators based on the WoS as standard. This should empower China’s research institutions and
funding organizations to define new standards, but this is a process that needs to be coordinated
(Zhang & Sivertsen, 2020). We contend that one comprehensive journal list, both domestic and
International, should be created that reflects the full continuum of research fields, einschließlich
interdisciplinary and marginalized fields. The list needs to be dynamic to reflect the changing
journal market, and the evaluations need to be organized, balanced, and representative of a range
of interinstitutional expert advice. A national evaluation system would not only conserve
Quantitative Science Studies
321
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
resources but also increase the credibility and authority of the core journal list. Als Beispiel,
South Korea has only one system managed by the National Research Foundation of Korea. Der
same is true of several European, African, and Latin-American countries. Also, while a national
system is not a new idea, it is perhaps a proposal that deserves to be reconsidered for the scientific
community in China.
ACKNOWLEDGMENTS
We would like to thank Professor Xiaomin LIU (National Science Library, Chinese Academy of
Wissenschaften), Professor Liying YANG (National Science Library, Chinesische Akademie der Wissenschaft),
Professor Jinyan SU (Chinese Academy of Social Sciences Evaluation Studies), and Professor
Jianhua LIU ( Wanfang Data Co., LTD) for providing valuable data and materials. We also
thank Ronald Rousseau (KU Leuven & University of Antwerp) and Tim Engels (Universität
Antwerp) for providing insightful comments.
BEITRÄGE DES AUTORS
Ying Huang: Akquise von Fördermitteln, Formale Analyse, Methodik, Ressourcen, Aufsicht, Schreiben-
original draft, Writing—review & Bearbeitung. Ruinan Li: Formale Analyse, Untersuchung, Writing—original
Entwurf, Writing—review & Bearbeitung. Lin Zhang: Akquise von Fördermitteln, Untersuchung, Ressourcen,
Aufsicht, Writing—review & Bearbeitung. Gunnar Sivertsen: Akquise von Fördermitteln, Methodik,
Writing—review & Bearbeitung.
COMPETING INTERESTS
The authors have no competing interests.
FUNDING INFORMATION
This work is supported by the National Natural Science Foundation of China (Grant No. 72004169;
71974150; 71904096; 71573085), the Research Council of Norway (Grant No. 256223), und das
MOE (Ministry of Education in China) project of humanities and social sciences (18YJC630066),
and the National Laboratory Center for Library and Information Science in Wuhan University.
DATA AVAILABILITY
The raw bibliometric data were collected from Clarivate Analytics. A license is required to
access the Web of Science database. daher, the data used in this paper cannot be posted
in a repository.
VERWEISE
Aksnes, D. W., & Sivertsen, G. (2019). A criteria-based assessment
of the coverage of Scopus and Web of Science. Journal of Data
and Information Science, 4(1), 1. DOI: https://doi.org/10.2478/jdis
-2019-0001
Bales, S., Hubbard, D. E., vanDuinkerken, W., Sare, L., & Olivarez, J.
(2019). The use of departmental journal lists in promotion and
tenure decisions at American research universities. Zeitschrift für
Academic Librarianship, 45(2), 153–161. DOI: https://doi.org/10
.1016/j.acalib.2019.02.005
Beets, S. D., Kelton, A. S., & Lewis, B. R. (2015). An assessment of
accounting journal quality based on departmental lists. Sciento-
metrics, 102(1), 315–332. DOI: https://doi.org/10.1007/s11192-014
-1353-0
Black, E. L., Stainbank, L., Elnathan, D., Giner, B., Gray, S. J., …
Holz, D. A. (2017). Usage of journal rankings: an international
Perspektive. Journal of International Accounting Research, 16(3),
1-15. DOI: https://doi.org/10.2308/jiar-10571
Bradford, S. C. (1934). Sources of information on specific subjects.
Maschinenbau, 137, 85–86.
Bradford, S. C. (1984). Documentation. London: Crosby Lockwood.
CASSES. (2018). Chinese Humanities and Social Sciences Journal
AMI Comprehensive Evaluation Report (2018). Retrieved from
http://ex.cssn.cn/xspj/xspj_yw/201811/t20181119_4777954
.shtml (in Chinese).
CDGDC. (2016). The evaluation result for college and universities
of the 4th China University Subject Rankings (CUSR). Retrieved
Quantitative Science Studies
322
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
from http://www.chinadegrees.cn/xwyyjsjyxx/xkpgjg/2016phden
/index.shtml
Centre of Scientometrics of NSLC. (2020). The trial version of the par-
tition journal list in 2019: Improvements and advantages. Retrieved
from https://mp.weixin.qq.com/s/ WHbd7VDt4ucO_moSvkcOyg
(in Chinese).
Chen, J., Zhu, Q., Zhang, J., & Cai, R. (2018). A guide to the core
journals of China (2017 Edition). Peking: Peking University Press
(in Chinese).
China Computer Federation. (2019). The latest edition of the “list of
international academic conferences and periodicals recommended
by CCF” was officially released. Retrieved from https://www.ccf
.org.cn/Focus/2019-04-25/663625.shtml
CNKI. (2018A). Annual report for World Academic Journal Impact
Index. Retrieved from https://cjcr.cnki.net (in Chinese).
CNKI. (2018B). Product introduction. Retrieved from https://cjcr
.cnki.net (in Chinese).
CNKI. (2019). Annual report for World Academic Journal Impact
Index. Retrieved from https://cjcr.cnki.net (in Chinese).
CNKI. (2020). Database introduction. Retrieved from http://neu
.oversea.cnki.net/kns/brief/result.aspx?dbPrefix=CJFQ
Committee for a Guide to the Core Journals of China. (2018).
Research project on core journals. Retrieved from https://www
.lib.pku.edu.cn/portal/en/bggk/qikanyaomu
CSSCI Editorial Department. (2018). The selection method of source
journals (collections) of the “Chinese Social Science Citation Index
(CSSCI)” (Trial). Retrieved from https://cssrac.nju.edu.cn/a/gywm
/lxbz/20200102/i64328.html (in Chinese).
Dobson, ICH. R. (2014). Using data and experts to make the wrong de-
Entscheidung: The rise and fall of journal ranking in Australia. In Using
data to improve higher education (S. 229–242). Leiden: Brill |
Sense. DOI: https://doi.org/10.1007/978-94-6209-794-0_14
Franceschini, F., Galetto, M., Maisano, D., & Mastrogiacomo, L.
(2012). The success-index: An alternative approach to the h-index
for evaluating an individual’s research output. Scientometrics,
92(3), 621–641. DOI: https://doi.org/10.1007/s11192-011
-0570-z
Garfield, E. (1955). Citation indexes for science: A new dimension in
documentation through association of ideas. Wissenschaft, 122(3159),
108–111. DOI: https://doi.org/10.1126/science.122.3159.108,
PMID: 14385826
Garfield, E. (1963). Citation indexes in sociological and historical
Forschung. American Documentation, 14(4), 289–291. DOI: https://
doi.org/10.1002/asi.5090140405
Garfield, E. (1964). “Science Citation Index”—a new dimension in
indexing. Wissenschaft, 144(3619), 649–654. DOI: https://doi.org/10.1126
/science.144.3619.649, PMID: 17806988
Brutto, P. L., & Brutto, E. M. (1927). College libraries and chemical
Ausbildung. Wissenschaft, 66(1713), 385–389. DOI: https://doi.org/10
.1126/science.66.1713.385, PMID: 17782476
Huang, Y., Zhu, D., Lv, Q., Porter, A. L., Robinson, D. K. R., &
Wang, X. (2017). Early insights on the Emerging Sources
Citation Index (ESCI): An overlay map-based bibliometric study.
Scientometrics, 111(3), 2041–2057. DOI: https://doi.org/10
.1007/s11192-017-2349-3
ICSSRA. (2019). CSSCI source journal catalog (2019–2020) (einschließlich
extended version). Retrieved from http://www.sohu.com/a
/303788449_745007 (in Chinese).
ISTIC. (2020A). The selection process of statistical source journals of
Chinese S&T papers. Retrieved from https://www.istic.ac.cn/ (In
Chinese).
ISTIC. (2020B). Statistical data of Chinese S&T papers. Retrieved
from https://www.istic.ac.cn/ (in Chinese).
Jin, B., & Wang, B. (1999). Chinese science citation database: Es ist
construction and application. Scientometrics, 45(2), 325–332.
DOI: https://doi.org/10.1007/BF02458440
Jin, B., Zhang, J., Chen, D., & Zhu, X. (2002). Development of the
Chinese Scientometric Indicators (CSI). Scientometrics, 54(1),
145–154. DOI: https://doi.org/10.1023/A:1015644823690
Jing, Q., & Xian, J. (1988). China’s core journals in natural sciences.
World Books (1). (in Chinese).
Kosmulski, M. (2011). Successful papers: A new idea in evaluation
of scientific output. Journal of Informetrics, 5(3), 481–485. DOI:
https://doi.org/10.1016/j.joi.2011.03.001
Li, J., Lu, X., Li, J., & Wu, D. (2019). Evaluating journal quality by
integrating department journal lists in a developing country: Are
they representative? Journal of Academic Librarianship, 45(6),
1–10. DOI: https://doi.org/10.1016/j.acalib.2019.102067
Li, X., Rong, W., Shi, H., Tang, J., & Xiong, Z. (2018). The impact of
conference ranking systems in computer science: A comparative
regression analysis. Scientometrics, 116(2), 879–907. DOI:
https://doi.org/10.1007/s11192-018-2763-1
Ma, L. (2016). Comparative research on the evaluation system of aca-
demic journals in China. Information Science, 34(1), 167–170. DOI:
https://doi.org/10.13833/j.cnki.is.2016.01.032 (in Chinese)
Ma, Z. (2019). The relevance of national journals from a Chinese
Perspektive. In W. Glänzel, H. F. Moed, U. Schmoch, & M.
Thelwall (Hrsg.), Springer handbook of science and technology
indicators (S. 505–562). Cham: Springer. DOI: https://doi.org
/10.1007/978-3-030-02511-3_20
Meng, L. (1982). Chinese science citation analysis. Peking: Chinese
Akademie der Wissenschaften.
Milojevic(cid:1), S., Radicchi, F., & Bar-Ilan, J. (2017). Citation success
index—An intuitive pair-wise journal comparison metric. Zeitschrift
of Informetrics, 11(1), 223–231. DOI: https://doi.org/10.1016/j.joi
.2016.12.006
Mingers, J., & Yang, L. (2017). Evaluating journal quality: Eine Rezension
of journal citation indicators and ranking in business and man-
agement. European Journal of Operational Research, 257(1),
323–337. DOI: https://doi.org/10.1016/j.ejor.2016.07.058
Ministry of Education. (2016A). Letter of Response to Proposal No.
4405 of the Fourth Session of the 12th National Committee of the
Chinese People’s Political Consultative Conference. Retrieved
from http://www.moe.gov.cn/jyb_xxgk/xxgk_jyta/jyta_xwb
/201702/t20170207_295800.html (in Chinese).
Ministry of Education. (2016B). Reply from the Ministry of Education
to Proposal No. 9752 of the Fourth Session of the 12th National
People’s Congress. Retrieved from http://www.moe.gov.cn
/jyb_xxgk/xxgk_jyta/jyta_xwb/201612/t20161214_291920.html
(in Chinese).
Ministry of Education. (2020). Some opinions on standardizing the
use of related indicators of SCI papers in universities and estab-
lishing a correct evaluation orientation. Retrieved from http://
www.moe.gov.cn/srcsite/A16/moe_784/202002/t20200223
_423334.html (in Chinese).
National Science Library of CAS. (2019A). China Science Citation
Datenbank (CSCD) source journal selection report (2019–2020).
Retrieved from http://sciencechina.cn/cscd_source.jsp (in Chinese).
National Science Library of CAS. (2019B). Chinese Science Citation
Datenbank. Retrieved from http://sciencechina.cn/scichina2/index
_more_en1.jsp (in Chinese).
Qian, R. (2006). “Core journal” and journal evaluation. Peking:
Communication University of China Press (in Chinese).
Qiu, J. (2011). Journal Conference Report III: The significance, Verfahren
and features of Chinese academic journal evaluation. Retrieved
from http://www.nseac.com/html/221/222923.html (in Chinese).
Quantitative Science Studies
323
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
Qiu, J., Li, A., & Shu, M. (2009). An empirical study on the classi-
fication and grading evaluation of Chinese academic journals.
China Publishing Journal, 4, 38–42 (in Chinese).
Qiu, J., & Lou, W. (2014). Constructing an information science
resource ontology based on the Chinese Social Science Citation
Index. Aslib Journal of Information Management, 66(2), 202–218.
DOI: https://doi.org/10.1108/AJIM-10-2013-0114
Quan, W., Chen, B., & Shu, F. (2017). Publish or impoverish: Ein
investigation of the monetary reward system of science in China
(1999–2016). Aslib Journal of Information Management, 69(5),
486–502. DOI: https://doi.org/10.1108/AJIM-01-2017-0014
Ren, S., & Rousseau, R. (2004). The role of China’s English-language
scientific journals in scientific communication. Learned Publishing,
17(2), 99–104. DOI: https://doi.org/10.1087/095315104322958472
Research Services Group at Clarivate. (2019). Classification of
papers in multidisciplinary journals. Retrieved from http://Archiv
.sciencewatch.com/about/met/classpapmultijour/
Rousseau, R., Jin, B., & Yang, N. (2001). Observations concerning
the two- and three-year synchronous impact factor, basierend auf
Chinese science citation database. Journal of Documentation, 57(3),
349–357. DOI: https://doi.org/10.1108/eum0000000007086
SAPPRFT. (2014). The first list of recognized academic journals is
officially announced. Retrieved from http://www.gapp.gov.cn
/news/1663/233978.shtml (in Chinese).
SAPPRFT. (2017). The second list of recognized academic journals
is officially announced. Retrieved from http://www.sapprft.gov.cn
/sapprft/contents/6588/320556.shtml (in Chinese).
Shu, F., Quan, W., Chen, B., Qiu, J., Sugimoto, C. R., & Larivière, V.
(2020). The role of Web of Science publications in China’s tenure
System. Scientometrics, 122, 1683–1695. DOI: https://doi.org/10
.1007/s11192-019-03339-x
Su, J. (2019). The design and implementation of evaluation of
humanities and social science journals under new policies. China
Social Science Review, 3, 131–140, 144 (in Chinese).
Su, P., Shang, C., Chen, T., & Shen, Q. (2017). Exploiting data reliability
and fuzzy clustering for journal ranking. IEEE Transactions on Fuzzy
Systeme, 25(5), 1306–1319. DOI: https://doi.org/10.1109/tfuzz
.2016.2612265
Su, X., Deng, S., & Shen, S. (2012). The design and application
value of the Chinese Social Science Citation Index. Scientometrics,
98(3), 1567–1582. DOI: https://doi.org/10.1007/s11192-012
-0921-4
Su, X., Han, X., & Han, X. (2001). Developing the Chinese Social
Science Citation Index. Online Information Review, 25(6), 365–369.
DOI: https://doi.org/10.1108/EUM0000000006535
Waltman, L., & van Eck, N. J. (2012). A new methodology for
constructing a publication-level classification system of science.
Journal of the American Society for Information Science and
Technologie, 63(12), 2378–2392. DOI: https://doi.org/10.1002
/asi.22748
Wang, J. (2018). Where did nuclear physics go: A turmoil in journal
zoning. Legal Daily Press. Retrieved from https://baijiahao.baidu.com
/S?id=1619611872537294570&wfr=spider&for=pc (in Chinese).
Web of Science Group. (2019). 2019 Journal citation reports: Full
journal list. Retrieved from https://clarivate.com/webofsciencegroup
/wp-content/uploads/sites/2/dlm_uploads/2019/08/ JCR_Full
_Journal_list140619.pdf
Wu, E. (1973). World chemistry core journal. World Book, 11–12
(in Chinese).
Wu, J., Xiao, H., Sheng, S., Zhang, Y., Sun, X., & Zhang, Y. (2015).
The research purpose, methods and results of the “Annual Report
for International Citations of China’s Academic Journals”. In A. A.
Salah, Y. Tonta, A. A. A. Salah, C. Sugimoto, & U. Al (Hrsg.),
Proceedings of the 15th International Society of Scientometrics
and Informetrics Conference (S. 978–987). Istanbul: ISSI.
Wu, Y., Pan, Y., Zhang, Y., Ma, Z., Pang, J., … Yang, Z. (2004).
China Scientific and Technical Papers and Citations (CSTPC):
Geschichte, impact and outlook. Scientometrics, 60(3), 385–397.
DOI: https://doi.org/10.1023/B:SCIE.0000034381.64865.2B
Zhang, L., Rousseau, R., & Sivertsen, G. (2017). Science deserves to
be judged by its contents, not by its wrapping: Revisiting Seglen’s
work on journal impact and research evaluation. PLOS ONE, 12(3),
e0174205. DOI: https://doi.org/10.1371/journal.pone.0174205,
PMID: 28350849, PMCID: PMC5369779
Zhang, L., & Sivertsen, G. (2020). The new research assessment reform
in China and its implementation. Scholarly Assessment Reports,
2(1). DOI: https://doi.org/10.29024/sar.15
Zhang, Y. (2015). Problems, causes and treatment of academic
evaluation. Journal of Tsinghua University (Philosophy and Socia
Wissenschaften), 30(6), 73–88 (in Chinese).
Zhang, Y., & Lun, H. (2019). Is Google Scholar useful for the evalua-
tion of non-English scientific journals? The case of Chinese
journals. Learned Publishing, 32(2), 154–162. DOI: https://doi
.org/10.1002/leap.1208
Quantitative Science Studies
324
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
/
.
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
APPENDIX
Abkürzungen
A&HCI
AH
AMI
CAE
CAS
CASS
CASSES
CAST
CCF
CDGDC
CI
CJCR
CLC
CN
CNKI
CSCD
CSCI
CSSCI
Table A1.
Abbreviations and full names used in the article
Full Names
Arts & Humanities Citation Index
Arts & Humanities
Attraction, Management, Impact
Chinese Academy of Engineering
Chinesische Akademie der Wissenschaft
Chinese Academy of Social Sciences
Chinese Academy of Social Sciences Evaluation Studies
China Association for Science and Technology
China Computer Federation
China Academic Degrees and Graduate Education
Development Centre
Clout Index
Chinese S&T Journal Citation Report
Chinese Library Classification
China Domestic Uniform Serial Publication Numbers
China National Knowledge Infrastructure
Chinese Science Citation Database
Chinese Science Citation Index
Chinese Social Sciences Citation Index
CSTPCD
China Scientific and Technical Papers and
CUSR
ESCI
GCJC
GOSC
ICS
ICSSRA
ISR
ISSN
ISTIC
Citations Database
China University Subject Rankings
Emerging Sources Citation Index
A Guide to the Core Journals of China
General Office of the State Council of the People’s
Republic of China
International Classification for Standards
Institute for Chinese Social Science Research
and Assessment
Index to Scientific Reviews
International Standard Serial Number
Institute of Scientific and Technical Information of China
Quantitative Science Studies
325
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
A comprehensive analysis of the journal evaluation system in China
Table A1.
(Fortsetzung )
Abkürzungen
ISTP
JCR
JIF
JPT
LB
MOE
MOST
NRTA
NSFC
NSLC
PS
S&T
SAPPRFT
SCI
SCIE
SS
SSH
STEM
STM
TC
DER
WAJCI
WoS
Index to Scientific&Technical Proceeding
Full Names
Journal Citation Reports
Journal Impact Factor
Journal Partition Table
Life Sciences & Biomedicine
Ministry of Education of the People’s Republic of China
Ministry of Science and Technology of the People’s
Republic of China
National Radio and Television Administration of the
Volksrepublik China
National Natural Science Foundation of China
National Science Library, Chinesische Akademie der Wissenschaft
l
D
Ö
w
N
Ö
A
D
e
D
F
R
Ö
M
H
T
T
P
:
/
/
D
ich
R
e
C
T
.
M
ich
T
.
Physical Sciences
Science and Technology
State Administration of Press, Publication, Radio,
Film and Television of the People’s Republic of China
Science Citation Index
Science Citation Index Expanded
Social Sciences
Social Sciences and Humanities
Wissenschaft, Technologie, Engineering and Medicine
Scientific, Technical and Medical
Total Citations
Technology Engineering
World Academic Journal Clout Index
Web of Science
/
e
D
u
Q
S
S
/
A
R
T
ich
C
e
–
P
D
l
F
/
/
/
/
2
1
3
0
0
1
9
0
6
6
3
3
Q
S
S
_
A
_
0
0
1
0
3
P
D
.
/
F
B
j
G
u
e
S
T
T
Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3
Quantitative Science Studies
326
