RESEARCH ARTICLE

How can citation impact in bibliometrics be
normalized? A new approach combining
citing-side normalization and
citation percentiles

开放访问

杂志

Lutz Bornmann

Division for Science and Innovation Studies, Administrative Headquarters of the Max Planck Society,
Hofgartenstr. 8, 80539 慕尼黑, 德国

关键词: bibliometrics, citation analysis, citation percentiles, citing-side normalization

抽象的

自20世纪80年代以来, many different methods have been proposed to field-normalize citations. 在这个
学习, an approach is introduced that combines two previously introduced methods: citing-side
normalization and citation percentiles. The advantage of combining two methods is that their
advantages can be integrated in one solution. Based on citing-side normalization, each citation
is field weighted and, 所以, contextualized in its field. The most important advantage of
citing-side normalization is that it is not necessary to work with a specific field categorization
scheme for the normalization procedure. The disadvantages of citing-side normalization—the
calculation is complex and the numbers are elusive—can be compensated for by calculating
percentiles based on weighted citations that result from citing-side normalization. 就其一而言
手, percentiles are easy to understand: They are the percentage of papers published in the
same year with a lower citation impact. 另一方面, weighted citation distributions are
skewed distributions with outliers. Percentiles are well suited to assigning the position of a focal
paper in such distributions of comparable papers. The new approach of calculating percentiles
based on weighted citations is demonstrated in this study on the basis of a citation impact
comparison between several countries.

介绍

Research systematically investigates what is (仍然) not known. In order to demonstrate the lag in
current knowledge and the shoulders on which the exploration of the lag by new studies stand,
authors of papers (理想地) cite all relevant previous publications (Kostoff, Murday, 等人。, 2006).
On the basis of this norm in science to cite the relevant past literature, citations have been estab-
lished as a proxy for scientific quality—measuring science “impact” as an important component
of quality (Aksnes, Langfeldt, & Wouters, 2019). Narin (1976) proposed the term evaluative
bibliometrics for methods using citation-based metrics for measuring cognitive influence
(Moed, 2017; van Raan, 2019). Bornmann and Marewski (2019) introduced the bibliometrics-
based heuristics (BBHs) concept concretizing the evaluative use of bibliometrics: “BBHs charac-
terize decision strategies in research evaluations based on bibliometrics data (publications and
citations). Other data (指标) besides bibliometrics are not considered” (Bornmann, 2020).

According to Moed and Halevi (2015), research assessment (based on bibliometrics) is an
integral part of any scientific activity these days: “it is an ongoing process aimed at improving

引文: Bornmann, L. (2020). How can
citation impact in bibliometrics be
normalized? A new approach
combining citing-side normalization
and citation percentiles. Quantitative
Science Studies, 1(4), 1553–1569.
https://doi.org/10.1162/qss_a_00089

DOI:
https://doi.org/10.1162/qss_a_00089

已收到: 8 可能 2020
公认: 30 七月 2020

通讯作者:
Lutz Bornmann
bornmann@gv.mpg.de

处理编辑器:
Ludo Waltman

版权: © 2020 Lutz Bornmann.
在知识共享下发布
归因 4.0 国际的 (抄送 4.0)
执照.

麻省理工学院出版社

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

the quality of scientific/scholarly research. It includes evaluation of research quality and measure-
ments of research inputs, outputs, and impacts, and embraces both qualitative and quantitative
methodologies, including the application of bibliometric indicators and peer review” (p. 1988).
Current research evaluation processes concern single researchers (Bornmann & 马克思, 2014),
research groups, 机构, 组织 (Bornmann, Bowman, 等人。, 2014), and countries
(莱德斯多夫, 瓦格纳, & Bornmann, 2014). Since the beginning of the 20th century, 每年
produced international university rankings have become more and more popular (Vernon,
Balas, & Momani, 2018).

The analysis of citations is at the core of bibliometrics: “citation impact is an important indi-
cator of scientific contribution because it is valid, relatively objective, 和, with existing data-
bases and search tools, straightforward to compute” (Nosek, Graham, 等人。, 2010, p. 1292).
The problem of citation analysis is, 然而, that fields differ in their publication, citation, 和
authorship practices (Waltman & van Eck, 2013乙). Crespo, 李, and Ruiz-Castillo (2013) estimated
那 14% of overall citation inequality can be attributed to field-specific differences in citation
实践. These and similar findings from bibliometrics research make clear that the results of
citation analyses from different fields cannot be compared. Whereas single publications and
researchers can be compared within one field, this is not possible with universities and many
research-focused institutions. For citation analyses in which cross-field comparisons are neces-
sary, field-normalized citation impact indicators have been developed. Field normalization aims
to remove the noise that traces back to the fields while maintaining the signal that reflects (true)
performance differences (Waltman & van Eck, 2019). It is an indication of advanced bibliometrics
to use “reliable statistics, 例如, corrections for differences in publication and citation practices
between scientific disciplines” (van Raan, 2019, p. 244).

自20世纪80年代以来, many approaches have been developed in the scientometrics field to field-
normalize citations. Although some approaches (例如, the number of publications published by an
institution that belongs to the 10% most frequently cited publications in the corresponding fields)
could reach the status of quasistandards, each approach has its specific disadvantages. 在这个
纸, an approach is introduced combining the advantages of two published approaches and
smoothing their specific disadvantages. The first approach is citing-side normalization, whereby
each single citation of a paper is field-normalized. The second approach is the citation percentile,
which is the percentage of papers in a given set of papers with lower citation impact than the
focal paper.

2. LITERATURE OVERVIEW: A SHORT HISTORY OF FIELD NORMALIZATION

Field normalization has a long tradition in bibliometrics. Literature overviews on the developments
in the field can be found in Mingers and Leydesdorff (2015), Bornmann and Marx (2015), 和
Waltman (2016). Field normalizations start from the basic premise that “not all citations are equal.
所以, normalization can be seen as a process of benchmarking that is needed to enhance
comparability across diverse scientists, fields, 文件, time periods, and so forth” (Ioannidis,
Boyack, & Wouters, 2016). Many studies on field normalization either deal with technical issues
(例如, the development of improved indicator variants) or with the way fields should be defined for
use in normalization (例如, by using journal sets or human-based assignments; see Wilsdon, 艾伦,
等人。, 2015). One of the earliest attempts in bibliometrics to field-normalize citations was made by
Schubert and Braun (1986) and Vinkler (1986). They proposed to calculate the average citation
rate for a journal or field and to use this reference score to field-normalize (单身的) papers published
in the journal or field (by dividing the citation counts of every single papers by the reference score).
The resulting metric was named the relative citation rate (RCR) by Schubert and Braun (1986).

Quantitative Science Studies

1554

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

Since its introduction, the RCR has been criticized for its use of the arithmetic average in the
normalization. The arithmetic average should not be used as a measure of central tendency for
skewed citation data. According to Glänzel and Moed (2013), “the mean should certainly not be
used if the underlying distribution is very skewed, and has a long tail” (p. 383). The fact that arith-
metic averages of citation data and, 因此, field normalized citation scores are sensitive to outliers
has been named by van Raan (2019) as the Göttingen effect: “在 2008, a paper published by a
researcher of the University of Göttingen became extremely highly cited, many thousands of times
一年, within a very short time … As a result, for several years after this publication, Göttingen
achieved a very high position in … [university] rankings” (p. 260).

To deal with the problem of skewed distributions in field normalization, McAllister, Narin, 和
Corrigan (1983, p. 207) already proposed in the 1980s that percentiles should be used for citation
数据:

the pth percentile of a distribution is defined as the number of citations Xp the percent of papers
receiving Xp such that or fewer citations is equal to p. Since citation distributions are discrete, 这
pth percentile is defined only for certain p that occur in the particular distribution of interest. 因此
we would say that a 1974 physiology paper receiving one citation falls in the 18th percentile of
the distribution. 这意味着 82 百分 (100 - 18) 全部的 1974 我们. physiology papers
received more than one citation. For any paper in the 18th percentile of any subject area citation
分配, 18 percent of the papers performed at a level less than or equal to the particular
纸, 和 82 percent of the papers in the subject area outperformed the particular paper.

For Schreiber (2013) “percentiles … have become a standard instrument in bibliometrics”
(p. 822) in current bibliometrics. Percentiles are recommended in the Leiden manifesto which
includes 10 principles to guide research evaluation (希克斯, Wouters, 等人。, 2015). 最近的
field-normalizing percentile approach has been published by Bornmann and Williams (2020).

One of the biggest challenges in field normalizing citations is the selection of the system cate-
gorizing papers to fields. The overview by Sugimoto and Weingart (2015) shows that existing sys-
tems emphasize cognitive, 社会的, or institutional orientations of fields to a different extent. 各种各样的
field categorization schemes are in use to normalize citations and there exists no standard use in
bibliometrics. The most frequently used schemes are multidisciplinary schemes that span all fields
(Sugimoto & Weingart, 2015; 王 & Waltman, 2016). These schemes are typically based on
journal sets: the Web of Science (WoS) subject categories of Clarivate Analytics and the Scopus
subject areas of Elsevier. The use of journal sets can be justified quite well: according to Milojevic(西德:1)
(2020, p. 184) “journals often serve as anchors for individual research communities, and new jour-
nals may signify the formations of disciplines.” Each journal is a well-crafted folder sustained by
编辑, reviewers, and authors who usually know and use that outlet. Authors typically direct their
manuscripts in an informed way to reach the appropriate audience for the content and argument.

There are two problems with these schemes, 然而, which is why Waltman and van Eck
(2012) proposed a new method for algorithmically constructing classification systems (see also
Boyack & Klavans, 2010): (A) Because journals publish many different papers, journals are usually
assigned to more than one category; 和 (乙) journal sets represent broad fields which is why
papers from specific fields might be misclassified (see Strotmann & 赵, 2010). The results by
Shu, Julien, 等人. (2019) reveal that about half of the papers published in a journal are not from the
field to which the journal has been assigned.

The system proposed by Waltman and van Eck (2012) is based on citation relations between
single publications. The advantages of the system are that (A) it assigns single publications (并不是

Quantitative Science Studies

1555

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

journals) to fields and (乙) it provides a fine-grained categorization scheme of publications. Ruiz-
Castillo and Waltman (2015) demonstrate the use of the system for field normalization. The system,
然而, has not remained without criticism: 因为

“fields” are algorithmic artifacts, they cannot easily be named (as against numbered), 和
therefore cannot be validated. 此外, a paper has to be cited or contain references in
order to be classified, since the approach is based on direct citation relations … However,
algorithmically generated classifications of journals have characteristics very different from
content-based (那是, semantically meaningful) classifications … The new Leiden system is
not only difficult to validate, it also cannot be accessed or replicated from outside its context of
production in Leiden (莱德斯多夫 & Milojevic(西德:1), 2015, p. 201).

As the recent results by Sjögårde, Ahlgren, and Waltman (2020) 展示, at least the labeling

problem of the fields can be solved.

Another critical point is that the field assignments based on citation relations change with new
citations. The approach does not lead to stable results, and it is elusive why the field assignment of
a paper should change. Further critical remarks can be found in Haunschild, Schier, 等人. (2018).
Based on the critique of the system proposed by Waltman and van Eck (2012), Colliander and
Ahlgren (2019) introduced an item-oriented approach that avoids clustering, but uses publication-
level features to estimate subject similarities. The empirical comparison of this approach with
standard approaches in bibliometrics by the authors revealed promising results. Future indepen-
dent studies will demonstrate whether these first positive results can be confirmed.

As an alternative to multidisciplinary schemes, monodisciplinary schemes have been proposed
for field normalization. The advantages of these schemes are that papers are usually assigned to a
single research field and human indexers (field experts or authors of papers) assign the relevant field
to a paper intellectually (Bornmann, 马克思, & Barth, 2013). 最近几年, studies have used dif-
ferent monodisciplinary schemes to field-normalize citations in certain fields: Bornmann and
Wohlrabe (2019) used Journal of Economic Literature classification ( JEL) codes in economics,
Bornmann, Schier, 等人. (2011) and Bornmann and Daniel (2008) used Chemical Abstracts (CA)
sections in chemistry and related areas, Radicchi and Castellano (2011) used Physics and
Astronomy Classification Scheme (PACS) codes in physics and related areas, and Smolinsky and
Lercher (2012) used the MathSciNet’s Mathematics Subject Classification (MSC) system in math-
信息学. The disadvantages of monodisciplinary schemes are that they are restricted to single fields
and the assignments by the indexers may be affected by subjective biases.

One problem that affects many field classification systems (mono- and multidisciplinary) 是
that they exhibit different aggregation levels, and it is not clear which level should be used to
field-normalize citations (Waltman & van Eck, 2019; Wouters, Thelwall, 等人。, 2015). In biblio-
指标, different results and opinions have been published as to whether an aggregation level
change has any (重要的) influence on the field-normalized scores: Zitt, Ramanana-Rahary,
and Bassecoulard (2005) report a lack of stability of these scores; Colliander and Ahlgren (2011)
arrive at another conclusion. 王 (2013) holds the opinion that “normalization at finer level is
still unable to achieve its goal of improving homogeneity for a fairer comparison” (p. 867).

3. CITING-SIDE NORMALIZATION

The literature overview in section 2 has shown that there are many problems with field normal-
ization in bibliometrics and it has not yet been possible to establish a standard. One can expect
that some problems will remain unsolved without finding a perfect solution. 例如, it will

Quantitative Science Studies

1556

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

remain a normative decision as to which field categorization scheme is used (and on what level).
Independently of the system that is used, fields are not isolated and research based on between-
field collaborations is common (Ioannidis et al., 2016). “With the population of researchers,
scientific literature and knowledge ever growing, the scientific endeavour increasingly integrates
across boundaries” (Gates, Ke, 等人。, 2019, p. 34). According to Waltman and van Eck (2013A),
“the idea of science being subdivided into a number of clearly delineated fields is artificial. 在
现实, boundaries between fields tend to be rather fuzzy” (p. 700).

A possible solution to these problems might be to avoid the use of field categorization schemes
(Bornmann, 马克思, 等人。, 2013), clustering (Waltman & van Eck, 2012), and similarity approaches
(Colliander & Ahlgren, 2019), and for each focal paper (that is assessed) to manually search some
papers for comparison that are thematically similar (Kostoff, 2002; Waltman, 2016). This solution
corresponds to the judgement by Hou, Pan, 等人. (2019) that field normalization cannot be solved
by statistical techniques. The manual collection of papers for the comparison with a focal paper
might be possible in the evaluation of small sets of papers; 然而, it is not practicable for large
套 (例如, all papers published by a university over several years). 此外, one needs experts
from the fields to find the papers for comparison.

Another solution that can be applied to large sets of papers is not to normalize citation impact
based on expected citations from the reference sets, but to normalize single citations directly.
So-called citing-side field normalizing approaches have been proposed in recent years that
normalize each single citation of a focal paper. van Raan (2014) sees these “field-independent
normalization procedures” (p. 22) as an important and topical issue in bibliometrics. 最简单的
procedure is to divide each citation by the number of cited references of the citing paper. The use
of the number of cited references is intended to reflect the disciplinary context of the citing paper
and to standardize the citation field specifically. It is a decisive advantage of citing-side normal-
ization that it “does not require a field classification system” (Waltman & van Eck, 2013A, p. 700).
Citing-side normalization, 因此, solves the problem with the selection of a field-categorization
scheme by refraining from it.

Citing-side normalization might be a reasonable approach for citation analysis, as the goal of
field normalization is the normalization of citation impact (see Waltman, van Eck, 等人。, 2013).
Given the different directions of the two basic field normalization approaches, citing-side
approaches are more focused on the aim of field normalization than approaches that are based
on reference sets on the cited side: Citing-side approaches normalize each single citation of a
focal paper. Bornmann and Marx (2015) demonstrated the problem of field normalization based
on cited-side normalization by using the well-known paper by Hirsch (2005) on the h-index as an
例子. This paper is a typical bibliometrics paper (it introduces a new indicator based on
publication and citation data), but receives citations from many fields (not only from the biblio-
metrics field). If a focal paper is attractive for authors publishing in other fields with high citation
density, it has an advantage over another focal paper that is not as attractive for these fields.
Although both focal papers might belong to the same field (viewed from the cited-side perspec-
主动的), they have different chances of being cited.

The paper by Hirsch (2005) is concerned with another “problem” (for field normalization): 它
was published in the Proceedings of the National Academy of Sciences of the United States of
美国. This is a multidisciplinary journal and is assigned to another journal set than most of the
papers published in bibliometrics (which are assigned to library and information science). 因此,
by using journal sets as a field categorization scheme, the paper would not be compared with its
“true” reference papers, but with various papers from many different fields, which are usually
published in multidisciplinary journals. An appropriate reference set for this paper would be

Quantitative Science Studies

1557

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

all papers published in journals in the library and information science set. If one decides to
manually collect the reference papers for comparison (see above), the ideal reference set for the
paper by Hirsch (2005) would consist of all papers publishing a variant of the h-index or all papers
having introduced an indicator combining the number of publications and the number of citations
in a single number.

The idea of citing-side normalization has been introduced by Zitt and Small (2008). They pro-
posed a modification of the Journal Impact Factor ( JIF) by fractional citation weighting. The JIF is a
popular journal metric that is published in the Journal Citation Reports by Clarivate Analytics. 这
indicator measures the average citation rate of papers published in a journal within 1 年. Citing-
side normalization is also named as source normalization, fractional counting of citations, 或者
a priori normalization (Waltman, 2016; Waltman & van Eck, 2013A). The method focuses on
the citation environment of single citations and weights each citation depending on its cita-
tion environment: A citation from a field with high citation density (一般, authors in
these fields include many cited references in their papers) receives a lower weight than a ci-
tation from a field with low citation density (一般, authors in these fields include only a
few cited references in their papers). The basic idea of the method is as follows: Each citation
is adjusted for the number of references in the citing publication or in the citing journal (作为一个
representative for the entire field). In the recent decade, some variants of citing-side indicators
have been published (Waltman, 2016; Waltman & van Eck, 2013A). These variants are pre-
sented in the following based on the explanations by Bornmann and Marx (2015).

SNCS1 ¼

i=1

人工智能

(1)

The first variant has been named SNCS1 (Source Normalized Citation Score 1). In the formula,
ai is the average number of linked references in those publications that appeared in the same
journal and in the same publication year as the citing publication i. Linked references are the part
of cited references that refers to papers from journals covered by the citation index (例如, WoS or
Scopus). The limitation to linked references (instead of all references) is intended to prevent a
situation in which fields that frequently cite publications are not indexed in WoS are disadvan-
taged (see Marx & Bornmann, 2015). The calculation of the average number of linked references
in SNCS1 is restricted to certain referenced publication years. Imagine a focal paper published in
2008 with a citation window covering a period of 4 年 (2008 到 2011). 在这种情况下, every ci-
tation of the focal paper is divided by the average number of linked references to the four previous
年. 换句话说, a citation from 2010 is divided by the linked cited references from the
时期 2007 到 2010. This restriction to recent publication years is designed to prevent fields
that cite rather older literature from being disadvantaged in the normalization (Waltman &
van Eck, 2013乙).

SNCS2 ¼

i=1

(2)

SNCS2 is the second variant of citing-side indicators. 这里, each citation is divided by the
number of linked cited references in the citing publication. 所以, the journal perspective
is not considered in this variant. The selection of the reference publication years is analogous
to SNCS1.

SNCS3 ¼

i=1

1
piri

(3)

1558

Quantitative Science Studies

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

SNCS3 is a combination of SNCS1 and SNCS2. ri is equally defined as in SNCS2. pi is the share
of papers that contain at least one linked cited reference among the following papers: 从
same journal and publication year as the citing paper i. The selection of the referenced publica-
tion years is analogous to SNCS1 and SNC2.

Whereas Leydesdorff, Radicchi, 等人. (2013) concluded that cited-side normalization outper-
forms citing-side normalization, the empirical results of Waltman and van Eck (2013A) 和
Bornmann and Marx (2015) demonstrated that citing-side normalization is more successful in
field-normalizing citation impact than cited-side normalization. 所以, it seems reasonable
for reaching the goal of field normalization to weight each citation “based on the referencing
behavior of the citing publication or the citing journal” (Waltman & van Eck, 2013A, p. 703).
The comparison of the three citing-side approaches by Waltman and van Eck (2013乙, p. 842)
revealed that

SNCS(2) should not be used. 此外, the SNCS(3) approach appears to be preferable
over the SNCS(1) 方法. The excellent performance of the SNCS(3) approach in the case
of classification system C … suggests that this approach may be especially well suited for
fine-grained analyses aimed for instance at comparing researchers or research groups active
in closely related areas of research.

The results by Bornmann and Marx (2015), 然而, did not reveal these large differences

between the three indicator variants.

Cited-side normalization is frequently confronted with the problem that the used field catego-
rization scheme assigns papers to more than one field. 因此, it is necessary to consider these
multiple assignments in the calculation of field-normalized indicators (see Waltman, van Eck,
等人。, 2011). As multiple assignments are not possible with citing-side normalization, this prob-
lem is no longer existent—a further decisive advantage of the approach.

4. PURPOSE OF THE STUDY—THE COMBINATION OF CITING-SIDE NORMALIZATION AND
CITATION PERCENTILES

In section 3, the advantages of field normalization using citing-side approaches have been
demonstrated based on the previous literature. Although these advantages have been reported
in several papers over many years, these approaches have not been established as standard indi-
cators in (applied) bibliometrics. 例如, the Leiden Ranking (see https://www.leidenranking
.com) does not consider citing-side indicators, but percentile-based cited-side indicators. 一
important reason for the avoidance of citing-side indicators might be that these indicators are more
complicated to understand (and explain) than many cited-side indicators and indicators that are
not based on field normalization. The results by Hammarfelt and Rushforth (2017) 显示
“simple and well-established indicators, like the JIF and the h-index, are preferred” (PP. 177–178)
when indicators are used in practice. Jappe, Pithan, and Heinze (2018) similarly wrote that “the
journal impact factor (JIF) … and the Hirsch Index (h-index or HI) … have spread widely among
research administrators and funding agencies over the last decade.” According to the University of
Waterloo Working Group on Bibliometrics (2016), “there is often a demand for simple measures
because they are easier to use and can facilitate comparisons” (p. 2).

This study is intended to propose a field normalization approach that combines citing-side
normalization and citation percentiles. The advantage of the combination lies in the abandonment
of a field classification system (by using citing-side normalization) and the realization of field
normalized scores (percentiles) that are relatively simple to understand and being applied in

Quantitative Science Studies

1559

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

research evaluation. In the first step of the approach, weighted citation counts are calculated
based on the formula (see above) presented by Waltman and van Eck (2013A). In this study,
the SNCS3 is used, as Waltman and van Eck (2013乙) recommended its use (based on their em-
pirical results). 然而, the approach is not bound to this SNCS variant. In the second step, 这
percentile approach proposed by Bornmann and Williams (2020) is used to calculate citation
percentiles based on SNCS3. In this step, 也, it is possible to use another percentile approach
such as those proposed by Bornmann, 莱德斯多夫, and Mutz (2013) or Bornmann and Mutz
(2014). This study prefers the approach by Bornmann and Williams (2020), because the authors
point out the advantages of their approach over previous approaches.

Bornmann and Williams (2020) calculated cumulative frequencies in percentages (CPs) 作为
demonstrated in Table 1 based on the size-frequency distribution (Egghe, 2005) to receive
citation percentiles. The table shows the citation counts and SNCS3 for 24 fictitious papers.
例如, there are five papers in the set with 12 citations and a weighted citation impact
的 0.45 each. Note that not all papers with five citations have an SNCS3 score of 0.45 and vice
versa. For the indicator CP-EXWC (the subscript WC stands for weighted citations), the first
percentage (for papers with 1 citation) is set at 0. The calculation of the cumulative percentage
starts in the second row with the percentage of the lowest citation count (16.67%). By setting the
first row to zero, CP-EXWC measures exactly the percentage of papers with lower citation impact
in the set of papers. 例如, CP-EXWC = 95.83 means that exactly 95.83% of the papers
in the set of 24 papers received a citation impact—measured by SNCS3—that is below the
weighted citation impact of 4.51. 16.67% of the papers received less impact than the weighted
citations of 0.20.

CP-EXWC can be calculated for all papers in a database (例如, all WoS papers) with SNCS3
scores included (or the scores based on another variant). 因为 (weighted) citation impact
depends on the length of the citation window, CP-EXWC should be calculated based on all papers
在 1 年 (IE。, separated by publication years). With CP-EXWC calculated using SNCS3, one re-
ceives a field-normalized indicator that is simple to understand—because the scores are

桌子 1.

Cumulative percentages (CP-EXWC) 基于 24 fictitious papers

Citations
1

SNCS3
(rounded)
0.00

数字
of papers
4

Percentage
16.67

Cumulative percentage
(CP-EXWC)
0

0.20

0.37

0.45

0.48

0.67

1.16

1.63

2.17

4.51

12.50

4.17

20.83

8.33

12.50

4.17

8.33

4.17

全部的

100.00

16.67

29.17

33.33

54.17

62.50

70.83

83.33

87.50

95.83

Quantitative Science Studies

1560

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

cumulative percentages—and it is based on an advantageous method of field normalization (看
多于). The definition of CP-EXWC for a focal paper is that x% of papers published in the same
year received a lower weighted citation impact than the focal paper. Weighted citation impact
means that each citation of the focal paper is weighted by the citation behavior in its field. 这
definition is simple to understand, not only by bibliometric experts but also by laypersons.

As citation impact is dependent not only on the publication year but also on the document
type of the cited publication (看, 例如, Lundberg, 2007), the CP-EXWC calculation should not
only be separated by publication year, but also by document type. In this study, it was not neces-
sary to consider the document type in the calculation, because only articles were included.

5. 方法

The bibliometric data used in this paper are from an in-house version of the WoS used at the Max
Planck Society (慕尼黑, 德国). In this study, all papers are included from this database with
the document type “article” and published between 2011 和 2015. The data set contains n =
7,908,584 文件; for n = 914,472 papers no SNCS3 values are available in the in-house data-
根据. 因此, the study is based on n = 6,994,112 文件. The SNCS3 scores and CP-EXWC values
have been calculated as explained in the sections above. In the calculation of the SNCS3 indi-
cator, we followed the procedure as explained by Waltman and van Eck (2013乙). Whereas
Waltman and van Eck (2013乙), 然而, only included selected core journals from the WoS
database in the SNCS3 calculation, the SNCS3 scores for the present study were calculated
based on all journals in the WoS database.

6. 结果

数字 1 shows the distribution of SNCS3 scores for 6 years using frequency distributions. 这是
clearly visible that the SNCS3 distributions are very skewed and characterized by outliers (文章
with very high weighted citation impact).

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 1. Quantile plots (考克斯, 2005) of SNCS3 scores for papers published between 2011 和 2015.

Quantitative Science Studies

1561

How can citation impact in bibliometrics be normalized?

Against the backdrop of these skewed distributions (despite citation weighting by citing-side
normalization), it sounds reasonable (more than ever) to calculate percentiles based on SNCS3
scores. According to Seglen (1992), skewed citation distributions “should probably be regarded
as the basic probability distribution of citations, reflecting both the wide range of citedness values
potentially attainable and the low probability of achieving a high citation rate” (p. 632). 这
basic probability distribution does not appear to be valid only for citation distributions, 但是也
weighted citation distributions (based on SNCS3). Similar to citations, the SNCS3 indicator
appears to follow the so-called “bibliometric laws” (de Bellis, 2009, p. xxiv). This is a set of
regularities working behind citation processes according to which a certain number of citations
is related to the authors generating them (in their papers). The common feature of these processes
(and similar processes based on the number of publications or text words) is an “amazingly steady
tendency to the concentration of items on a relatively small stratum of sources” (de Bellis, 2009,
p. xxiv).

One of these regularities leading to skewed citation distributions might be (larger) 质量
differences between the research published in the papers (Aksnes et al., 2019). A second regu-
larity might be the type of contribution made by the paper: 例如, one can expect many
more citations for methods papers than for papers contributing empirical results (Bornmann,
2015; van Noorden, Maher, & Nuzzo, 2014). A third regularity might be a cumulative advantage
effect by which “already frequently cited [出版物] have a higher probability of receiving
even more citations” (van Raan, 2019, p. 239). According to Ruocco, Daraio, 等人. (2017),
“Price’s [Derek J. de Solla Price] assumption was that the papers to be cited are chosen at random
with a probability that is proportional to the number of citations those same papers already have.
因此, highly cited papers are likely to gain additional citations, giving rise to the rich get richer
cumulative effect.”

数字 2 shows the distribution of CP-EXWC values for papers published between 2010 和
2015. Comparison of Figure 2 with Figure 1 reveals that the scores are no longer skewed.

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 2. Distribution of CP-EXWC values for papers published between 2010 和 2015.

Quantitative Science Studies

1562

How can citation impact in bibliometrics be normalized?

Papers with low citation impact (IE。, low CP-EXWC scores) are prevalent, but the distributions
approximate a uniform distribution.

In this study, the proposed indicator CP-EXWC has been exemplarily applied to publication
and citation data of some countries: 瑞士, 英国, 美国, 德国,
中国, 和日本. The results are shown in Figure 3. The upper graph in the figure is based on
full counting of the countries’ papers. 因此, each paper contributes to the citation impact of a
country with a weight of 1—independent of the additional number of countries involved. 这
score for a country shown in Figure 3 is its CP-EXWC median value. The dotted line in the graph
marks the worldwide average. The score for Switzerland in the upper graph is above that line and
方法, 例如, that on average, 60.85% of the papers worldwide have a weighted citation
impact that is below the weighted citation impact of papers with a Swiss address.

The results in the upper graph correspond to results based on other (field-normalized) citation-
based indicators (例如, Bornmann & 莱德斯多夫, 2013; Bornmann, 瓦格纳, & 莱德斯多夫, 2018;
斯蒂芬, Stahlschmidt, & Hinze, 2020). When citation impact is measured size independently,
certain small countries such as Switzerland show an excellent performance (the Netherlands is
another example, although it is not considered here). It follows the United Kingdom in the upper
graph of Figure 3, which has exceeded the United States in citation impact in recent years. 中国
and Japan are at the bottom of the country list. Although these results come as no real surprise,
differences from previous results are also observable. One difference refers to the performance
differences between the countries that do not appear to be very large. 例如, 不同之处-
ences between Switzerland, 英国, and the United States exceed no more than
four percentage points. Another difference from previous studies concerns the performance level.
In previous studies, countries such as Switzerland show an excellent performance far away
from midlevel performance. If we assume that the dotted line in Figure 3 represents a midlevel

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 3. CP-EXWC for papers published between 2010 和 2015 by six countries: 瑞士 (n = 138,947), 英国 (n = 540,287),
美国 (n = 1,949,391), 德国 (n = 510,207), 中国 (n = 1,096,608), 和日本 (n = 394,328). The national numbers of papers are based
on full counting.

Quantitative Science Studies

1563

How can citation impact in bibliometrics be normalized?

表现 (50% of the papers worldwide exhibit a lower performance), the best countries
(and also the worst) are not far away from 50%. 平均而言, 例如, papers from
Switzerland are (仅有的) 大约 10 percentage points above the midlevel performance.

The lower graph in Figure 3 is based on fractional counting. 因此, it has been considered
that many papers were published by more than one country. In this study (which is based on
the SNCS3 impact indicator), the CP-EXWC score for a paper has been weighted by the number
of countries given on a paper (Bornmann & 威廉姆斯, 2020).

The following formula leads to a fractionally counted mean CP-EXWC score for a country:

(西德:4)

ð
CPEX1 (西德:2) FR1

mCPEX Fð Þ ¼

Þ þ CPEX2 (西德:2) FR2
磷
y
i¼1 FRi

(西德:2)

Þ þ … þ CPEXy (西德:2) FRy

(西德:3)

(西德:5)

(4)

where CPEX1 to CPEXy are weighted by the number of countries given on a paper. 例如, 如果
a paper was published by authors from four countries, the paper is weighted by 0.25. The frac-
tional assignment (weighting) is included by the notation FRi for paper i = 1 to paper y. The sums of
the CP-EXWC scores for paper 1 to paper y published by the unit are divided by the sums of the
weightings for paper 1 to paper y.

By applying fractional counting, citation impact benefits arising from collaborations are
adjusted. As the results in the lower graph in Figure 3 展示, fractional impact counting changes
the national results differently: Whereas larger differences are visible for Switzerland, 联合
王国, 和德国, the differences are smaller for Japan and China. Compared with the
upper graph in Figure 3, China and Japan do not really profit from controlling international
collaborations in the lower graph: The CP-EXWC scores only change from 46.80% 到 46.49%
(中国) 和 46.62% 到 46.07% ( 日本). In contrast to China, Switzerland appears to profit signif-
icantly in terms of citation impact from international collaboration: Its CP-EXWC decreases from
60.85% (upper graph) 到 55.5% (lower graph). The other two countries that also appear to profit
from international collaboration are the United Kingdom and Germany (around four percentage
点).

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

7. 讨论

Because only experts from the same field can properly assess the research of their colleagues, 这
peer review process is the dominant research evaluation method. Since around the 1980s, the use
of indicators in research evaluation has become increasingly popular. One reason might be that
“direct assessment of research activity needs expert judgment, which is costly and onerous, 所以
proxy indicators based on metadata around research inputs and outputs are widely used”
(Adams, Loach, & Szomszor, 2016, p. 2). For Lamont (2012), another reason is that “governments
have turned to new public management tools to ensure greater efficacy, with the result that quan-
titative measures of performance and benchmarking are diffusing rapidly” (p. 202). 然而,
peer review and the use of indicators do not have to be incompatible approaches; it is seen as
the “ideal way” in research evaluation to combine both methods in the so-called informed peer
review process. According to Waltman and van Eck (2016, p. 542), “scientometric indicators can
… be used by the peer review committee to complement the results of in-depth peer review with
quantitative information, especially for scientific outputs that have not been evaluated in detail by
the committee.” In the confrontation of peer review and bibliometrics, one should consider
that both methods are related: “citations provide a built-in form of peer review” (McAllister
等人。, 1983, p. 205).

Citation analysis is one of the most important methods in bibliometrics, as the method appears
to measure quality issues: “at high frequency, citations are good indicators of utility, significance,

Quantitative Science Studies

1564

How can citation impact in bibliometrics be normalized?

even the notion of impact. The late sociologist of science, Robert Merton likened citations to
repayments of intellectual debts. The normative process in science requires authors to acknowl-
edge relevant previous contributions” (Panchal & Pendlebury, 2012, p. 1144). One of the major
challenges of citation analyses is the field dependency of citations. If larger units in science are
evaluated that are working in many fields, it is necessary to consider these differences in the
statistical analyses (Bornmann, 2020). According to Kostoff (2002), “citation counts depend
strongly on the specific technical discipline, or sub-discipline, being examined … The documen-
tation and citation culture can vary strongly by sub-discipline. Since citation counts can vary
sharply across sub-disciplines, absolute counts have little meaning, especially in the absence
of absolute citation count performance standards” (p. 53; see also Fok & Franses, 2007).

One solution to the problem of field-specific differences in citation counts is to contextualize the
results of citation analyses “case by case, considering all the relevant information” (D’Agostino,
Dardanoni, & Ricci, 2017, p. 826). According to Waltman and van Eck (2019), one can “use
straightforward non-normalized indicators and to contextualize these indicators with additional
information that enables evaluators to take into account the effect of field differences” (p. 295).
This might be the best solution if smaller research groups or institutions working in clearly definable
fields are evaluated. For this solution, 然而, it is necessary to involve not only a bibliometric
expert in the evaluation but also an expert from the evaluated field to contextualize these indica-
托尔斯. 例如, for the identification of research groups working in the same field as the focal
团体, it is necessary for an expert to identify these groups that can be used for comparison of the
focal group. This solution of contextualizing the number of times when research is cited is stretched
to its limits when large units such as organizations or countries are addressed in evaluations. 这些
units are multidisciplinary by nature.

自20世纪80年代以来, many different methods have been proposed to field-normalize citations. 它
has not been possible to establish a standard method until now. In this study, an approach is pro-
posed that combines two previously introduced methods: citing-side normalization and percen-
tiles. The advantage of combining two methods is that their advantages can be integrated into a
single solution. Based on citing-side normalization, each citation is field weighted and, 所以,
contextualized in its field. The most important advantage of citing-side normalization is that it is
not necessary to work with a specific field categorization scheme. The disadvantages of citing-
side normalization—the calculation is complex and the values elusive—can be compensated by
calculating percentiles based on the field-weighted citations. 一方面, percentiles are
well understandable: It is the percentage of papers published in the same year with lower citation
impact. 另一方面, weighted citation distributions are skewed distributions including
异常值. Percentiles are well suited to assigning the position of a focal paper in such skewed
distributions including a field-specific set of papers.

Many different approaches of percentile calculation exist (Bornmann, 莱德斯多夫, 等人。,
2013). According to Schreiber (2013, p. 829) “all the discussed methods have advantages and
disadvantages. Further investigations are needed to clarify what the optimal solution to the prob-
lem of calculating percentiles and assigning papers to PRCs [percentile rank classes] 可能,
especially for large numbers of tied papers.” Bornmann and Williams (2020) appear to have
found a percentile solution with comparably good properties. In this study, their percentile ap-
proach based on weighted citations (CP-EXWC) has been applied to the analysis of several coun-
尝试. The country results are similar to many other published results. This correspondence in the
results can be interpreted as a good sign for the new approach: It appears to measure field-
normalized citation impact in a similar way to other indicators. 然而, the approach also
reveals the importance of measuring citation impact based on fractional counting. Several coun-
tries are strongly internationally oriented, which has a larger influence on the results.

Quantitative Science Studies

1565

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

Further studies are necessary to investigate the new approach introduced here. These studies
could also focus on other units than those considered in this study (例如, institutions and research
团体). 此外, it would be interesting to know how the new approach can be understood
by people who are not bibliometric experts: Is it as easy to understand as expected, or are there
difficulties in understanding it?

致谢

The bibliometric data used in this paper are from an in-house database developed and maintained
by the Max Planck Digital Library (MPDL, 慕尼黑) and derived from the Science Citation Index
Expanded (SCI-E), Social Sciences Citation Index (SSCI), and Arts and Humanities Citation Index
(AHCI) prepared by Clarivate Analytics, formerly the IP & Science business of Thomson Reuters
(费城, 宾夕法尼亚州, 美国).

COMPETING INTERESTS

The author has no competing interests.

资金信息

No funding has been received for this research.

DATA AVAILABILITY

The data cannot be made available in a data repository because the provider of the data
(Clarivate Analytics) does not allow this.

参考

Adams, J。, Loach, T。, & Szomszor, 中号. (2016). Interdisciplinary research:
Methodologies for identification and assessment. Do we know what
we are measuring? 伦敦, 英国: Digital Science.

Aksnes, D. W., Langfeldt, L。, & Wouters, 磷. (2019). Citations, citation
指标, and research quality: An overview of basic concepts
and theories. Sage Open, 9(1). DOI: https://doi.org/10.1177
/2158244019829575

Bornmann, L. (2015). Nature’s top 100 revisited. Journal of the
Association For Information Science and Technology, 66(10),
2166. DOI: https://doi.org/10.1002/asi.23554

Bornmann, L. (2020). Bibliometrics-based decision trees (BBDTs)
based on bibliometrics-based heuristics (BBHs): Visualized
guidelines for the use of bibliometrics in research evaluation.
Quantitative Science Studies, 1(1), 171–182. DOI: https://土井
.org/10.1162/qss_a_00012

Bornmann, L。, Bowman, 乙. F。, Bauer, J。, 马克思, W., Schier, H。, &
Palzenberger, 中号. (2014). Bibliometric standards for evaluating
research institutes in the natural sciences. 在乙. 克朗的 & C.
Sugimoto (编辑。), Beyond bibliometrics: Harnessing multidimen-
sional indicators of scholarly impact (PP. 201–223). 剑桥,
嘛: 与新闻界.

Bornmann, L。, & Daniel, H.-D. (2008). Selecting manuscripts for a
high impact journal through peer review: A citation analysis of
communications that were accepted by Angewandte Chemie
International Edition, or rejected but published elsewhere. 杂志
of the American Society for Information Science and Technology,
59(11), 1841–1852. DOI: https://doi.org/10.1002/asi.20901

Bornmann, L。, & 莱德斯多夫, L. (2013). Macro-indicators of citation
impacts of six prolific countries: InCites data and the statistical sig-
nificance of trends. PLOS ONE, 8(2), e56768. DOI: https://doi.org
/10.1371/journal.pone.0056768, PMID: 23418600, PMCID:
PMC3572076

Bornmann, L。, 莱德斯多夫, L。, & Mutz, 右. (2013). The use of percen-
tiles and percentile rank classes in the analysis of bibliometric data:
Opportunities and limits. Journal of Informetrics, 7(1), 158–165.
DOI: https://doi.org/10.1016/j.joi.2012.10.001

Bornmann, L。, & Marewski, J. 氮. (2019). Heuristics as conceptual
lens for understanding and studying the usage of bibliometrics in
research evaluation. Scientometrics, 120(2), 419–459. DOI:
https://doi.org/10.1007/s11192-019-03018-x

Bornmann, L。, & 马克思, 瓦. (2014). How to evaluate individual re-
searchers working in the natural and life sciences meaningfully?
A proposal of methods based on percentiles of citations.
Scientometrics, 98(1), 487–509. DOI: https://doi.org/10.1007
/s11192-013-1161-y

Bornmann, L。, & 马克思, 瓦. (2015). Methods for the generation of
normalized citation impact scores in bibliometrics: Which method
best reflects the judgements of experts? Journal of Informetrics, 9(2),
408–418. DOI: https://doi.org/10.1016/j.joi.2015.01.006

Bornmann, L。, 马克思, W., & Barth, A. (2013). The normalization of
citation counts based on classification systems. Publications, 1(2),
78–86. DOI: https://doi.org/10.3390/publications1020078

Bornmann, L。, & Mutz, 右. (2014). From P100 to P100’: A new citation-
rank approach. Journal of the Association For Information Science

Quantitative Science Studies

1566

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

and Technology, 65(9), 1939–1943. DOI: https://doi.org/10.1002
/asi.23152

Bornmann, L。, Schier, H。, 马克思, W., & Daniel, H.-D. (2011). Is inter-
active open access publishing able to identify high-impact sub-
使命? A study on the predictive validity of Atmospheric
Chemistry and Physics by using percentile rank classes. 杂志
the American Society for Information Science and Technology,
62(1), 61–71. DOI: https://doi.org/10.1002/asi.21418

Bornmann, L。, 瓦格纳, C。, & 莱德斯多夫, L. (2018). The geography
of references in elite articles: What countries contribute to the
archives of knowledge. PLOS ONE, 13(3), e0194805. DOI:
https://doi.org/10.1371/journal.pone.0194805, PMID: 29579088,
PMCID: PMC5868817

Bornmann, L。, & 威廉姆斯, 右. (2020). An evaluation of percentile
measures of citation impact, and a proposal for making them better.
Scientometrics, 124, 1457–1478. DOI: https://doi.org/10.1007
/s11192-020-03512-7

Bornmann, L。, & Wohlrabe, K. (2019). Normalisation of citation
impact in economics. Scientometrics, 120(2), 841–884. DOI:
https://doi.org/10.1007/s11192-019-03140-w

Boyack, K. W., & Klavans, 右. (2010). Co-citation analysis, biblio-
graphic coupling, and direct citation: Which citation approach
represents the research front most accurately? Journal of the
American Society for Information Science and Technology, 61(12),
2389–2404. DOI: https://doi.org/10.1002/asi.21419

Colliander, C。, & Ahlgren, 磷. (2011). The effects and their stability of
field normalization baseline on relative performance with respect
to citation impact: A case study of 20 natural science departments.
Journal of Informetrics, 5(1), 101–113. DOI: https://doi.org/10
.1016/j.joi.2010.09.003

Colliander, C。, & Ahlgren, 磷. (2019). Comparison of publication-
level approaches to ex-post citation normalization. Scientometrics,
120(1), 283–300. DOI: https://doi.org/10.1007/s11192-019
-03121-z

考克斯, 氮. J. (2005). Speaking Stata: The protean quantile plot.
Stata Journal, 5(3), 442–460. DOI: https://doi.org/10.1177
/1536867X0500500312

Crespo, J. A。, 李, 是. R。, & Ruiz-Castillo, J. (2013). The measurement of
the effect on citation inequality of differences in citation practices
across scientific fields. PLOS ONE, 8(3). DOI: https://doi.org/10.1371
/journal.pone.0058727, PMID: 23516542, PMCID: PMC3597729
D’Agostino, M。, Dardanoni, 五、, & Ricci, 右. G. (2017). How to stan-
dardize (if you must). Scientometrics, 113(2), 825–843. DOI:
https://doi.org/10.1007/s11192-017-2495-7

de Bellis, 氮. (2009). Bibliometrics and citation analysis: From the Science

Citation Index to cybermetrics. 拉纳姆, 医学博士: Scarecrow Press.

Egghe, L. (2005). Power laws in the information production process:
Lotkaian informetrics. Kidlington, 英国: Emerald Group Publishing
Limited. DOI: https://doi.org/10.1108/S1876-0562(2005)05

Fok, D ., & Franses, 磷. H. (2007). Modeling the diffusion of scientific
出版物. 计量经济学杂志, 139(2), 376–390. DOI:
https://doi.org/10.1016/j.jeconom.2006.10.021

Gates, A. J。, Ke, Q., Varol, 奥。, & Barabasi, A. L. (2019). Nature’s
抵达: Narrow work has broad impact. 自然, 575(7781), 32–34.
DOI: https://doi.org/10.1038/d41586-019-03308-7, PMID:
31695218

Glänzel, W., & Moed, H. (2013). Opinion paper: Thoughts and
facts on bibliometric indicators. Scientometrics, 96(1), 381–394.
DOI: https://doi.org/10.1007/s11192-012-0898-z

Hammarfelt, B., & Rushforth, A. D. (2017). Indicators as judgment de-
恶习: An empirical study of citizen bibliometrics in research
评估. Research Evaluation, 26(3), 169–180. DOI: https://
doi.org/10.1093/reseval/rvx018

Haunschild, R。, Schier, H。, 马克思, W., & Bornmann, L. (2018).
Algorithmically generated subject categories based on citation
关系: An empirical micro study using papers on overall water
splitting. Journal of Informetrics, 12(2), 436–447. DOI: https://土井
.org/10.1016/j.joi.2018.03.004

希克斯, D ., Wouters, P。, Waltman, L。, de Rijcke, S。, & Rafols, 我.
(2015). Bibliometrics: The Leiden Manifesto for research metrics.
自然, 520(7548), 429–431. DOI: https://doi.org/10.1038/520429a,
PMID: 25903611

Hirsch, J. 乙. (2005). An index to quantify an individual’s scientific
research output. 美国国家科学院院刊
美利坚合众国, 102(46), 16569–16572. DOI:
https://doi.org/10.1073/pnas.0507655102, PMID: 16275915,
PMCID: PMC1283832

Hou, J。, Pan, H. X。, Guo, T。, 李, 我。, 孔, X. J。, & Xia, F. (2019).
Prediction methods and applications in the science of science:
一项调查. Computer Science Review, 34. DOI: https://doi.org
/10.1016/j.cosrev.2019.100197

Ioannidis, J. 磷. A。, Boyack, K., & Wouters, 磷. F. (2016). Citation met-
rics: A primer on how (不是) to normalize. PLOS Biology, 14(9),
e1002542. DOI: https://doi.org/10.1371/journal.pbio.1002542,
PMID: 27599158, PMCID: PMC5012555

Jappe, A。, Pithan, D ., & Heinze, 时间. (2018). Does bibliometric research
confer legitimacy to research assessment practice? A sociological
study of reputational control, 1972–2016. PLOS ONE, 13(6),
e0199031. DOI: https://doi.org/10.1371/journal.pone.0199031,
PMID: 29902239, PMCID: PMC6002049

Kostoff, 右. 氮. (2002). Citation analysis of research performer quality.
Scientometrics, 53(1), 49–71. DOI: https://doi.org/10.1023/A
:1014831920172

Kostoff, 右. N。, Murday, J. S。, Lau, C. G. Y。, & Tolles, 瓦. 中号. (2006).
The seminal literature of nanotechnology research. 杂志
Nanoparticle Research, 8(2), 193–213. DOI: https://doi.org/10
.1007/s11051-005-9034-9

Lamont, 中号. (2012). Toward a comparative sociology of valuation
和评价. Annual Review of Sociology, 38(1), 201–221.
DOI: https://doi.org/10.1146/annurev-soc-070308-120022

莱德斯多夫, L。, & Milojevic(西德:1), S. (2015). The citation impact of German
sociology journals: Some problems with the use of scientometric
indicators in journal and research evaluations. Soziale Welt, 66(2),
193–204. DOI: https://doi.org/10.5771/0038-6073-2015-2-193
莱德斯多夫, L。, Radicchi, F。, Bornmann, L。, Castellano, C。, & 的
Nooy, 瓦. (2013). Field-normalized impact factors (IFs): A com-
parison of rescaling and fractionally counted IFs. Journal of the
American Society for Information Science and Technology, 64(11),
2299–2309. DOI: https://doi.org/10.1002/asi.22911

莱德斯多夫, L。, 瓦格纳, C. S。, & Bornmann, L. (2014). The European
联盟, 中国, and the United States in the top-1% and top-10%
layers of most-frequently cited publications: Competition and
collaborations. Journal of Informetrics, 8(3), 606–617. DOI:
https://doi.org/10.1016/j.joi.2014.05.002

Lundberg, J. (2007). Lifting the crown—citation z-score. 杂志
Informetrics, 1(2), 145–154. DOI: https://doi.org/10.1016/j.joi
.2006.09.007

马克思, W., & Bornmann, L. (2015). On the causes of subject-specific
citation rates in Web of Science. Scientometrics, 102(2), 1823–1827.
DOI: https://doi.org/10.1007/s11192-014-1499-9

McAllister, 磷. R。, Narin, F。, & Corrigan, J. G. (1983). Programmatic
evaluation and comparison based on standardized citation scores.
IEEE Transactions on Engineering Management, 30(4), 205–211.
DOI: https://doi.org/10.1109/TEM.1983.6448622

Milojevic(西德:1), S. (2020). Practical method to reclassify Web of Science
articles into unique subject categories and broad disciplines.

Quantitative Science Studies

1567

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

Quantitative Science Studies, 1(1), 183–206. DOI: https://doi.org
/10.1162/qss_a_00014

Mingers, J。, & 莱德斯多夫, L. (2015). A review of theory and prac-
tice in scientometrics. European Journal of Operational Research,
246(1), 1–19. DOI: https://doi.org/10.1016/j.ejor.2015.04.002
Moed, H. F. (2017). Applied evaluative informetrics. Heidelberg,
德国: 施普林格. DOI: https://doi.org/10.1007/978-3-319
-60522-7

Moed, H. F。, & Halevi, G. (2015). The multidimensional assessment
of scholarly research impact. Journal of the American Society for
Information Science and Technology, 66(10), 1988–2002. DOI:
https://doi.org/10.1002/asi.23314

Narin, F. (1976). Evaluative bibliometrics: The use of publication
and citation analysis in the evaluation of scientific activity.
Cherry Hill, 新泽西州: Computer Horizons.

Nosek, 乙. A。, Graham, J。, 林德纳, 氮. M。, Kesebir, S。, Hawkins, C. B.,
Hahn, C。, … Tenney, 乙. 右. (2010). Cumulative and career-stage
citation impact of social-personality psychology programs and their
members. Personality and Social Psychology Bulletin, 36(10),
1283–1300. DOI: https://doi.org/10.1177/0146167210378111,
PMID: 20668215

Panchal, H。, & Pendlebury, D. A. (2012). 赋予生命. Pendlebury.

Current Science, 103(10), 1144–1145.

Radicchi, F。, & Castellano, C. (2011). Rescaling citations of publi-
cations in physics. Physical Review E, 83(4). DOI: https://doi.org
/10.1103/PhysRevE.83.046116, PMID: 21599249

Ruiz-Castillo, J。, & Waltman, L. (2015). Field-normalized citation
impact indicators using algorithmically constructed classification
systems of science. Journal of Informetrics, 9(1), 102–117. DOI:
https://doi.org/10.1016/j.joi.2014.11.010

Ruocco, G。, Daraio, C。, Folli, 五、, & Leonetti, 中号. (2017). Bibliometric
指标: The origin of their log-normal distribution and why
they are not a reliable proxy for an individual scholar’s talent.
Palgrave Communications, 3, 17064. DOI: https://doi.org/10
.1057/palcomms.2017.64

Schreiber, 中号. (2013). How much do different ways of calculating
percentiles influence the derived performance indicators? A
case study. Scientometrics, 97(3), 821–829. DOI: https://doi.org
/10.1007/s11192-013-0984-x

Schubert, A。, & 布劳恩, 时间. (1986). Relative indicators and relational
charts for comparative assessment of publication output and
citation impact. Scientometrics, 9(5–6), 281–291. DOI: https://土井
.org/10.1007/BF02017249

Seglen, 磷. 氧. (1992). The skewness of science. Journal of the American
Society for Information Science, 43(9), 628–638. DOI: https://土井
.org/10.1002/(SICI)1097-4571(199210)43:9<628::AID-ASI5>3.0
.一氧化碳;2-0

Shu, F。, Julien, C.-A., 张, L。, Qiu, J。, 张, J。, & Larivière, V. (2019).
Comparing journal and paper level classifications of science.
Journal of Informetrics, 13(1), 202–225. DOI: https://doi.org
/10.1016/j.joi.2018.12.005

Sjögårde, P。, Ahlgren, P。, & Waltman, L. (2020). Algorithmic labeling
in hierarchical classifications of publications: Evaluation of biblio-
graphic fields and term weighting approaches. Retrieved July 29,
2020, 来自 https://arxiv.org/abs/2004.08090

Smolinsky, L。, & Lercher, A. (2012). Citation rates in mathematics: A
study of variation by subdiscipline. Scientometrics, 91(3), 911–924.
DOI: https://doi.org/10.1007/s11192-012-0647-3

斯蒂芬, D ., Stahlschmidt, S。, & Hinze, S. (2020). Performance and
structures of the German science system 2020. Studies on the
German innovation system No. 5-2020. 柏林, 德国: 德语
Centre for Higher Education Research and Science Studies
(DZHW).

Strotmann, A。, & 赵, D. (2010). Combining commercial citation
indexes and open-access bibliographic databases to delimit
highly interdisciplinary research fields for citation analysis. 杂志
of Informetrics, 4(2), 194–200. DOI: https://doi.org/10.1016/j.joi
.2009.12.001

Sugimoto, C. R。, & Weingart, S. (2015). The kaleidoscope of disci-
plinarity. Journal of Documentation, 71(4), 775–794. DOI:
https://doi.org/10.1108/Jd-06-2014-0082

University of Waterloo Working Group on Bibliometrics. (2016).
White paper on bibliometrics, measuring research outputs through
bibliometrics. 滑铁卢, 加拿大.

van Noorden, R。, Maher, B., & Nuzzo, 右. (2014). The top 100 文件.
自然, 514(7524), 550–553. DOI: https://doi.org/10.1038
/514550A, PMID: 25355343

van Raan, A. F. J. (2014). Advances in bibliometric analysis: 研究
performance assessment and science mapping. In W. Blockmans,
L. Engwall, & D. Weaire (编辑。), Bibliometrics: Use and Abuse in
the Review of Research Performance (PP. 17–28). 伦敦, 英国:
Portland Press.

van Raan, A. F. J. (2019). Measuring science: Basic principles and
application of advanced bibliometrics. In W. Glänzel, H. F.
Moed, U. Schmoch, & 中号. Thelwall (编辑。), Springer Handbook
of Science and Technology Indicators (PP. 237–280). 占婆,
瑞士: Springer International Publishing. DOI: https://土井
.org/10.1007/978-3-030-02511-3_10

Vernon, 中号. M。, Balas, 乙. A。, & Momani, S. (2018). Are university
rankings useful to improve research? A systematic review. PLOS
ONE, 13(3). DOI: https://doi.org/10.1371/journal.pone.0193762,
PMID: 29513762, PMCID: PMC5841788

Vinkler, 磷. (1986). Evaluation of some methods for the relative assess-
ment of scientific publications. Scientometrics, 10(3–4), 157–177.
DOI: https://doi.org/10.1007/BF02026039

Waltman, L. (2016). A review of the literature on citation impact
指标. Journal of Informetrics, 10(2), 365–391. DOI: https://土井
.org/10.1016/j.joi.2016.02.007

Waltman, L。, & van Eck, 氮. J. (2012). A new methodology for construct-
ing a publication-level classification system of science. Journal of the
American Society for Information Science and Technology, 63(12),
2378–2392. DOI: https://doi.org/10.1002/asi.22748

Waltman, L。, & van Eck, 氮. J. (2013A). Source normalized indica-
tors of citation impact: An overview of different approaches and
an empirical comparison. Scientometrics, 96(3), 699–716. DOI:
https://doi.org/10.1007/s11192-012-0913-4

Waltman, L。, & van Eck, 氮. J. (2013乙). A systematic empirical com-
parison of different approaches for normalizing citation impact
指标. Journal of Informetrics, 7(4), 833–849. DOI: https://土井
.org/10.1016/j.joi.2013.08.002

Waltman, L。, & van Eck, 氮. J. (2016). The need for contextualized scien-
tometric analysis: An opinion paper. In I. Ràfols, J. Molas-Gallart, 乙.
Castro-Martínez, & 右. 伍利 (编辑。), Proceedings of the 21st
International Conference on Science and Technology Indicator
(PP. 541–549). València, 西班牙: Universitat Politècnica de València.
Waltman, L。, & van Eck, 氮. J. (2019). Field normalization of scien-
tometric indicators. In W. Glänzel, H. F. Moed, U. Schmoch, &
中号. Thelwall (编辑。), Springer Handbook of Science and Technology
指标 (PP. 281–300). Heidelberg, 德国: 施普林格. DOI:
https://doi.org/10.1007/978-3-030-02511-3_11

Waltman, L。, van Eck, 氮. J。, van Leeuwen, 时间. N。, & Visser, 中号. S.
(2013). Some modifications to the SNIP journal impact indica-
托尔. Journal of Informetrics, 7(2), 272–285. DOI: https://doi.org
/10.1016/j.joi.2012.11.011

Waltman, L。, van Eck, 氮. J。, van Leeuwen, 时间. N。, Visser, 中号. S。, &
van Raan, A. F. J. (2011). Towards a new crown indicator: 一些

Quantitative Science Studies

1568

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

How can citation impact in bibliometrics be normalized?

theoretical considerations. Journal of Informetrics, 5(1), 37–47.
DOI: https://doi.org/10.1016/j.joi.2010.08.001

王, J. (2013). Citation time window choice for research impact
评估. Scientometrics, 94(3), 851–872. DOI: https://doi.org
/10.1007/s11192-012-0775-9

王, Q., & Waltman, L. (2016). Large-scale analysis of the accuracy
of the journal classification systems of Web of Science and Scopus.
ournal of Informetrics, 10(2), 347–364. DOI: https://doi.org
/10.1016/j.joi.2016.02.003

Wilsdon, J。, 艾伦, L。, Belfiore, E., 坎贝尔, P。, Curry, S。, 爬坡道, S。, ……
约翰逊, 乙. (2015). The metric tide: Report of the independent
review of the role of metrics in research assessment and manage-
蒙特. 布里斯托尔, 英国: Higher Education Funding Council for England
(HEFCE). DOI: https://doi.org/10.4135/9781473978782

Wouters, P。, Thelwall, M。, Kousha, K., Waltman, L。, de Rijcke, S。,
Rushforth, A。, & Franssen, 时间. (2015). The metric tide: Literature re-
看法 (supplementary report I to the independent review of the
role of metrics in research assessment and management).
伦敦, 英国: Higher Education Funding Council for England
(HEFCE).

Zitt, M。, Ramanana-Rahary, S。, & Bassecoulard, 乙. (2005). Relativity of
citation performance and excellence measures: From cross-field to
cross-scale effects of field-normalisation. Scientometrics, 63(2),
373–401. DOI: https://doi.org/10.1007/s11192-005-0218-y

Zitt, M。, & 小的, H. (2008). Modifying the journal impact factor by
fractional citation weighting: The audience factor. 杂志
the American Society for Information Science and Technology,
59(11), 1856–1860. DOI: https://doi.org/10.1002/asi.20880

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
q
s
s
/
A
r
t
我
C
e
–
p
d

我

F
/

1
4
1
5
5
3
1
8
7
1
0
0
0
q
s
s
_
A
_
0
0
0
8
9
p
d