RESEARCH ARTICLE - IA de Investigación especializada en el MIT

ARTÍCULO DE INVESTIGACIÓN

Consistency pays off in science

Şirag Erkol1

, Satyaki Sikdar1

, Filippo Radicchi1

, and Santo Fortunato1,2

1Center for Complex Networks and Systems Research, Luddy School of Informatics, Informática, and Engineering,
Indiana University Bloomington, Bloomington, EN, EE.UU
2Indiana University Network Science Institute (IUNI), Indiana University Bloomington, Bloomington, EN, EE.UU

un acceso abierto

diario

Palabras clave: citation, Nobel Prize, science of science, success

ABSTRACTO

The exponentially growing number of scientific papers stimulates a discussion on the interplay
between quantity and quality in science. En particular, one may wonder which publication
strategy may offer more chances of success: publishing lots of papers, producing a few hit
documentos, or something in between. Here we tackle this question by studying the scientific
portfolios of Nobel Prize laureates. A comparative analysis of different citation-based
indicators of individual impact suggests that the best path to success may rely on consistently
producing high-quality work. Such a pattern is especially rewarded by a new metric, el
E-index, which identifies excellence better than state-of-the-art measures.

Citación: Erkol, Ş., Sikdar, S., Radicchi,
F., & Fortunato, S. (2023). Consistency
pays off in science. Quantitative
Science Studies, 4(2), 491–500. https://
doi.org/10.1162/qss_a_00252

DOI:
https://doi.org/10.1162/qss_a_00252

Revisión por pares:
https://www.webofscience.com/api
/gateway/wos/peer-review/10.1162
/qss_a_00252

Supporting Information:
https://doi.org/10.1162/qss_a_00252

INTRODUCCIÓN

Recibió: 9 Noviembre 2022
Aceptado: 12 Febrero 2023

Autor correspondiente:
Santo Fortunato
santo@indiana.edu

Editor de manejo:
Juego Waltman

Derechos de autor: © 2023 Şirag Erkol, Satyaki
Sikdar, Filippo Radicchi, and Santo
Fortunato. Published under a Creative
Commons Attribution 4.0 Internacional
(CC POR 4.0) licencia.

La prensa del MIT

The number of scientific papers has been growing exponentially for over a century (Dong, Mamá
et al., 2017; Fortunato, Bergstrom et al., 2018). The number of papers per author has been
relatively stable for a long time, but it has been increasing over the past decades (Dong
et al., 2017), favored by the growing tendency of scientists to work in teams (Wuchty, jones,
& Uzzi, 2007).

Such increased productivity is incentivized by career evaluation criteria that typically reward
large outputs, making scientists less risk averse when choosing research directions (Franzoni &
Rossi-Lamastra, 2017). Este, sin embargo, may come at the expense of the quality of research out-
comes (Bornmann & Tekles, 2019; Sunahara, Perc, & Ribeiro, 2021). En efecto, it has been shown
that the exponential growth of the number of publications corresponds to a much slower
increase in the number of new or disruptive ideas (Chu & evans, 2021; Milojević, 2015).

Sin embargo, although scholars should focus on quality, it is unclear whether it is more reward-
ing to pursue rare hit papers, have a consistent track record of valuable outputs, or be in
between these scenarios. Analyzing the careers of arguably the most successful class of scien-
tistas, Nobel Prize laureates, may help address this issue. En particular, we would like to check if
there is a dominant path to success in the careers of such illustrious scholars.

To that effect, we consider a broad range of evaluation metrics that reward one-hit wonders
alongside those that favor a consistent production of high-quality research and investigate their
effectiveness in identifying Nobelists from within a more extensive set of similarly productive
científicos. We find that the best-performing metrics are indeed the ones that prioritize a con-
sistent stream of high-quality research.

The rest of this article is organized as follows. We first describe the data collection and
curation in Section 2. Entonces, we briefly review some popularly adopted impact metrics and

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Consistency pays off in science

introduce two new ones. En la sección 3, we describe and discuss the two sets of experiments we
used to check which of the two competing scenarios is more common. Finalmente, we give our
conclusions in Section 4.

2. MÉTODOS

2.1. Datos

We consider three fields in which the Nobel Prize is awarded: Physics, Chemistry, and Phys-
iology or Medicine (abbreviated henceforth as Medicine).

The publication records of scientists are obtained from two sources. For Nobelists, we use
the hand-curated data set with explicit annotations for prize-winning papers (li, Yin et al.,
2019). As a baseline, we consider scientists with verified Google Scholar (GS) profiles tagged
with Physics, Chemistry, Physiology, or Medicine as of May 2021.

We use the 2017 version of the Web of Science ( WoS) database to compile the citation
statistics of the articles. We rely on gathering data from different sources on purpose, como
WoS and GS complement each other well. GS offers the possibility of obtaining accurate pub-
lication records of individual scientists without the need to perform name disambiguation
(Radicchi & Castellano, 2013). WoS lets us reconstruct the citation history of individual papers.
Both ingredients are necessary for the type of analysis that we perform in this paper.

We adopt a similar methodology to that of Sinatra, Wang et al. (2016) to match papers
across databases. Given a paper ^p written by author a in GS, we list the papers Pa in WoS
authored by people with the same last name as a. From Pa, we select the paper p with the
highest normalized Levenshtein similarity between the corresponding paper titles (Levenshtein,
1966). We consider it a successful match only if the similarity exceeds 90%. De lo contrario, nosotros
discard ^p from further analysis. Following this procedure, we could match 78.1% of papers
by Nobelists and 49.6% of papers by baseline scientists, respectivamente. For our analysis, nosotros sólo
consider scientists who published their first paper after 1960 and have a portfolio with at least
10 documentos. Detailed statistics are provided in Table 1.

2.2. Métrica

Let us consider a portfolio P = {c1, …, cN} of N = |PAG| papers that collectively receive Ctot
citas (es decir., Ctot =

norte
i ci). We consider the following metrics:

PAG

norte: total number of papers.
Ctot: total number of citations.
Cavg: average number of citations (es decir., Cavg(PAG) = Ctot
norte ).
Cmax: citations received by the most cited paper (es decir., Cmax(PAG) = max{c1, ⋯, cN}).
h: H-index (es decir., the largest number H of the top-cited papers with at least H citations;

Hirsch, 2005).

GRAMO: G-index (es decir., the largest number G of the top-cited papers with at least G2 combined

citas; Egghe, 2006).

Mesa 1. Number of scientists in each category and field

Category
Nobelists

Baseline scientists

Physics
55

4,081

Chemistry

3,330

Medicamento

2,715

492

Estudios de ciencias cuantitativas

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Consistency pays off in science

(cid:4)

q: Q-index, proposed by Sinatra et al. (2016), q(PAG) = exp

,
up to a constant factor, where Θ is the Heaviside function (es decir., Θ(X) = 1 if x > 0 y 0
de lo contrario), and c10,i is the citations gained by paper i within 10 years of publication. Nosotros
normalize c10,i by dividing it with the average c10 of all papers published in the same
discipline and year as paper i (Sinatra et al., 2016).

log c10;i

Q~: a variant of the unnormalized Q-index, where we use the total number of citations ci

d
Θ c10;i

norte
i¼1

1PAG

norte
i¼1

PAG

(cid:2)
Θ c10;i

(cid:3)

(cid:5)

instead of c10,i.

We observe that these measures have their unique preferences for ranking portfolios.
Alguno, like Cmax, appear to reward one-hit wonders, y otros, like H, reward consistency.
One of the goals of this work is to identify and differentiate Nobelists from baseline scientists.
Por lo tanto, we argue that we need a new, simple, yet interpretable metric covering the whole
portfolio spectrum.

2.3. Citation Moment and E-Index

Given a publication portfolio P, one may consider the following extreme scenarios:

(cid:129) Citations are equally distributed among the papers, with each paper having Ctot/N

citas.

(cid:129) A single paper accounts for all citations.

In the first case, there is a sustained production of work of similar quality, while the second

represents a one-hit-wonder situation.

2.3.1. Citation moment

We propose the citation moment Mα, a new parametric measure that can reward both scenar-
ios, as well as the ones in between, depending on the value of the parameter α. It is defined as

Mα Pð Þ ¼ 1
norte

i¼1

cα
i

;

(1)

where α is a real positive number. We remark that Mα is essentially an average of the citation
scores of the papers, where the weight of each score is modulated by the exponent α. We can
make the following observations of the behavior of our metric for different values of α.

α → 0: Mα behaves like Q~ as c α ≈ log c, but unlike Q~, it accounts for uncited papers.
0 < α < 1: Mα is higher for balanced portfolios (i.e., ones with a more uniform distribution of citations). α = 1: Mα becomes identical to Cavg. α > 1: Mα is higher for unbalanced portfolios.
α → ∞: Mα closely imitates Cmax.

2.3.2.

E-index

We also propose an additional parameter-free measure that, like Mα, is sensitive to the distri-
bution of citations. We call this metric E-index, defined as

E Pð Þ ¼ − 1
norte

ci log

i¼1

ci
Ctot

;

(2)

493

Estudios de ciencias cuantitativas

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Consistency pays off in science

Mesa 2. Values of metrics for portfolios with N papers with Ctot citations, of which n are equally
cited and N − n are uncited

Metric
h

GRAMO
Q~

Mα

Value
mín.{⌊Ctot/n⌋, norte}
pag

ﬃﬃﬃﬃﬃﬃﬃﬃ
Ctot

mín.{⌊

⌋, ⌊Ctot/n⌋, norte}

C tot/n
C α
tot
N nα−1

Ctot
N log n

which reaches its maximum Cavg log N when citations are distributed equally among papers,
favoring authors with large average numbers of citations. De hecho, mi(PAG) is just the product of the
average number of citations Cavg and of the Shannon entropy of the citation distribution.

2.4. Behavior of Metrics on Stylized Portfolios

To better understand the behavior of the different metrics in our analysis, we consider a port-
folio with n cited papers with Ctot /n citations each and N − n uncited papers. En mesa 2, nosotros
show the values that several key metrics take in this case.

We see that the citation moment Mα (for α ≠ 0, 1), E-index, and G-index depend on n, norte,
and Ctot. The H-index and the Q~ depend only on the cited papers. So, Por ejemplo, two port-
folios with identical values of Ctot and n would have the same H-index, regardless of the num-
ber of uncited papers. Además, even though the G-index depends on all three parameters,
it depends on them in a somehow undesirable way. Por ejemplo, a portfolio with more
uncited papers may have a G-index value greater than or equal to the G-index of another
portfolio with identical Ctot and n values. En cambio, ranking the portfolio with fewer uncited
works higher (lower N − n), as Mα and E would, seems more intuitive.

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

3. RESULTADOS

En figura 1, we plot Nobelists and baseline scientists according to their number of papers and
the total number of citations. As expected, most Nobelists lie in the top right region, indicando
high levels of both productivity and impact. Sin embargo, there appear to be a few Nobelists in
the top left, indicating that they only produced a handful of high-impact papers. To further
illustrate this difference, we consider two Nobelists in Physics, David J. Bruto (2004) y juan
METRO. Kosterlitz (2016), and plot their publication timelines in Figure 2. Gross has a consistent
production of high-impact works, but Kosterlitz stands out for having a single big paper.

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 1. Total number of citations vs. total number of papers for Nobelists (purple dots) and baseline scientists (gray dots).

Estudios de ciencias cuantitativas

494

Consistency pays off in science

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 2. Consistency versus single-hit scenario. On the x-axis, we indicate the temporal sequence
of papers, and on the y-axis the citations accrued by each paper. The two panels show the profiles
of D. j. Bruto (arriba) y j. METRO. Kosterlitz (abajo). The former has a portfolio with multiple highly cited
documentos, and the latter has one highly cited paper. D. j. Bruto: norte = 122, Ctot = 24,144, Cavg = 197.9,
mi = 768.6. j. METRO. Kosterlitz: norte = 63, Ctot = 11,688, Cavg = 185.5, mi = 348.8.

We now focus on two tasks: portfolio classification and future Nobelist identification.

3.1. Portfolio Classification

We test the performance of the metrics in distinguishing the portfolios of Nobelists from those
of the baseline scientists. We consider two subtasks which we describe below. We use the area
under the precision-recall curve (AUC-PR) in each task as the performance metric. This curve
shows the trade-off between precision and recall at different thresholds. Bounded between 0
y 1, higher AUC-PR values indicate better classification performance. For random predic-
ciones, AUC-PR is the fraction of positive samples. AUC-PR is better suited for imbalanced data
sets than the area under the receiver operating characteristic curve (ROC-AUC) (saito &
Rehmsmeier, 2015). Results for the ROC-AUC are reported in the Supplementary material
and are consistent with the analysis done using AUC-PR.

(cid:129) Lleno. We use the entire portfolio of the scientists described in Section 2.1.
(cid:129) Preaward. We construct the preaward portfolio of Nobelists (es decir., the set of papers
published until the year of the prize-winning paper), discarding those with fewer than
10 documentos. We find that 15 (27%), 28 (55%), y 22 (39%) of Nobelists in Physics,
Chemistry, and Medicine, respectivamente, satisfy the above criteria.

Específicamente, for a Nobelist who published their first paper in year y0 and wrote their prize-
winning article in year yp, we consider the papers published and citations accrued between
years y0 and yp − 1. We then pair the Nobelist with 20 baseline scientists who published their
first papers around the year y0 and wrote at least 10 papers in their careers’ first yp − y0 years.
(cid:129) Optimal α selection. Recall that, unlike other measures, Mα has a tunable parameter α.
Por lo tanto, for each task, we record the performance of Mα across a range of α values

Estudios de ciencias cuantitativas

495

Consistency pays off in science

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d

F
/

4
2
4
9
1
2
1
3
6
3
8
0
q
s
s
_
a
_
0
0
2
5
2
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 3. Classification performance of Mα for varying α. Different symbols denote different fields.
The dashed line α = 1 separates the two regimes. We use the optimal values of α (α*) in our analyses.

AUC-PR values for the Full and Preaward (Pensilvania) portfolio classification tasks. The best-
Mesa 3.
performing metrics for each field are marked in bold type. Mα and E are the standout performers.
Note that values across columns are not comparable as the baseline values are determined by the
respective class imbalance ratios

Physics

Chemistry

Medicamento

Metric
norte

Ctot

Cavg

Cmax

GRAMO
Q~

Mα

Lleno
0.03

0.21

0.42

0.24

0.12

0.15

0.30

0.08

0.43

0.44

Pensilvania
0.07

0.15

0.19

0.12

0.16

0.15

0.19

0.15

0.34

0.23

Lleno
0.13

0.43

0.32

0.25

0.44

0.41

0.32

0.13

0.49

0.53

Pensilvania
0.12

0.34

0.39

0.21

0.36

0.33

0.41

0.20

0.53

0.45

Lleno
0.06

0.52

0.68

0.49

0.50

0.48

0.67

0.26

0.78

0.75

Pensilvania
0.06

0.24

0.46

0.18

0.24

0.17

0.48

0.45

0.68

0.44

496

Estudios de ciencias cuantitativas

Consistency pays off in science

and plot the results in Figure 3. We observe a slight dependence of the optimal α-value (α*)
on the task and the field. We use the corresponding α* values while comparing the per-
formance of Mα with other metrics. En cada caso, sin embargo, we find α* < 1, which indi- cates that portfolios are most separable when the metric prioritizes consistent impact. We record the metrics’ performance in Table 3. In the Supplementary material, we report the classification results on the American Physical Society (APS) bibliographic data set. Metrics agnostic to the distribution of citations appear to perform worse than their coun- terparts across either task. This includes the total number of papers N, as well as total cita- tions Ctot, and maximum citations Cmax. We highlight the performance of three metrics: N, Cavg, and Cmax. N is consistently the worst performer because it does not account for the impact, only volume. Cavg is among the top performers considering the whole portfolio. We believe that is partly due to the nature of the distributions observed in Figure 1, where the Top 20 baseline scholars with the largest E-index in each discipline. The ones marked in Table 4. bold type received the Nobel Prize between 2018 and 2022. Some authors are assigned multiple labels, so they may appear in multiple lists l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u q s s / a r t i c e - p d l f / / / / 4 2 4 9 1 2 1 3 6 3 8 0 q s s _ a _ 0 0 2 5 2 p d / . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Physics H. Dai A. L. Barabási D. Finkbeiner P. McEuen I. Bloch A. Ashkin U. Seljak S. Inouye S. Manabe M. Tegmark J. R. Heath L. Verde S. G. Louie D. I. Schuster N. D. Lang B. Hammer D. Holmgren M. Lazzeri Chemistry H. Dai J. Godwin R. Ruoff K. L. Kelly H. Wang M. Egholm L. Umayam L. Zhang R. Freeman P. Cieplak G. Church Medicine S. Kumar R. A. Larson A. L. Barabási G. L. Semenza A. S. Levey S. Paabo R. A. North A. Patapoutian J. Goldberger M. Snyder J. Magee D. Macmillan M. Houghton G. Winter J. Kuriyan J. R. Heath G. Loewenstein S. Via R. Jaeschke E. H. Schroeter G. Hollopeter W. Lin S. J. Wagner W. L. Jorgensen V. V. Fokin L. P. Kouwenhoven M. Buttiker J. Clardy D. Zhao J. Allison B. Moss Quantitative Science Studies 497 Consistency pays off in science Nobelists are likely to accumulate higher than average citations over their careers. However, performance for the preaward portfolios is a bit worse, probably because we only consider the preaward period of their careers. Winning the prize has been shown to provide a tangible boost to the overall visibility of a scientist, resulting in more citations (Inhaber & Przednowek, 1976). The number of citations of the most cited paper Cmax is among the worst performers, which suggests that the one big-hit portfolio is not typical among Nobelists. This finding supports the idea that scientists win the Nobel Prize after years of consistent, high-quality work. We now shift our focus to the other category of indicators (i.e., ones sensitive to the citation distributions). We find that H records mediocre performance despite rewarding consistency. Its dependence on productivity likely fails to account for the Nobelists with a few highly cited papers. The Q-index performs poorly. However, its variant, Q~, fares considerably better, which is consistent with the fact that it is similar to Mα for small α. Mα and E consistently rank in the top two positions. This further supports the hypothesis that Nobelists set themselves apart by producing a steady stream of high-impact work. 3.2. Identifying Future Nobelists As a test of the predictive power of the metrics, we check whether we can identify scholars who received the Nobel Prize from 2018 to 2022 (i.e., the period not covered by our WoS data set). First, we note that our set of baseline scientists may be missing some of these new Nobelists, in which case we add them manually, provided they have a GS profile. Then, for each metric, we construct a top 20 list of baseline scientists by ranking them in descending order and highlighting the Nobelists. We report the table for the E -index in the main text (Table 4), while the remaining lists can be found in the Supplementary material. In Table 5, we show how many Nobelists appeared in the top 20 lists for each metric. E -index outperforms all other indicators, proving particularly effective for Medicine. Count of Nobelists awarded in the period [2018, 2022] identified in the top 20 lists of Table 5. various metrics. The numbers in parentheses indicate how many such Nobelists have a GS profile Metric N Ctot Cavg Cmax H G Q~ Q Mα E Physics (9) 1 Chemistry (8) 0 Medicine (5) 0 2 1 0 2 2 1 0 1 2 0 0 0 1 0 1 1 0 2 3 3 2 2 3 3 1 2 5 Quantitative Science Studies 498 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u q s s / a r t i c e - p d l f / / / / 4 2 4 9 1 2 1 3 6 3 8 0 q s s _ a _ 0 0 2 5 2 p d . / f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Consistency pays off in science E-index of Nobelists versus baseline scientists with comparable numbers of papers and citations. We see a prevalent trend towards Figure 4. larger E values for Nobelists. Some 58.2% (Physics), 86.3% (Chemistry), and 87.5% (Medicine) of Nobelists have larger E values than their counterparts. To further corroborate this conclusion, we matched each Nobelist with a baseline scientist with (nearly) identical N and Ctot values. In Figure 4, we plot the E-index of each Nobelist and matched baseline pair. We find that the E-index of Nobelists usually exceeds that of their matches. Some exceptions correspond to Nobelists with a low number of highly cited papers. Other outliers might be prominent scholars who have not yet received the award but might receive it in the future. 4. CONCLUSION In this work, we searched for productivity patterns in excellent scientific careers. Specifically, we aimed to assess whether the output of high-profile scientists is more likely to be character- ized by a low number of hit papers or by a consistent production of high-quality work. To address this question, we have examined the scientific portfolios of Nobel Prize winners in Physics, Chemistry, and Medicine and checked which citation-based metrics are most suitable to recognize them among a much larger number of baseline scholars. In addition, we intro- duced two new metrics, the E-index and Mα, that reward both consistency and high average impact (when α < 1). We found that the best-performing metrics are the ones that peak when citations are dis- tributed among a considerable number of works rather than being concentrated on a few hit papers. The E-index, in particular, proves especially effective in identifying future Nobelists. A portal for the calculation of E-index and other scores of individual performance can be found at e-index.net. While there are Nobelists whose success relied on isolated hit papers, the most successful scientists usually stayed on top of their game for most of their careers. ACKNOWLEDGMENTS We acknowledge Aditya Tandon’s help in this study’s initial phase. This work uses WoS data by Clarivate Analytics provided by the Indiana University Network Science Institute and the Cyberinfrastructure for Network Science Center at Indiana University. COMPETING INTERESTS The authors have no competing interests. Quantitative Science Studies 499 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u q s s / a r t i c e - p d l f / / / / 4 2 4 9 1 2 1 3 6 3 8 0 q s s _ a _ 0 0 2 5 2 p d . / f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Consistency pays off in science FUNDING INFORMATION This project was partially supported by grants from the Army Research Office (#W911NF-21-1- 0194) and the Air Force Office of Scientific Research (#FA9550-19-1-0391, #FA9550-19-1-0354). DATA AVAILABILITY The data for Nobel laureates is available at Li et al. (2019). The disambiguated APS data set is available at Sinatra et al. (2016). The raw data set for the APS can be requested at https:// journals.aps.org/datasets. The code is available at https://github.com/siragerkol/Consistency -pays-off-in-science. WoS data are not publicly available. REFERENCES Bornmann, L., & Tekles, A. (2019). Productivity does not equal use- fulness. Scientometrics, 118(2), 705–707. https://doi.org/10.1007 /s11192-018-2982-5 Chu, J. S., & Evans, J. A. (2021). Slowed canonical progress in large fields of science. Proceedings of the National Academy of Sciences, 118(41), e2021636118. https://doi.org/10.1073/pnas .2021636118, PubMed: 34607941 Dong, Y., Ma, H., Shen, Z., & Wang, K. (2017). A century of sci- ence: Globalization of scientific collaborations, citations, and innovations. In Proceedings of the 23rd ACM SIGKDD Interna- tional Conference on Knowledge Discovery and Data Mining (pp. 1437–1446). https://doi.org/10.1145/3097983.3098016 Egghe, L. (2006). Theory and practise of the g-index. Scientomet- rics, 69(1), 131–152. https://doi.org/10.1007/s11192-006-0144-7 Fortunato, S., Bergstrom, C. T., Börner, K., Evans, J. A., Helbing, D., … Barabási, A.-L. (2018). Science of science. Science, 359(6379), eaao0185. https://doi.org/10.1126/science.aao0185, PubMed: 29496846 Franzoni, C., & Rossi-Lamastra, C. (2017). Academic tenure, risk-taking and the diversification of scientific research. Industry and Innovation, 24(7), 691–712. https://doi.org/10.1080 /13662716.2016.1264067 Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences, 102(46), 16569–16572. https://doi.org/10.1073/pnas .0507655102, PubMed: 16275915 Inhaber, H., & Przednowek, K. (1976). Quality of research and the Nobel Prizes. Social Studies of Science, 6(1), 33–50. https://doi .org/10.1177/030631277600600102 Levenshtein, V. I. (1966). Binary codes capable of correcting dele- tions, insertions and reversals. Soviet Physics Doklady, 10(8), 707–710. Li, J., Yin, Y., Fortunato, S., & Wang, D. (2019). A dataset of publica- tion records for Nobel laureates. Scientific Data, 6(1), 33. https:// doi.org/10.1038/s41597-019-0033-6, PubMed: 31000709 Milojević, S. (2015). Quantifying the cognitive extent of science. Journal of Informetrics, 9(4), 962–973. https://doi.org/10.1016/j .joi.2015.10.005 Radicchi, F., & Castellano, C. (2013). Analysis of bibliometric indi- cators for individual scholars in a large data set. Scientometrics, 97(3), 627–637. https://doi.org/10.1007/s11192-013-1027-3 Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), e0118432. https://doi.org/10.1371/journal.pone.0118432, PubMed: 25738806 Sinatra, R., Wang, D., Deville, P., Song, C., & Barabási, A.-L. (2016). Quantifying the evolution of individual scientific impact. Science, 354(6312), aaf5239. https://doi.org/10.1126/science .aaf5239, PubMed: 27811240 Sunahara, A. S., Perc, M., & Ribeiro, H. V. (2021). Association between productivity and journal impact across disciplines and career age. Physical Review Research, 3(3), 033158. https://doi .org/10.1103/PhysRevResearch.3.033158 Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036–1039. https://doi.org/10.1126/science.1136099, PubMed: 17431139 Quantitative Science Studies 500 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u q s s / a r t i c e - p d l f / / / / 4 2 4 9 1 2 1 3 6 3 8 0 q s s _ a _ 0 0 2 5 2 p d . / f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 ARTÍCULO DE INVESTIGACIÓN imagen

Descargar PDF