性别 (我是)balance in Citation Practices in
Cognitive Neuroscience
Jacqueline M. Fulvio1
, Ileri Akinnola2, and Bradley R. Postle1
抽象的
■ In the field of neuroscience, despite the fact that the proportion
of peer-reviewed publications authored by women has increased in
recent decades, the proportion of citations of women-led publica-
tions has not seen a commensurate increase: In five broad-scope
journals, citations of papers first- and/or last-authored by women
have been shown to be fewer than would be expected if gender
was not a factor in citation decisions [Dworkin, J. D ., Linn, K. A。,
Teich, 乙. G。, Zurn, P。, 筱原, 右. T。, & Bassett, D. S. 程度
and drivers of gender imbalance in neuroscience reference lists.
自然神经科学, 23, 918–926, 2020]. Given the important
implications that such underrepresentation may have on the
careers of women researchers, it is important to determine
whether this same trend is true in subdisciplines of the field, 在哪里
interventions might be more targeted. 这里, we report the results
of an extension of the analyses carried out by Dworkin et al. (2020)
to citation patterns in the Journal of Cognitive Neuroscience. 这
results indicate that the underrepresentation of women-led publi-
cations in reference sections is also characteristic of papers pub-
lished in Journal of Cognitive Neuroscience over the past
十年. 此外, this pattern of citation imbalances is present
regardless of author gender, implicating systemic factors. 这些重新-
sults contribute to the growing body of evidence that intentional
action is needed to address inequities in the way that we carry
out and communicate our science. ■
介绍
The public dissemination of research findings is critical for
the advancement of any field of scientific inquiry. 相似地,
evidence of impactful publication in peer-reviewed jour-
nals is critical for a researcher’s advancement in their field.
例如, citation-based metrics such as impact factors,
the h-index (Hirsch, 2005), and the i10-index (Connor,
2011) contribute to the evaluation of one’s scholarly
“worth” (Fairhall & Marder, 2020). It is problematic, 那里-
fore, that citation-based metrics of scholarship in neurosci-
ence show a gender bias. A recent study evaluating citation
practices in five broad-scope neuroscience journals—
Brain, 神经科学杂志, 自然神经科学,
Neuroimage, and Neuron—demonstrated over-citation
of papers published by men as first and last authors com-
pared with the rate expected if gender did not play a role in
citation choices, whereas papers published by a woman in
the first- and/or last-author position have been undercited
(Dworkin et al., 2020).
The findings of Dworkin et al. (2020) are a cautionary
tale for fields grappling with gender disparities because
bias in citation practices may limit the advancement of
individual researchers as well as the advancement of their
approaches and ideas. Quantification and dissemination
of evidence of such biases is an important first step toward
developing more equitable practices. 这里, we sought to
1University of Wisconsin–Madison, 2University of Maryland,
Baltimore County
© 2020 麻省理工学院
determine whether the gender imbalance in citation prac-
tices reported for broad-scope neuroscience journals is
also characteristic of the Journal of Cognitive Neuroscience
( JoCN ), the flagship journal of this subdiscipline of
神经科学.
方法
We applied the methodological approach used by Dworkin
等人. (2020), using their open-source R code (osf.io/h79g8/).
Where necessary, we modified the code to support the
JoCN-specific analysis.
Data Acquisition
The data for the analysis were obtained from the Web of
Science Web site (www.webofknowledge.com/). Metadata
为了 2106 research papers and review papers published
in JoCN from January 2009 to July 2020 were downloaded.
We note that metadata for JoCN papers are available dating
back to 1995, but metadata from before 2009 contain
author initials rather than full first names, with the latter
being necessary for the analysis. 此外, preprocessed
metadata from broad-scope neuroscience journals—
Brain, 神经科学杂志, 自然神经科学,
Neuroimage, and Neuron (from here forward, the “broad-
scope journals”)—were obtained from Jordan Dworkin
with permission for use in the analysis described here.
认知神经科学杂志 33:1, PP. 3–7
https://doi.org/10.1162/jocn_a_01643
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
3
3
1
3
1
8
6
2
4
6
7
/
j
哦
C
n
_
A
_
0
1
6
4
3
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Gender Category Assignment
Gender Category Assignment of Papers Published
in JoCN
We first extracted the names of each author of each publi-
cation from the metadata into an array with the general
format “last name, first name; last name, first name.” We
then implemented the algorithm used by Dworkin et al.
(2020) to disambiguate authors with different versions of
their names across papers (such as instances with and with-
out middle initials or with and without nicknames). In brief,
the algorithm matches entries first by last name and then
by the same first and/or middle name or initials and assigns
the most common first name variant to all instances.
下一个, the first names of the first and last authors of each
paper were assigned a probability of belonging to someone
self-identifying with either of two gender labels—“man” or
“woman.” (Note that the a priori assumption that gender
identification is a binary variable is not valid but was neces-
sitated by limitations of our method.) 第一的, each name was
queried within the Social Security Administration baby
name data set, which assigns labels based on the sex as-
signed at birth. If the name was found, the probabilities of
that name belonging to someone self-identifying as “man”
and “woman” were returned. If the name was not found, 它
was submitted to Gender API (gender-api.com/) for proba-
bility assignment. Gender API includes approximately
815,000 unique first names from 189 countries and assigns
labels based on a combination of the sex assigned at birth
and genders detected in social media profiles. 使用
same criteria as Dion, Sumner, and Mitchell (2018) 和
Dworkin et al. (2020), we assigned a gender label to each
author if their name had a probability ≥ .70 of belonging
to someone of either gender. The author’s gender label
was manually assigned in instances where Gender API
returned a probability < .70 if the author had publicly avail-
able pronouns (e.g., on their personal or university faculty
Web sites) following Dworkin et al. (2020).
The 2106 research papers published in JoCN were then
assigned an authorship gender category (“man/man”
[MM], “woman/man” [WM], “man/woman” [MW], or
“woman/woman” [WW]) based on the assigned gender
labels of the first and last authors. Upon completion of this
step, ∼9% of the papers had incomplete authorship gender
category designations because of single authorship, first
name initials, poor formatting because of incorrect parsing
of middle initials, or other formatting problems that arose
during metadata extraction that impeded the name query
steps. In these cases, we performed a manual correction
step to hand-code the category designations (Dion et al.,
2018), sometimes entailing visits to the paper’s page on
the journal’s Web site or the author’s Web site. Single-
authored papers were assigned “MM” or “WW” according
to the assigned gender label of the author.
We note that the automated, probabilistic nature of
gender assignment method may be subject to sources of
bias in the results. For example, bias could arise if missing
metadata were skewed across gender lines, or the Gender
API queries tended to return a higher proportion of equivocal
results on names for which the ground truth label is
“woman,” for example. However, as indicated above, these
and other impediments to gender assignment occurred for a
small proportion (∼9% of papers), to which we then applied
manual correction. Furthermore, Dworkin et al. (2020) car-
ried out an independent test of the method and found high
accuracy (>90%) on both individual author and paper gen-
der category assignments.
Gender Category Assignment of Papers Cited in Papers
Published in JoCN
A final critical preprocessing step was to assign authorship
gender categories to the papers cited in the 2106 JoCN
文件. We first extracted each JoCN paper’s citation list
from the metadata. 重要的, although the extracted
citation lists did not contain author first names (仅有的
initials), the metadata did include the Digital Object
Identifier (DOI) associated with each citation. 这些
DOIs allowed us to use the preprocessed data from six
journals—the five broad-scope journals plus JoCN—as a
lookup table, in which we attempted to match each cita-
tion DOI with a DOI of a paper published in one of the
six journals. If a match was found, we assigned that cited
paper to the authorship gender category assigned to the
original paper (from the previous step for JoCN and from
analogous data for the five broad-scope journals analyzed
by Dworkin et al. [2020]). If citation list metadata for a par-
ticular paper were incomplete, the matching algorithm
attempted to match any available DOIs in the list; if the meta-
data were missing, the matching algorithm proceeded with
the next paper. No manual correction of citation list data was
执行的. In all, citations from 2069 的 2106 文件
(98.3%) published in JoCN from 2009 to July 2020 是
matched with other papers from JoCN and the five broad-
scope journals, 和 20.5% of the total references being
assigned an authorship gender category including self-
citations. Self-citations constituted 28.1% of categorized
citations. As described below, self-citations were removed
when computing Gender Balance Citation Indices, 哪个
therefore were based on 14.7% of the total references
cited by JoCN papers published from 2009 to July 2020.
Categorical Gender Quantification
Quantification of JoCN Authorship
The gender balance of authorship in JoCN was quantified in
three ways: collapsing across the January 2009 to July 2020
time frame, broken out into each publication month, 和
as the cumulative portion of the overall time frame leading
up to each publication month and year. This latter set of
proportions served as the base rate (IE。, the “expected”
比例) for the Gender Citation Balance Index calcu-
lations described below.
4
认知神经科学杂志
体积 33, 数字 1
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
3
3
1
3
1
8
6
2
4
6
7
/
j
哦
C
n
_
A
_
0
1
6
4
3
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Computation of Gender Citation Balance Indices
Our primary goal was to determine how the gender pro-
portions in the reference lists of papers published in
JoCN corresponded to the gender proportions of JoCN
authorship. Following Dworkin et al. (2020), 我们删除了
self-citations (IE。, cited papers for which either the first or
last author was the either first or last author on the citing
纸) to remove effects of gender differences in self-
citation behaviors (国王, 伯格斯特罗姆, Correll, Jacquet, &
西方, 2017) and focused instead on authors’ citation of
other researchers in the field. For each of the 2069 文件
that cited other papers from JoCN and the five broad-scope
journals, we computed the proportion of those citations
assigned to each of the four gender citation categories,
which were designated the “observed” proportions.
We computed Gender Citation Balance Indices for each
of the four gender citation categories as
Gender Citation Balance Index
¼ observed prorportion−expected proportion
expected proportion
(1)
因此, positive values corresponded to more frequent
citations of the category than expected, and negative values
corresponded to less frequent citations of the category
than expected. (注意, because we did not have JoCN
publication data before January 2009, we used the JoCN
authorship in each category during January 2009 为了
expected rates of papers published in January 2009.)
最后, we bootstrapped the 95% confidence interval for
each category using 1000 iterations of random sampling
with replacement from the 2069 文件. For each iteration,
we determined the Gender Citation Balance Index for each
类别, and the 2.5th and 97.5th percentiles corre-
sponded to the lower and upper bounds of the confidence
interval for that category, 分别.
结果
The categorical gender breakdown in JoCN authorship has
been relatively stable from 2009 to mid-2020, with an in-
crease in WW-authored publications in the most recent
年 (数字 1). 全面的, 40.8% of JoCN papers published
during this timeframe were MM, with the remaining 59.2%
having at least one woman in the first or last author posi-
系统蒸发散 (IE。, W ∪ W). 有趣的是, this proportion of W ∪ W
papers in JoCN is considerably larger than the average of
44.7% W ∪ W-authored papers for the five broad-scope
neuroscience journals that are also included in our
分析.
The Gender Citation Balance Indices of papers pub-
lished in JoCN reveal an overcitation of MM papers and
an undercitation of WM, MW, and WW papers (数字 2).
During the 2009 to July 2020 timeframe, MM papers had
a base rate of 40.8%, but they accounted for 57.9% of cate-
gorized citations when self-citations were removed. WM
papers had a base rate of 33.5% but accounted for 24.3%
of categorized citations. MW papers had a base rate of
10.8% but accounted for 10.2% of categorized citations.
最后, WW papers had a base rate of 14.9% but accounted
为了 7.6% of categorized citations.
重要的, this qualitative pattern is observed across
author “gender subgroupings” when papers are broken
out by author gender category, although MW papers have
a positive Gender Citation Balance Index for papers from
the MW and WW gender subgroups (数字 3).
数字 1. Gender breakdown in
JoCN authorship from 2009 到
2020. Proportion of JoCN papers
assigned to four categories:
men as the first and last author
(毫米; purple), women as the
first author and men as the last
作者 ( WM; darker green), 男人
as the first author and women
as the last author (MW; lighter
绿色的), and women as both
the first and last author ( WW;
salmon). For ease of comparison
across time, the proportions of
each category are indicated for
2009 (左边) 和 2020 (正确的).
Fulvio, Akinnola, and Postle
5
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
3
3
1
3
1
8
6
2
4
6
7
/
j
哦
C
n
_
A
_
0
1
6
4
3
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
数字 2. Gender Citation
Balance Indices for the
four gender categories of
peer-reviewed papers published
in JoCN. Error bars correspond to
自力更生 95% confidence
intervals.
数字 3. Gender Citation
Balance Indices for
peer-reviewed papers published
in JoCN, broken down by citing
papers’ gender category. Error
bars correspond to bootstrapped
95% 置信区间.
6
认知神经科学杂志
体积 33, 数字 1
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
3
3
1
3
1
8
6
2
4
6
7
/
j
哦
C
n
_
A
_
0
1
6
4
3
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
讨论
We measured the degree to which the categorical gender
balance of papers cited in JoCN from 2009 到 2020 关于-
flected the gender balance of the authorship of the jour-
nal during that timeframe. The results indicate that
papers authored by men as the first and last authors have
been overcited compared with what would be expected
based on the number of papers published by the journal
that were authored by “MM” teams. 相比之下, 文件
authored by teams with at least one woman in the first-
and/or last-author position have been undercited.
These findings indicate that the gender imbalance in
citation practices that was previously reported for broad-
scope neuroscience journals (Dworkin et al., 2020) extends
to the subfield of cognitive neuroscience. The fact that this
pattern of imbalance is present in JoCN papers published
by each of the four gender-defined groups that we have
considered here (毫米, WM, MW, and WW) 表明
this imbalance results, at least in part, from systemic factors
at play in the field overall.
Limitations
There are caveats to bear in mind when interpreting these
数据. A fundamental one is that it assumes that all scientists
self-identify within a gender-binary framework. 这是, 的
课程, not true and results in two limitations. 第一的, 它
introduces error into the estimates for authors who do
identify as female or male. 第二, it highlights that this
study does not speak to inequities faced by noncisgender
individuals and by members of the LGTBQ+ community.
Methodologically, our reliance on data from the five broad-
scope neuroscience journals may have somewhat skewed
our estimates of Gender Citation Balance Index toward
positive values for MM papers because the base rate of au-
thorship in those journals is more heavily weighted toward
MM than is the base rate for JoCN. Working in the opposite
方向, 然而, is the fact that our method included
removing self-citations. Because men self-cite at a higher
rate than do women (金等人。, 2017), including self-
citations would be expected to push the Gender Citation
Balance Index for MM papers further in the positive direc-
的. (It is worthy of note, 然而, that it is unlikely that
readers of journal papers somehow selectively “remove”
the influence of self-citations from their internal model of
gender balances in scientific publishing.)
结论
In carrying out these analyses, we deliberately limited our-
selves to the “simple first step” (比照. Dworkin et al., 2020) 的
quantifying and describing this phenomenon. 虽然
we lack the expertise to propose specific interventions that
may encourage prosocial behavior, it is our hope that this
work contributes, in some modest way, to social norm
消息传递 (Murrar, 坎贝尔, & Brauer, 2020) about the
need to address inequities in the way that we carry out and
communicate our science.
致谢
This work was supported by NIH grant MH064498. 我. A. received
support from National Science Foundation grant 1757785.
Reprint requests should be sent to Jacqueline M. Fulvio, Depart-
ment of Psychology, University of Wisconsin–Madison, 1202 瓦.
Johnson St., 麦迪逊, WI 53706, or via e-mail: jacqueline.fulvio@
wisc.edu.
Author Contributions
Jacqueline Fulvio: 概念化; 数据管理; 正式的
分析; 调查; 方法; 项目管理;
软件; 监督; 验证; 可视化; Writing –
Original Draft; Writing – 审查 & Editing. Ileri Akinnola:
数据管理; 形式分析; 调查; 方法;
可视化. Brad Postle: 概念化; Formal analy-
姐姐; 资金获取; 调查; 方法;
项目管理; 资源; 监督; 验证;
Writing – 审查 & Editing.
Funding Information
Jacqueline Fulvio, National Science Foundation (http://dx
.doi.org/10.13039/100000001), Grant number: 1757785.
Jacqueline Fulvio, 美国国立卫生研究院 (http://dx
.doi.org/10.13039/100000002), Grant number: MH064498.
参考
Connor, J. (2011). Google Scholar citations open to all. 谷歌
Scholar Blog. https://scholar.googleblog.com/2011/11/google
-scholar-citations-open-to-all.html.
Dion, 中号. L。, Sumner, J. L。, & 米切尔, S. 中号. (2018). Gendered
citation patterns across political science and social science
methodology fields. Political Analysis, 26, 312–327. DOI:
https://doi.org/10.1017/pan.2018.12
Dworkin, J. D ., Linn, K. A。, Teich, 乙. G。, Zurn, P。, 筱原, 右. T。,
& Bassett, D. S. (2020). The extent and drivers of gender
imbalance in neuroscience reference lists. 自然神经科学,
23, 918–926. DOI: https://doi.org/10.1038/s41593-020-0658-y,
PMID: 32561883
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
j
/
哦
C
n
A
r
t
我
C
e
–
p
d
我
F
/
/
/
/
3
3
1
3
1
8
6
2
4
6
7
/
j
哦
C
n
_
A
_
0
1
6
4
3
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Fairhall, A. L。, & Marder, 乙. (2020). Acknowledging female
voices. 自然神经科学, 23, 904–905. DOI: https://土井
.org/10.1038/s41593-020-0667-x, PMID: 32591766
Hirsch, J. 乙. (2005). An index to quantify an individual’s scientific
research output. 美国国家科学院院刊
科学, 美国。, 102, 16569–16572. DOI: https://doi.org/10
.1073/普纳斯.0507655102, PMID: 16275915, PMCID: PMC1283832
国王, 中号. M。, 伯格斯特罗姆, C. T。, Correll, S. J。, Jacquet, J。, & 西方, J. D.
(2017). Men set their own cites high: Gender and self-citation
across fields and over time. Socius: Sociological Research for a
Dynamic World, 3, 237802311773890. DOI: https://doi.org
/10.1177/2378023117738903
Murrar, S。, 坎贝尔, 中号. R。, & Brauer, 中号. (2020). Exposure to
peers’ pro-diversity attitudes increases inclusion and reduces
the achievement gap. Nature Human Behaviour, 4, 889–897.
DOI: https://doi.org/10.1038/s41562-020-0899-5, PMID:
32601460
Fulvio, Akinnola, and Postle
7
