CARTA
L’Affaire Strumia reveals troubling
gatekeeping values and outcomes
at Quantitative Science Studies
Kyle Siler
un acceso abierto
diario
École de bibliothéconomie et des sciences de l’information, Université de Montréal, Montréal, Québec (Canada)
Palabras clave: gatekeeping, peer review, quality control
Citación: Siler, k. (2021). L’Affaire
Strumia reveals troubling gatekeeping
values and outcomes at Quantitative
Science Studies. Quantitative Science
Estudios, 2(3), 1119–1122. https://doi.org
/10.1162/qss_c_00153
DOI:
https://doi.org/10.1162/qss_c_00153
Recibió: 16 Puede 2021
Aceptado: 5 Julio 2021
Autor correspondiente:
Kyle Siler
ksiler@gmail.com
Editor de manejo:
Staša Milojević
Derechos de autor: © 2021 Kyle Siler.
Publicado bajo Creative Commons
Atribución 4.0 Internacional
(CC POR 4.0) licencia.
La prensa del MIT
Much ink has already been spilled about Strumia (2021). Shortly after QSS agreed to publish the
artículo, Cassidy Sugimoto—current ISSI president and QSS editorial board member—told
Science the article was “methodologically flawed,” contained “several unsubstantiated claims,"
and “fails to meet the standards of the bibliometric community” (Chawla, 2019). Following
widespread criticism, QSS published four rejoinders—including from two additional editorial
board members—identifying substantial empirical flaws and unfounded claims in Strumia
(2021) (Andersen et al., 2021; Baile et al., 2021; Hossenfelder, 2021; Thelwall, 2021). This letter
is not intended to further relitigate Strumia (2021). En cambio, it focuses on subsequent editorial
statements by QSS editor-in-chief Ludo Waltman and MIT Press that reveal discomfiting philos-
ophies about academic gatekeeping, community building, and scientific communication.
waltman (2021) offers vacillating justifications for publishing Strumia (2021). Waltman en-
gages in hedging and backside-covering by vaguely claiming that he and the reviewers did not
necessarily agree with all of the article, yet still somehow believed the article was meritorious
enough to be published in QSS. This approach is reminiscent of the editors of the postmod-
ernist journal Social Text responding to publishing the infamous Sokal hoax with a mixture of
criticism and defense of the article, and a naïve hope that the ensuing debate about the piece
would be productive (robbins & ross, 1996). Waltman also leaves unanswered exactly what
he found meritorious about Strumia (2021), which is an important question, as the empirics,
theory and implications of the article were all widely criticized. Waltman’s statement is a
motte-and-bailey1, as Waltman can claim that he and QSS are merely engaging with “a diver-
sity of perspectives” (motte), while at the same time legitimating and propagating Strumia’s
puntos de vista (bailey). This is a particularly bad omen for QSS on gender and science issues, given
that a precedent of questionable quality control has now been established. Scholars working
on gender issues in science may now rightfully question what the editor-in-chief and some of
the QSS reviewer pool believes—or at least tolerates—when considering submission outlets.
Waltman’s judgment should also be questioned regarding his choice of peer reviewers. Es
decepcionante (and telling) that the peer reviewers Waltman selected refused to anonymously
release their peer review reports. If the QSS peer reviewers actually felt strongly about the merits
of their reviews and Strumia (2021), it is cowardly and unproductive of them to deny the
1 The motte-and-bailey is an informal fallacy coined by philosopher Nicholas Shackel (2005). The arguer
advances two positions: one modest (motte) and one controversial (bailey). After facing criticism for the
controversial claim, the arguer retreats to the modest position. The bailey goes unrefuted and the motte
can be defended by the arguer claiming critics are unreasonable for equating the motte with the bailey.
yo
D
oh
w
norte
oh
a
d
mi
d
F
r
oh
metro
h
t
t
pag
:
/
/
d
i
r
mi
C
t
.
metro
i
t
.
/
mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d
yo
F
/
/
/
/
2
3
1
1
1
9
1
9
7
0
7
5
8
q
s
s
_
C
_
0
0
1
5
3
pag
d
.
/
F
b
y
gramo
tu
mi
s
t
t
oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3
L’Affaire Strumia reveals troubling gatekeeping values
anonymous publication of reports. This refusal is also contrary to the spirit of QSS, which has
since venerably implemented a pilot program of online sharing of peer reviews. Following a
controversial 2020 Nature Communications article about gender and academic mentorship
(later retracted by the authors), releasing peer review reports revealed reservations of peer
reviewers (wessel, 2020), holding editors accountable for publication decisions and enabling
stakeholders to identify where mistakes may have occurred in the gatekeeping process.
There is a reasonable argument that it is best not to legitimate bad articles with attention.
Sin embargo, as QSS legitimated and published Strumia (2021), it is perhaps not possible to
ignore the article. Even retracted articles still get cited (Schneider et al., 2020). The institution-
alized legitimation of an article by an academic journal is socially powerful. De este modo, journal
gatekeeping is an important responsibility, even if postpublication debate is possible.
Publishing questionable articles based on the notion that they can be debated publicly in
the future risks devaluing the imprimatur of an academic journal.
Future citation and altmetric analyses of Strumia (2021) will be revealing. Will the article be
seriously used by reputable academics conducting solid research on gender issues, or will it
find favor with men’s rights activists, gender essentialists, and/or alt-right outlets? I believe the
latter possibility is more likely and entails irresponsible intellectual and social contributions by
QSS. Publishing Strumia (2021) also placed demands on scholars who spent considerable time
and effort rebutting a bad—if not also damaging—article. I would rather scholars spend their
time innovating and creating new science, as opposed to having to rebut bad articles2.
MIT Press appended a statement to Waltman’s editorial piece. I was also disappointed with
this response. MIT Press stated by publishing Strumia (2021) with critical rejoinders, ellos
“welcome the opportunity to publish these wide-ranging views,” hoping “this approach sheds
more light than heat on gender bias in the sciences.”3 This response is either naïve or disin-
genuous. Strumia has an infamous track record with his public conduct and statements about
gender and science, which precipitated his removal from European particle physics research
center CERN in 2019. Thousands of physicists signed a petition condemning Strumia’s work
and ideas (Banks, 2018). It is unlikely that someone with this history will generate “more light
than heat,” especially based on a widely panned QSS article. What light can be found in
publishing and legitimating a bellicose ideologue pushing ideas that women are inferior at
física, and physics was “invented and built by men”?
As a general principle, “wide-ranging views” are desirable. Academia tends to skew liberal
ideologically (Bruto, 2013), which can be an intellectual and sociopolitical liability.
Sin embargo, controversial perspectives require careful judgment, and can entail dangerous
slippery slopes. There are many “wide-ranging views” on topics such as the virtues of African
colonialism (Gilley, 2018), the competency of LGBT parents (Regnerus, 2012), eugenics
and the intellectual inferiority of people in developing countries (Lynn & Vanhanen, 2006),
and the public health and economic benefits of permitting COVID-19 spread to achieve herd
immunity (Kulldorff, Gupta, & Bhattacharya, 2020). That does not mean those “wide-ranging
views” should be promoted in any reputable academic journal! To analogize based on the title
of Waltman’s editorial response, understanding astronomy does not require the “diverse” and
“controversial” perspectives of Flat Earthers.
2 I would rather be doing other things than writing this letter.
3 No names were appended to the MIT Press statement, indicative of a lack of accountability. Presumiblemente,
specific humans conceived and wrote the statement, as opposed to the entirety of a large, anonymous
institución.
Estudios de ciencias cuantitativas
1120
yo
D
oh
w
norte
oh
a
d
mi
d
F
r
oh
metro
h
t
t
pag
:
/
/
d
i
r
mi
C
t
.
metro
i
t
.
/
mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d
yo
F
/
/
/
/
2
3
1
1
1
9
1
9
7
0
7
5
8
q
s
s
_
C
_
0
0
1
5
3
pag
d
.
/
F
b
y
gramo
tu
mi
s
t
t
oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3
L’Affaire Strumia reveals troubling gatekeeping values
Science is a human endeavor. Peer review is flawed (Herrero, 2006). None of us—whether
individuals or institutions—are perfect. Given the overwhelmingly negative response to
Strumia (2021), I am astounded that neither Waltman nor MIT Press publicly acknowledged
the mere possibility that mistakes or shortcomings may have occurred in peer review and
gatekeeping at QSS. The stubbornness—if not arrogance—in Waltman’s statement is disap-
apuntando. Mistakes are inevitable, but failing to acknowledge and learn from them means they
will likely be repeated.
A 2019 BBC article chronicling Strumia’s removal from CERN noted, “[Strumia’s] análisis
was quickly dismissed as being unscientific and incorrect by experts in the impact of research.
But Professor Strumia told BBC News that his analysis would eventually be proved correct if
scientific journals would allow him to publish his research” (Ghosh, 2019)4. Since QSS
allowed Strumia to publish his research, this bestows him and his ideas with a veneer of
academic legitimacy. The internet is vast; “wide-ranging” and controversial views—including
Strumia’s—are omnipresent and easily accessed. QSS does not have to endorse questionable
research and ideas to enable “controversial” debate.
Having published five issues in just over 1 año, QSS is still a new, developing institution.
Nascent periods are especially important for identity development and survival prospects of
new institutions (Aldrich & Fiol, 1994). QSS is not currently indexed in Scopus or the Web of
Ciencia, nor does it have an impact factor. The relative place of QSS in the hierarchy of similar
journals is uncertain5. There are many good people affiliated with QSS, but a successful
launch is not assured. Publishing a widely panned article is bad enough, but it is especially
perilous when a journal is still making influential first impressions. Growth and development
are path dependent. Early missteps can inflict exponential long-term damage.
Hartley et al. (2019) characterized academic journals as clubs. Institutional networks and
affiliations are powerful social signals. Now that QSS includes Strumia, and some questionable
values and leadership, will the journal be a desirable club to patronize? It is easy to play games
with “controversial” ideas when one’s social location inures one from negative conse-
quences of legitimating and propagating those ideas. Sin embargo, I would argue that Waltman
harmed his reputation as a gatekeeper, as well as QSS more broadly with his decisions and
responses regarding Strumia (2021).
CONFLICTO DE INTERESES
The author has no competing interests.
INFORMACIÓN DE FINANCIACIÓN
Funding was received from the Alfred P. Sloan Foundation (grant G-2020-12678).
4 Strumia’s apparent frustration with journal editors not “allowing” him to publish his research seems to sug-
gest that before he published in QSS, more prudent gatekeepers may have rejected his work.
5 Whether you like academic rankings or not, they influence journal reputations and are often directly linked
to the hiring, tenure, and promotion reward structures of universities. Higher rankings (and merely being
ranked at all) incentivize academics to submit better articles to journals. Por ejemplo, in the Chartered
Association of Business Schools (2018) Academic Journal Guide, proximate journals to QSS have a variety
of rankings, que van desde 1 (lowest) a 4* (highest): Política de investigación (4*), JASIST (3), cienciometria (2), Social
Studies of Science (2), Journal of Informetrics (1). Where—if at all—will QSS be ranked relative to those
journals in the next edition? Even outside formal rankings, journal reputations and hierarchies matter,
influencing where articles are—and are not—submitted.
Estudios de ciencias cuantitativas
1121
yo
D
oh
w
norte
oh
a
d
mi
d
F
r
oh
metro
h
t
t
pag
:
/
/
d
i
r
mi
C
t
.
metro
i
t
.
/
mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d
yo
F
/
/
/
/
2
3
1
1
1
9
1
9
7
0
7
5
8
q
s
s
_
C
_
0
0
1
5
3
pag
d
.
/
F
b
y
gramo
tu
mi
s
t
t
oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3
L’Affaire Strumia reveals troubling gatekeeping values
REFERENCIAS
Aldrich, h. MI., & Fiol, C. METRO. (1994). Fools rush in? The institutional
context of industry creation. Academy of Management Review,
19(4), 645–670. https://doi.org/10.5465/amr.1994.9412190214
andersen, j. PAG., Nielsen, METRO. w., & Schneider, j. W.. (2021).
Selective referencing and questionable evidence in Strumia’s paper
on “Gender issues in fundamental physics.” Quantitative Science
Estudios, 2(1), 254–262. https://doi.org/10.1162/qss_a_00119
Ball, PAG., Britton, t. B., Hengel, MI., Moriarty, PAG., Oliver, R. A., … Wade,
j. (2021). Gender issues in fundamental physics: Strumia’s biblio-
metric analysis fails to account for key confounders and confuses
correlation with causation. Estudios de ciencias cuantitativas, 2(1),
263–272. https://doi.org/10.1162/qss_a_00117
Banks, METRO. (2018). “Thousands of physicists sign letter condemning
‘disgraceful’ Alessandro Strumia gender talk.” https://physicsworld
.com/a/thousands-of-physicists-sign-letter-condemning-disgraceful
-alessandro-strumia-gender-talk/
Chartered Association of Business Schools. (2018). 2018 journal
ranking guide. https://charteredabs.org/academic-journal-guide
-2018/
Chawla, D. S. (2019). In decision certain to draw fire, journal will
publish heavily criticized paper on gender differences in physics.
https://www.sciencemag.org/news/2019/11/decision-certain
-draw-fire-journal-will-publish-heavily-criticized-paper-gender
Ghosh, PAG. (2019). CERN cuts ties with “sexist” scientist Alessandro
Strumia. https://www.bbc.com/news/science-environment
-47478537
Gilley, B. (2018). The case for colonialism. Academic Questions,
31(2), 167–185. https://doi.org/10.1007/s12129-018-9696-2
Bruto, norte. (2013). Why are professors liberal and why do conserva-
tives care? Cambridge: Prensa de la Universidad de Harvard. https://doi.org
/10.4159/harvard.9780674074484
Hartley, J., Potts, J., Montgomery, l., Rennie, MI., & Neylon, C.
(2019). Do we need to move from communication technology
to user community? A new economic model of the journal as a
club. Learned Publishing, 32(1), 27–35. https://doi.org/10.1002
/leap.1228
Hossenfelder, S. (2021). Analyzing data is one thing, interpreting it
otro. Estudios de ciencias cuantitativas, 2(1), 273–274. https://doi
.org/10.1162/qss_c_00116
Kulldorff, METRO., Gupta, S., & Bhattacharya, j. (2020). The Great
Barrington Declaration. https://fregger.com/ Blogs/Great
%20Barrington%20Declaration.pdf
Lynn, r., & Vanhanen, t. (2006). IQ and global inequality.
Augusta, Georgia: Washington Summit Publishers.
Regnerus, METRO. (2012). How different are the adult children of parents
who have same-sex relationships? Findings from the New Family
Structures Study. Social Science Research, 41(4),752–770.
https://doi.org/10.1016/j.ssresearch.2012.03.009, PubMed:
23017845
robbins, B., & ross, A. (1996). No title. https://physics.nyu.edu
/sokal/SocialText_reply_LF.pdf
Schneider, J., Hacer, D., Colina, A. METRO., & Whitehorn, A. S. (2020).
Continued post-retraction citation of a fraudulent clinical trial
informe, 11 years after it was retracted for falsifying data.
cienciometria, 125(3), 2877–2913. https://doi.org/10.1007
/s11192-020-03631-1
Shackel, norte. (2005). The vacuity of postmodernist methodology.
Metaphilosophy, 36(Abril), 295–320. https://doi.org/10.1111/j
.1467-9973.2005.00370.X
Herrero, R. (2006). Peer review: A flawed process at the heart of
science and journals. Journal of the Royal Society of Medicine,
99(4), 178–182. https://doi.org/10.1177/014107680609900414,
PubMed: 16574968
Strumia, A. (2021). Gender issues in fundamental physics: A biblio-
metric analysis. Estudios de ciencias cuantitativas, 2(1), 225–253.
https://doi.org/10.1162/qss_a_00114
Thelwall, METRO. (2021). Female contributions to high-energy physics
in a wider context: Commentary on an article by Strumia.
Estudios de ciencias cuantitativas, 2(1), 275–276. https://doi.org/10
.1162/qss_c_00118
waltman, l. (2021). Understanding gender differences in science
requires a diversity of perspectives, including controversial ones.
Estudios de ciencias cuantitativas, 2(1), 224. https://doi.org/10.1162
/qss_e_00115
wessel, l. (2020). After scalding critiques of study on gender and
mentorship, journal says it is reviewing the work. https://www
.sciencemag.org/news/2020/11/after-scalding-critiques-study
-gender-and-mentorship-journal-says-it-reviewing-work
Estudios de ciencias cuantitativas
1122
yo
D
oh
w
norte
oh
a
d
mi
d
F
r
oh
metro
h
t
t
pag
:
/
/
d
i
r
mi
C
t
.
metro
i
t
.
/
mi
d
tu
q
s
s
/
a
r
t
i
C
mi
–
pag
d
yo
F
/
/
/
/
2
3
1
1
1
9
1
9
7
0
7
5
8
q
s
s
_
C
_
0
0
1
5
3
pag
d
/
.
F
b
y
gramo
tu
mi
s
t
t
oh
norte
0
7
S
mi
pag
mi
metro
b
mi
r
2
0
2
3
