RESEARCH ARTICLE
Attracting new users or business as usual?
A case study of converting academic
subscription-based journals to open access
a n o p e n a c c e s s
j o u r n a l
TIK Centre for Technology, Innovation and Culture, University of Oslo
Lars Wenaas
Keywords: journal conversion, open access, science policy
Citation: Wenaas, L. (2021). Attracting
new users or business as usual? A
case study of converting academic
subscription-based journals to open
access. Quantitative Science Studies,
2(2), 474–495. https://doi.org/10.1162
/qss_a_00126
DOI:
https://doi.org/10.1162/qss_a_00126
Peer Review:
https://publons.com/publon/10.1162
/qss_a_00126
Received: 2 November 2020
Accepted: 8 February 2021
Corresponding Author:
Lars Wenaas
larswen@tik.uio.no
Handling Editor:
Ludo Waltman
Copyright: © 2021 Lars Wenaas.
Published under a Creative Commons
Attribution 4.0 International (CC BY 4.0)
license.
The MIT Press
ABSTRACT
This paper studies a selection of 11 Norwegian journals in the humanities and social sciences and
their conversion from subscription to open access, a move heavily incentivized by governmental
mandates and open access policies. By investigating the journals’ visiting logs in the period 2014–
2019, the study finds that a conversion to open access induces higher visiting numbers; all journals
in the study had a significant increase, which can be attributed to the conversion. Converting a
journal had no spillover in terms of increased visits to previously published articles still behind the
paywall in the same journals. Visits from previously subscribing Norwegian higher education
institutions did not account for the increase in visits, indicating that the increase must be accounted
for by visitors from other sectors. The results could be relevant for policymakers concerning the
effects of strict policies targeting economically vulnerable national journals, and could further
inform journal owners and editors on the effects of converting to open access.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
1.
INTRODUCTION
Scientific journals are the core vehicle for the dissemination of scientific knowledge and are central
in the movement to open access. The rationale behind open access is more and better utilization of
research, both outside and inside academia, and is increasingly a core element in science policy.
The engagement for open access has spawned numerous strategies, most of which are connected to
choices taken by the author, such as choosing an open access journal or depositing an article in a
repository. However, strategies involving journal owners could be more effective. One such strat-
egy is when publishers convert subscription-based journals to open access, which would leave the
researchers’ preference of publishing outlet largely unchanged. Converting a journal is usually a
decision made by the owners of the journal on a voluntary basis, but the financial incentives are
often weak, as academic journals are a lucrative business for publishers and often a necessary
source of income for societies. Much of the current debate about open access revolves around
the business model of publishers; however, this is a discussion centered on large international pub-
lishers and English language journal flagships. There are many important national journals in local
languages that are at the other end of the spectrum. These often rely on subsidies and therefore
represent deviations from the narrative of international publishers with excessively high revenues
(Larivière, Haustein, & Mongeon, 2015). Such journals have been given less attention in the liter-
ature, but play a prominent role in local and national research cultures, particularly within the social
sciences and humanities (SSH).
Attracting new users or business as usual?
In Norway, governmental mandates and open access policies have provided strong incen-
tives for journals to convert from subscription to open access. The journals covered in my inves-
tigation are all within the SSH and part of a subsidizing program offered by Research Council
Norway (RCN). The program was initially launched in 1991 with the rationale of securing a
scientific infrastructure for research on Norwegian culture and society and in the Norwegian
language. From 2017 and onwards, RCN has demanded conversion to open access to be eligible
for support, and in 2018 the Ministry of Research and Education launched a program with the
aim of filling the financial gap following the corresponding loss of subscription fees. These
events created strong incentives to convert to open access for the journals that were dependent
on financial support. The policies involved in this setup are concerned not only with the survival
of national journals and scientific infrastructure, but argue for the usefulness of knowledge made
available to the public, particularly research within SSH. My research aim is therefore to identify
some of the effects of converting a journal to open access under these circumstances: Does
converting a journal to open access account for increased readership and is it plausible that
new audiences are reached?
The objective of the study is not only to identify a general increase in visits in the journals, but
to investigate to what extent an increase in readership can be credited to the actual conversion
from subscription to open access, and whether opening up the journal has a spillover effect on
previously published articles still behind the paywall. To accommodate this, I have chosen a core
group of 11 journals which all converted in 2017 and kept their back list closed. A second
objective is to investigate whether an increase after a conversion happens at previously subscribing
institutions, and thus be able to argue for whether new audiences are reached. The research ques-
tion for this study is What are the effects of converting subscription-based journals to open
access in terms of visits?
The novelty lies in the opportunity to study one of the largest noncommercial systemic con-
versions of journals to open access, within a short identical time frame, with the same basic fund-
ing mechanism and policies and also belonging to the same publisher and thus the same
technical infrastructure. The decision to keep the back list closed after conversion further con-
tributes to isolating the effects of converting journals.
The structure of the paper is as follows: First, I review the literature on open access and in
particular on journal conversion. This includes how research within SSH fields is connected
to the concept of public value. Second, I outline the background for the support of
Norwegian SSH journals and how this was set in motion due to more attention to open access.
Finally, I present the methodology and empirical results before a concluding discussion.
2. LITERATURE REVIEW
Open access is connected to the usefulness of research in a very direct way by removing barriers
to utilizing the scientific literature. However, this is not only for the benefit of the academic com-
munity, but also for readers in the wider society. Because the prospects of value for other sectors
is a part of the argument for open access, and specifically a part of the policies involving the
journals of this study, it is relevant to discuss briefly the usefulness of the SSH and why and
how the policies in question connect these disciplines to open access.
2.1. The Public Value of Science
In general, funding of public research has always been justified by the prospects of a positive
societal outcome and some kind of return on investment (Sarewitz & Pielke, 2007). Most often
“return on investment” is associated with impacts in terms of innovation, products, and markets.
Quantitative Science Studies
475
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Even though research evaluations have concentrated on scientific and economic impacts, the
nonscientific and noneconomic impacts are equally important, often referred to as public value
(Bozeman & Sarewitz, 2011).
Public value is a softer term and something that escapes the metrics unless covered by eco-
nomic or other proxies in evaluation programs. In a defense of the value of the social sciences,
Brewer issues a warning against the idea of measuring impact by proxies in general; it constitutes
a threat to the social sciences because “counting the countable, because the countable can be
easily counted, renders the impact illegitimate” (Brewer, 2013, pp. 90–91). Brewer further
claims that the public value of the social sciences lies in their ability to shape society through
their investigation of it and use of that evidence as premises for the challenges and complex
problems of the 21st century. The social sciences should be seen as essential tools for the further
development of society, and Brewer proposes a reframing of the value concept into a “new pub-
lic value of social sciences” (Brewer, 2013, pp. 168–169).
This would require researchers to be more problem-oriented rather than discipline-oriented
and to engage in the “wicked problems” of society, for example by engaging in normative issues
such as fighting political extremism, lack of trust in authorities, or the undemocratic intrusion in
social media networks. The arts and humanities are also under the pressure of economic indi-
cators and metrics in science evaluation, and play a similar role in society as the social sciences.
Their value is not reflected in simplistic claims about intrinsic values where arts and humanities
research is good for its own sake, nor that the arts and humanities are worth billions of euros in
the cultural industry. Their value lies in their influence on a society’s capacity for change and
transformation (Benneworth, Gulbrandsen, & Hazelkorn, 2016).
The SSH often have a regional component. A recent study covering seven European countries
argues that multilingual publishing in national SSH journals keeps locally relevant research alive
with an increased potential for creating impact (Kulczycki, Guns et al., 2020). This aligns with
the Helsinki Initiative on Multilingualism in Scholarly Communication (Federation of Finnish
Learned Societies et al., 2019) which highlights the importance of language diversity in pub-
lishing and dissemination.
The public value impact of the SSH is a useful concept for describing the rationale behind the
arrangement of supporting SSH journals and eventually why the arrangement in 2017 made
open access mandatory. Open access supports the idea of public value, and functions as an
“enhancer” of dissemination and means of obtaining knowledge, not only to the academic
community, but to user groups outside academia. The Ministry of Research and Education
especially points to user groups such as schools, archives, libraries, museums, and employees
within health care, municipalities, public offices, and the private sector as beneficiaries (Ministry
of Education, 2017, p. 53).
2.2. Open Access
Open access comes with several interpretations, but it can be achieved in three main ways in
accordance with the Budapest Open Access Initiative (BOAI, 2002), the first and most cited of
three authoritative declarations at the beginning of the millennium. Green open access means
depositing a prior (usually peer-reviewed) version of a published article in a dedicated reposi-
tory, with free access after an embargo period, usually in the range of 6–24 months (Björk,
Laakso et al., 2014). Gold open access means that all articles (publisher’s PDF/version of record)
in a journal is immediately free to access on the internet. Expenses for publishing must then be
covered by other means than subscription, for example by an article processing charge (APC).
Hybrid open access is associated with making a single article immediately freely available in an
Quantitative Science Studies
476
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
otherwise subscription-based journal by paying an APC. APCs are common in gold open access
journals. Non-APC financed gold open access journals are often called diamond open access,
which generally is classified under gold open access. There are other approaches that make the lit-
erature freely available, which have led to concepts such as delayed open access (Laakso & Björk,
2013) and bronze open access (Piwowar, Priem et al., 2018). Delayed and bronze are often used
interchangeably, but are nevertheless regarded as problematic because they often lack proper li-
censing and may be removed from the internet at the discretion of the publisher. Lack of proper
licensing also blocks usage in commercial services and text and data mining and makes the legality
of such use unclear. Even if the status of these types as open access may be disputed, they provide for
a significant proportion of freely available literature, especially for users whose primary objective is
to read the literature rather than seek benefits in other more advanced ways.
The benefits of open access are often categorized in accordance with the vocabulary of
Bozeman and Sarewitz. For example, in an evidence-based review of the advantages of open
access, the scientific impact, economic impact, and public value find their counterparts in
academic impact, economic impact and societal impact (Tennant, Waldner et al., 2016).
Studies of the economic impact of open access have emphasized possible cost savings at
universities and the role of large international publishers and the high margins in the international
publishing business (Larivière et al., 2015). A Max Planck Institute white paper argues that there are
more than enough funds worldwide to both convert the entire global production of scientific articles
to open access and to save money at the same time, even with high average APCs (Schimmer &
Geshunhn, 2015). However, the economic impact of open access in terms of innovation has been
studied to a lesser extent, and evidence for economic benefits has been called “patchy and diverse,”
although there are indications of such benefits (Fell, 2019).
Open access policies often argue for a positive effect in terms of societal impact, but there are few
empirical studies on this topic in the review by Tennant et al. The few existing ones focus on
Altmetrics and advantages when considering exposure in social media, blogs, and other general
media outlets. Altmetrics can also serve as gauges for usage outside academia (Tennant et al.,
2016), but there is little evidence for the direct usefulness of academic literature for wider society.
However, the principal idea that scientific knowledge should be under as few restrictions as possi-
ble, recommends opening the literature for the general audience. Dalrymple (2003, p. 36) has ex-
pressed the view that “scientific knowledge in its pure form is a classic public good” and as such
should be openly available. A related approach is the taxpayer argument, pointing out the public’s
principal rights to access what they have financed (Suber, 2003). Even if there is a lack of empir-
ical studies of nonacademic usage of the literature, there are many user groups who are expected
to benefit from open access literature. Zuccala (2009) argues for potential of the academic liter-
ature for the “layperson” which can be released by digital access and an increasing understanding
of science. Open access can support lifelong learning, medical patients and supporting networks,
health advocates, environmentalist organizations, and NGOs (Tennant et al., 2016).
Studies of the academic benefits of open access have traditionally been important for open
access advocates, primarily to convince researchers that publishing in open access journals is
equally or more beneficial compared to subscription-based journals when considering academic
impact. Studies have largely focused on citations and have led to what has been summarized as
the open access citation advantage (OACA). SPARC Europe, a higher education membership
organization advocating open access, maintained a list of studies up until 2015, a list now
adjourned “since the citation advantage evidence has now become far more common knowl-
edge” (SPARC Europe, n.d.). This statement needs to be nuanced; in a large-scale study on the
prevalence of open access, Piwowar et al. (2018) find that gold open access literature performed
Quantitative Science Studies
477
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
below average in terms of citations, while green and hybrid accounted for a small, but significant
OACA. Apparently, the “clear citation advantage” for open access can be interpreted as a disad-
vantage for gold open access and remains partially contested.
A different aspect of academic benefits is increased readership, where one can find a down-
load advantage in most studies examining the transition to open access. Björk (2017) holds that
“there is no question that open access per se increases downloads” and increased readership is
considered an intrinsic benefit of open access (Laakso, Solomon, & Björk, 2016). A randomized
control study of 3,245 articles in 36 journals found a clear download advantage with a 115%
increase in HTML views and a 62% increase in downloads of PDFs (Davis, 2011). A review by
Davis and Walters (2011) also concludes with a clear download advantage, whereas a citation
advantage remains at best unclear. In two studies of Oxford University Press open access
models, findings about a hybrid journal are particularly interesting, as the journal contains both
closed and open articles. The study reported a 40% increase in online usage of articles when
comparing open access articles and articles still behind the paywall. Although the visits could
contain white noise in the form of search engines, the conclusion was that hybrid or optional OA
models appear to be showing a significant increase in usage (Bird, 2008, 2010). A study by
Piwowar, Priem, and Orr (2019) predicts that by 2025, 70% of all article views will be of the
44% proportion of the literature which is open access, compared to 52% of article views in
2019 on a 31% proportion of open access articles.
2.3. Converting Journals to Open Access
Although there is a large body of literature on open access, less attention has been given to
studies of converted or “flipped” journals, and none studying national journals in a similar con-
text to that of Norway as far as I have discovered. Even though estimates hold that almost 2,400
journals had converted up until 2011, research on this phenomenon has been scarce (Laakso
et al., 2016). Up until 2016 no systematic review of the literature on journal conversion existed, a
gap that led to a project proposal by Peter Suber at the library of Harvard in March 2015. The
final report was published in 2016 which included commentaries by a panel of experts
(Solomon, Laakso et al., 2016) and later published as a review article by the same authors
(Laakso et al., 2016).
Their main finding is that there is no single universal approach to conversion that will fit all
journals, but rather a range of considerations that dictates the method for each individual jour-
nal. The report lays out 15 different journal flipping scenarios grouped on whether the journal
applies APCs or not, with the argument that the business model that a journal would base its
existence on is an important aspect when considering whether or not to convert a journal.
The decision to convert is usually made by the publisher or the society in charge of the jour-
nal, while reasons for converting range from idealistic views of making scholarship more acces-
sible or (more pragmatically) increasing readership or raising the number of submissions and/or
their quality. Positioning the journal and enhancing its reputation can also be a motivation for
flipping. This is a strategy that finds some support in the literature; the increase in citation rates in
terms of journal impact factor was investigated in a set of 171 flipped journals by analyzing their
characteristics using Web of Science. The study found that flipping journals had a positive effect
on the journal’s impact factor, but no obvious citation advantage at the article level (Momeni,
Fraser et al., 2019). Two reports by a BioMed Central publisher found a notable positive effect on
citations and the impact factors for five journals that had moved to BioMed Central. The effect
was linked to whether the journals were already well established in the Journals Citation Report,
one of the most common sources of impact factor metrics (Busch, 2014a, 2014b).
Quantitative Science Studies
478
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
In some cases, an open access business model is seen as more economically viable, or in the
case of society journals, used to gain more independence from the publisher (Solomon et al.,
2016). Converting a journal is not necessarily a peaceful process; there are cases where the
editorial board have abandoned their publisher for not taking steps towards open access. This
was the case with Glossa, a journal in linguistics, where the editorial board had come from
Lingua (Rooryck, 2017) and Quantitative Science studies (QSS ), a journal in the field of sciento-
metrics, where the editorial board came from Journal of Informetrics (Waltman, Larivière et al.,
2020). The editorial boards of both journals stepped down when negotiations on open access
with their publisher Elsevier failed. Technically, these cases could be classified as something
different than journal conversion because the editorial board started new journals and the old
journals still exist, but they serve as interesting cases on the power structures in the context of
converting a journal. Of the conversion scenarios elaborated in the report by Solomon et al.
(2016), the Norwegian initiative is listed as an example under the heading of “National
Journal Subsidies” (Solomon et al., 2016, p. 64). However, as will be described in the next sec-
tion, the financial model chosen for the Norwegian initiative equally resembles the “Consortium
or Library Partnership Subsidy.” An example in this category is the Open Library of Humanities
(OLH), a university library membership consortium funding OLH’s 27 gold open access jour-
nals. There are relevant differences between the Norwegian initiative and the consortium model
approach, which lie partly at the policy level; the Norwegian initiative is based on a governmen-
tal intervention approach and there is a strong degree of pressure involved that is not present in
OLH. In addition, the initiative also upholds subsidies. Stakeholder pressure and monetary
subsidies are strong conversion facilitators, but although achieving financial security is of high
importance, securing subsidies is, in most cases, not a primary goal of a conversion. The
Norwegian initiative is mentioned with interest in the report:
Norway offers an extremely interesting case of public funding used as means of both en-
couraging and enabling national subscription journals to convert to OA. In Norway, there
are around 40 such journals, mostly of which are published in Norwegian. While some are
subscription journals, from now on, receiving funds will require that the journal becomes
fully OA after a transition period which was previously not the case. This alone will force
many of the journals to flip. (Solomon et al., 2016, p. 64)
The interest in the Norwegian initiative thus lies in the opportunity to investigate a set of
converted journals under special conditions; this interest also serves as a motivation for this
study.
Interestingly, a study found that flipping a journal to open access is not an irreversible process.
Matthias, Jahn, and Laakso (2019) found 152 cases of reverse flips from open access to subscrip-
tion in the last 13 years, where 62% of journals across a range of disciplines at one point con-
verted from subscription to open access and back to subscription. This exploratory study
suggests that there are tensions within the scholarly publishing system that need to be resolved
for a full transition to open access to happen.
2.4. History and Background of the Support to Norwegian SSH-Journals
Open access has acquired a strong political position in Norway. The government has empha-
sized open access in several white papers (Ministry of Education, 2009, 2013) and developed
national guidelines for open access (Ministry of Education and Research, 2017). The Research
Council of Norway (RCN) acquired an open access policy in 2009 and is also one of the original
Quantitative Science Studies
479
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
signatories of Plan S (cOAlition S, 2018). Norway has further focused on consortia negotiations
with international publishers to prepare for a transition to open access. These negotiations took
place with a considerable political investment by the (then) Norwegian Minister of Research and
Education.
An important milestone for this study is a white paper issued in 2017, where the ministry ini-
tiated a project with the aim of securing finances for Norwegian journals converting to open
access (Ministry of Education, 2017). The project was at that point the latest in a series of initia-
tives from the Norwegian government, in alignment with the political ambitions for open access
over the preceding 10 years and a direct follow-up to RCN’s policy of funding only open access
journals from 2017 onward.
The political commitment to open access around 2010 coincided with a significant trend in
falling subscription income for Norwegian academic journals. Even though readership for many
of the journals in the SSH fields in terms of downloads was on the rise due to electronic publish-
ing, subscription income was dropping. One example is a journal investigated in this study,
Tidsskrift for Samfunnsforskning ( Journal of Social Research) where the number of subscribing
institutions fell from over 900 to below 300 in the period between 1994 and 2012, while the
download rate tripled from approximately 6,000 in 2007 to 18,000 in 2011 (Sivertsen, 2013,
p. 26). In addition, there was a drop in private subscriptions.
Many of the journals were already in a vulnerable situation before the drop in subscriptions
and were thus eligible for financial support from RCN, a type of grant given as baseline funding to
approximately 40 journals. RCN commissioned a report to investigate how the two trends could
meet and to explore possibilities for making the journals open access and what route would be
suitable (Sivertsen, 2013). The report argued for the great potential of open access, as the jour-
nals published articles concerned with Norwegian society and culture, written mostly in
Norwegian. Some 47% of Norwegian articles within the humanities and 38% within the social
sciences are written in Norwegian (Sivertsen, 2013). This also heightened the potential for use
outside academia.
As a result, RCN announced in 2014 that from 2017 onward, only open access journals
would be included in the funding arrangement, and at the same time RCN offered one-time
grants to support the conversion of journals to open access. This effectively forced many of
the journals not already open onto an open access platform. It also created a predictable finan-
cial challenge, as the grants from RCN covered only 50% of the journals’ expenditures and the
already falling subscription income would disappear entirely when the journals converted.
The aforementioned white paper from 2017 launched a pilot project from 2018 to 2021, with
the (translated) name “Norwegian Open Journals in Humanities and Social Science” (NOJ-SSH).
This was primarily launched to investigate and test new financing models and to rewire the fi-
nancial streams supporting the journals (Ministry of Education, 2017). The arguments from the
report (Sivertsen, 2013) seem to have found resonance in the setup of the project, where the
journals should be gold open access without APCs and part of a national consortium funding
scheme. The applicants to the NOJ-SSH were subject to an evaluation by a committee consisting
of university rectors and deans and was based on quality alone, essentially allocating funds from
the top of the quality list for as long as the budget allowed.
In 2020 there were 281 active Norwegian journals in the authoritative journal list maintained
by the Norwegian Centre for Research Data (NSD). These are all eligible for the Result Based
Financing Scheme (RBO) in Norway, a prerequisite for applying for NOJ-SSH. A total of 41 jour-
nals applied to NOJ-SSH and 25 received grants.
Quantitative Science Studies
480
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
The research question chosen for this study asks about the effects of converting journals in terms
of visits. Visits are defined as a page views or PDF downloads of articles in the journal, according
to the Counter protocol, a protocol used by academic publishers for reporting consistent use of their
electronic resources (Project Counter, n.d.). Based on previous studies, one can expect a positive
development in the number of visits, which should come in addition to any existing positive trend.
Articles still behind the paywall are expected to behave as regular subscription-based journal arti-
cles, and not be affected by a conversion to open access, which happens at a later stage. Because
users at previously subscribing higher education institutions already had full access, the expectation
is that conversion of journals will have no substantial impact on visiting numbers for this user group.
The study therefore proposes the following hypotheses:
(cid:129) H1: Converting a journal to open access will cause an (additional) increase in visiting
numbers for the journal.
(cid:129) H2: Converting a journal will not cause an (additional) increase in visiting numbers for
previously published closed articles (back list).
(cid:129) H3: Converting a journal will not lead to a substantial (additional) increase in visiting
numbers for previous subscribers in the HE sector, compared to other user groups.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
3. METHODOLOGY
The effects on converted journals investigated in this study are limited to the quantifiable effects
expressed in visiting data in the period 2014–2019 for a selection of NOJ-SSH journals belong-
ing to the publisher Universitetsforlaget. Other features, such as citation-based metrics, were
difficult to obtain, as national native-speaking journals are rarely indexed in commercial indexes
such as Web of Science and Scopus. In addition, the relatively short time span involved would
not provide valid measures on journal metrics based on citation counts.
The overall strategy was the following:
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
1.
Identify the journals among the NOJ-SSH applicants that had flipped from subscription to
open access in 2017 and at the same time kept articles published before conversion closed.
2. Use the time of conversion as an intervention point and identify changes in visiting
numbers in the time series, using a Bayesian approach. A selection of subscription-based
journals from the same publisher was used as a control.
Investigate the visiting numbers for previously subscribing higher education institutions in
Norway and compare these with the overall visiting patterns.
3.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
3.1. Selection of Journals
Open access journals are either born open access or converted from subscription. The latter
group can be divided into two new groups: those that make available the journal’s entire backlog
of articles at the time of flipping and those whose previously published articles still are behind
the paywall after the time of conversion (Table 1). Among the 41 applicants, all journals except
Historisk Tidsskrift were already open access; they were either born open access or converted to
open access as part of the RCN scheme effective from 2017. Historisk Tidsskrift was still sub-
scription based at the time of application, but converted in 2018.
At the time of application, all journals were either registered in the Directory of Open Access
Journals (DOAJ), had submitted an application to DOAJ, or were planning to. All journals except
Quantitative Science Studies
481
Attracting new users or business as usual?
Born open access
Converted, closed back list
Converted, open back list
Total
Table 1. Distribution of the applicants
Applicants
8
Receiving grants
4
13
20
41
12
9
25
one used double blind peer review; one used single blind. All journals used some version of the
creative commons license, except Mediehistorisk Tidsskrift ( Journal of Media History) which
was still undecided.
The selection of journals in this study are the journals that kept their back list closed after
conversion, and thus provide two distinct methods of access within the same journal.
3.1.1. Data sources and selection of sample
In cooperation with the publisher Universitetsforlaget, the study obtained access to the journals’
visiting logs. This is Norway’s largest academic publisher, with 75 journals in its portfolio, and
with the highest number of both applicants (19) and recipients (16) of grants in NOJ-SSH. All 13
NOJ-SSH applicants with a closed back list are represented by Universitetsforlaget. It supplied
three interlinked data sources; (a) monthly number of total visits per journal; (b) yearly number of
visits, separated by year of publication; and (c) yearly total visits per journal for each subscribing
customer/institution.
The data sources also provided the opportunity for cross-validation and verification of calcu-
lations throughout the study.
In addition to having a closed back list, the selection of journals depended on the coverage of
a particular set of criteria, where the first four were also NOJ-SSH requirements. All journals
should
(cid:129) be a part of NSD’s authoritative journal list;
(cid:129) have nonambiguous ISSN, e-ISSN, and title;
(cid:129) have articles primarily in Norwegian;
(cid:129) operate within the fields of SSH, according to NSD’s list of channels;
(cid:129) have complete data on visiting numbers for 2014–2019. This excludes newly started jour-
nals, terminated journals or journals recently moved to or acquired from another publisher;
and
(cid:129) have converted from subscription to open access in 2017.
In addition to the group of converted journals (Table 2), a selection of Universitetsforlaget’s
subscription-based journals was chosen as a control group. Besides being subscription based,
the profile of the journals should be equal to that of the primary group and adhere to the above-
listed criteria, except for the last criterion regarding conversion date. Of Universitetsforlaget’s
75 journals, of which 40 are subscription based, 12 journals met the criteria (Table 3).
The initial inspection of visiting data for the two groups of journals showed an almost equal
development in average growth from 2014 to 2016, which is up to the point of conversion. The
average increase for the groups was calculated by finding the increase in percentage for each
journal (respectively from 2014 to 2016 and 2014 to 2019) and then calculating the average of
Quantitative Science Studies
482
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Journal
Edda
Heimen
Kunst og Kultur
Norsk antropologisk tidsskrift
Norsk filosofisk tidsskrift
Norsk litteraturvitenskapelig tidsskrift
Norsk statsvitenskapelig tidsskrift
Studia Musicologica Norvegica
Teologisk Tidsskrift
Tidsskrift for kjønnsforskning
Table 2.
Selection of converted journals
Discipline
Scandinavian studies
History
Art History
Anthropology
Philosophy and History of Ideas
Literature
Political science
Musicology
Theology and religion
Gender studies
Tidsskrift for samfunnsforskning
Interdisciplinary Social Sciences
Conversion year
2017
2017
2017
2017
2017
2017
2017
2017
2017
2017
2017
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Journal
Agora1
Arbeidsrett
Jussens Venner
Kirke og kultur
Lov og rett
Table 3.
Selection of subscription-based journals
Discipline
Literature
Law
Law
Theology and religion
Law
Norsk pedagogisk Tidsskrift
Education and Educational Research
Nytt Norsk Tidsskrift
Interdisciplinary Social Sciences
Praktisk økonomi og finans
Business and Finance
Skatterett
Tidsskrift for psykisk helsearbeid
Tidsskrift for Rettsvitenskap
Tidsskriftet Norges barnevern
Law
Social Work
Law
Social Work
all journals. Both groups of journals had an average increase of approximately 29% from 2014–
2016, while the average increase for the whole period showed a larger increase (304.36%) for
the group of converted journals. The group of subscription journals had a smaller average in-
crease throughout the whole period (81.03 %), but still accounted for a general tendency of
growth in visiting numbers for the journals (see Table 4).
1 Agora is owned by the publisher Aschehoug but hosted by Universitetsforlaget.
Quantitative Science Studies
483
Attracting new users or business as usual?
Table 4. Mean percentage increase in visiting numbers for the two groups of journals
Selection of 11 converted journals with a closed back list
Selection of 12 subscription-based journals
Average increase
2014–2016
29.7%
29.1%
Average increase
2014–2019
304.4%
81.0%
3.2. Measuring the Effect of Conversion on Visits
The approach to provide evidence for any causal effects of conversion on visiting numbers was
to design a treatment group (converted journals) and a control group (subscription-based jour-
nals) and perform the analysis with a Bayesian structural time series model (BSTS). This approach
collectively measured any effect of conversion in the group of converted journals.
3.2.1. Analysis using Bayesian structural time series models
A BSTS was chosen as it is a suitable method when a randomized control study design is unavail-
able and a common strategy applied where a synthetic control is needed for prediction
(Brodersen, Gallusser et al., 2015). The method is used within different disciplines, such as eco-
nomics (Poyser, 2019) and medicine (Kurz, Rehm et al., 2019; Vocht, Tilling et al., 2017), while
the authors of the method particularly suggest studies of market interventions (Brodersen et al.,
2015), which shares important similarities with this study. Examples given are advertising cam-
paigns with the goal of generating more website visits and consequently more downloads and
sales of software. The campaign would then act as an intervention and the control group would
be defined as the same product in a different market or a very similar product in the same market.
The strategy is to synthesize a counterfactual development in the intervention group (had the
intervention not taken place) under the assumption that the development would be equal or very
similar to the development in the nonintervention group.
The most critical assumption of the model is that the control group is unaffected by the inter-
vention in the treatment group. For example, if there were links or referrals between converted
journals and subscription journals that could positively influence visiting numbers in the control
group, this would violate this assumption. No such violations were found. The model also as-
sumes that the relationship between covariates and the treated time series remains stable
throughout the postperiod, as established during the preperiod. The study could not identify
any such violations. An important factor is that the data adheres to the Counter protocol, which
accounts for a reasonable degree of stability by removing spikes and sudden boosts created by
aggressive search engines etc.
3.2.2. Preparation of data: Creating synthetic megajournals
The treatment group was created by adding the visiting numbers in the group of converted jour-
nals with a closed back list together, and likewise, the control group was created by adding the
visiting numbers in the group of subscription-based journals. These two aggregations were fur-
ther refined into four different data sets, which are described in this section.
Aggregating visiting numbers of the two groups of journals was a strategy that was deemed
appropriate, as the journals shared important similarities. The journals in the treatment group
had all converted with some degree of pressure at approximately the same time and all journals
in both groups operated within the same scientific disciplines as defined by NOJ-SSH. They are
also all handled by the same publisher and served by the same technical platform, so all visiting
Quantitative Science Studies
484
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
numbers were thus counted by the same protocol. Visiting numbers included downloads of
PDFs and visits to HTML pages and contain little white noise (search engines, crawlers, etc.),
as these were removed according to the Code of Practice issued by Project Counter (Project
Counter, n.d.).
The aggregation strategy was also justified as the results conceptually resemble what in open
access terminology are denoted as megajournals. These are journals that publish articles from a
spectrum of disciplines, and as such do not belong exclusively to a specific scientific field.
Examples of megajournals are PLOS ONE and Open Library of Humanities.
There are differences in how popular journals are in terms of visits, with some journals ac-
counting for more visits than others. When summarizing visiting numbers, data may be skewed
and effects in larger journals may overshadow the effect in smaller journals and thus invalidate
the analysis. To control for this, a second data set aggregating the relative increase in visits in
percentage from 2014 onwards was created, with the aim of running the same analysis as with
the number of visits, with a relative perspective on the same phenomenon.
A third data set was created as a subset of the first aggregation of visiting numbers, but visiting
numbers were limited up to the year of flipping and included only data within the period 2014–
2016. The data set was used to validate the statistical method by creating a fictitious intervention
point in 2015-01 and used to measure the accuracy in the predictions.
The fourth and last data set, also based on the aggregating principle, was limited to visits to
articles published before 2017. The purpose was to analyze the effect on visits to articles pub-
lished before flipping and thus still behind the paywall. This data set differed from the previous
data sets by being designed by yearly summaries, rather than monthly.
To summarize the aggregation procedure, the resulting data sets were the following:
1. Set-1: Aggregated visiting numbers
Result: a synthetic megajournal of flipped journals and a synthetic megajournal of
subscription-based journals created by summarizing monthly visiting numbers in the
two groups.
2. Set-2: Aggregated relative increase
Result: a synthetic megajournal of flipped journals and a synthetic megajournal of
subscription-based journals created by summarizing relative monthly increase in per-
centage in the two groups.
3. Set-3: Aggregated visiting numbers, limited to visiting numbers before 2017.
Result: a synthetic megajournal of flipped journals and a synthetic megajournal of
subscription-based journals created by summarizing monthly visiting numbers before
2017 in the two groups.
4. Set-4: Aggregated visiting numbers, limited to articles published before flipping.
Result: a synthetic megajournal of flipped journals and a synthetic megajournal of subscription-
based journals created by summarizing yearly visiting numbers in the two groups limited to
articles produced before 2017.
3.2.3. Creating models
Four separate models were created for each of the four data sets in the following steps.
1.
Identify and estimate the effect of conversion by applying the BSTS method to the synthetic
(flipped) megajournal with summarized monthly visiting number.
Quantitative Science Studies
485
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
–
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
2. Confirm the effect with the same procedure applied to the synthetic (flipped) megajournal
with summarized relative increase.
3. Verify the method by applying the BSTS method on the synthetic (flipped) megajournal
with summarized monthly visiting number limited up to 2017, with a fictitious interven-
tion date in the period before flipping.
Identify and estimate the effect of flipping on the synthetic (flipped) megajournal limited
to yearly visiting numbers of articles produced before flipping.
4.
3.2.4. Visiting data for higher education institutions
To address the hypothesis about whether the increase can be accounted for by previous sub-
scribing institutions, a separate set of visiting logs for a selection of Norwegian higher education
institutions (HEIs) were used. Universitetsforlaget has a consortium agreement with 38
Norwegian institutional subscribers of which 29 are institutions in the HE sector. Twenty-five
HE institutions had complete data covering 2014–2019, after adding data from an additional
15 previous members of the consortium which at a later stage merged with one of the current
members. The visiting numbers for past members were used to adjust the counts for present
members to give a more realistic picture of visiting patterns, as incorporating new users from
a merging institution could boost visiting numbers at the time of the merger. The visiting numbers
are delineated by the institutions’ IP range; any computer connected to an institutional network
infrastructure is counted as belonging to that institution. This includes freely available computers
at the library or other multiuser access points.
The second step was to limit visiting data to the list of converted journals in Table 2, using the
same 11 journals as in the previous leg. The visiting numbers from the selection of HEIs was
added together, forming an HEI group, while a non-HEI group was calculated by subtracting
the visits by the HEI group from total visits, as examined in the first leg. The non-HEI group is
thus characterized by not containing consortia HEI-members and includes all other visits, in-
cluding from other countries.
4. RESULTS
All analysis was run in R, using the library CausalImpact, also developed by Brodersen et al.
(2015). The functions in the library construct the model and visualize the results in the plots
below, along with the reported statistics.
4.1. The Effect Using Number of Visits
The first section of the plot in Figure 1 has a solid line showing the actual visiting numbers of the
intervention group and a dotted line with a blue confidence interval, showing the predicted
number of visits had the intervention not taken place. The middle section shows the absolute
increase in visiting numbers (i.e., the surplus accommodated by the intervention), while the bot-
tom pane accumulates the response variable “visits.”
An estimate of the causal effect that the intervention had on the response variable (number of
visits) showed an increase of +64%, with a 95% confidence interval of [+57%, +71%]. This
should be interpreted as the increase in total visits for all 11 journals in the sample. The positive
effect observed by the intervention is unlikely to have been caused by random fluctuations. The
probability of obtaining this effect by chance is considered very small (Bayesian one-sided tail-
area probability p < .001), and the causal effect can be considered statistically significant.
Quantitative Science Studies
486
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Figure 1. The plot shows the effect of converting journals using number of visits and intervention
date 2017-01. The expected average number of monthly visits was 10,553, 95% CI [9,829, 11,299]
(cumulative number of visits is 379,914, 95,% CI [353,856, 406,775]), while the actual average
number of visits was 17,346 (cumulative number of visits was 624,472). The absolute effect was
6,793 visits (cumulative 244,558) leading to an estimated relative effect of 64% CI95 % [57%,
71%]. The probability of obtaining this effect by chance is very small (Bayesian one-sided tail-area
probability p < .001), with a posterior probability of a causal effect ¼ 99.9%.
4.1.1. The effect using relative increase
To control for any spurious effect by some journals overshadowing the effects in others, the same
analysis was run on the response variable “relative increase.” The actual values of the variable
bear little meaning: The idea of the exercise is to treat all journals as equal contributors to the
collective effect of the increase and remove the possibility of skewed data invalidating the anal-
ysis. The analysis primarily provides evidence as to whether the previous analysis can be trusted.
Figure 2 shows a very similar development to that of Figure 1. The positive effect in relative
increase observed by the intervention is unlikely to have been caused by random fluctuations.
Figure 2. The plot shows the effect of converting journals using the relative increase in the number
of visits and intervention date 2017-01, with a pattern very similar to that of Figure 1.
Quantitative Science Studies
487
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
The probability of obtaining this effect by chance is considered small (Bayesian one-sided tail-
area probability p < .001) and the causal effect can be considered statistically significant.
4.1.2. Verification of method, converting with a fictitious flipping date
To control for the predictive strength of the model, a test was performed with a modified version
of the data set used in the first analysis. The data were limited to only the number of visits before
the actual intervention and introduced a fictitious intervention date at 2015-01. The model was
run with the same parameters as the first analysis, with the expectation of a nonsignificant result
and a prediction of visits close to the actual number of visits.
As Figure 3 and the statistics show, with an acceptable degree of accuracy, that the prediction
line follows the actual numbers of visits reasonably well throughout the period, missing the target
of actual number of total visits by only two visits. The causal effect of the intervention is, as
expected, not statistically significant, with a posterior tail-area probability of p ¼ 0.49.
4.1.3.
Influence of flipping on visiting numbers for previously published articles
The final BSTS analysis was used to measure whether conversion of the journal yielded a positive
effect in more visits for articles within the same journals that were still behind a paywall (i.e.,
published before the time of flipping).
As in Figure 1, the first pane of Figure 4 has a solid line showing the actual visiting numbers of
the intervention group and a dotted line with a confidence interval showing the counterfactual
predicted number of visits. The lines fluctuate less than in the previous results, as the numbers
from the data source were arranged as yearly summaries rather than monthly. The middle sec-
tion shows the absolute increase in visiting numbers (i.e., the surplus accommodated by the in-
tervention), while the bottom pane accumulates the response variable “visits.” It appears that
there is a slight increase from 2018 onward with a relative effect of 4.3%; however, this increase
is not statistically significant, with a posterior tail-area probability of p ¼ .19 and a posterior
probability of a causal effect of 81%.
Figure 3. Verification of method. The plot shows the effect of a fictitious conversion of the journals
using intervention date 2015-01. The expected average number of monthly visits was 8,415, 95%
CI [7,607, 9,255] while actual average number of visits was 8,413. The estimated relative effect is
−0.02%, CI 95% [−10%, 9.6%]. The probability of obtaining this effect by chance is p ¼ 0.49 and
the causal effect is not statistically significant.
Quantitative Science Studies
488
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Figure 4. The plot shows the effect of conversion on visiting numbers for previously published
articles. The expected average number of yearly visits was 95,344, 95% CI [86,289, 104,509],
while the actual average number of visits was 99,446. The estimated relative effect is 4.3%, CI
95% [−5.3%, 14%]. The probability of obtaining this effect by chance is p ¼ .19 and the causal
effect is not statistically significant.
4.2. Visiting Numbers for Norwegian HEIs
In the exploratory plot of Figure 5, the two groups are shown for each journal. HEI-visits is shown
by the blue line and non-HEIs by the green. Included for reference is the line for total visits (dot-
ted) and the intercept line on the x-axis showing the approximate time of conversion (placed at
the beginning of 2017). Note that the y-axis is floating.
Initial inspection of the data shows that all journals largely exhibit the same pattern, with an
increase in non-HEI-visits (green line) in the same year the journal converted, whereas the HEI-
visits (blue line) appear largely unaffected. The line with HEI-visits appears relatively stable,
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 5. Visits per year for individual journals, segmented on HEIs and non-HEIs.
Quantitative Science Studies
489
Attracting new users or business as usual?
while non-HEIs have a higher degree of deviance. In particular, the drop for several journals
between 2015 and 2016 is unexpected, and remains unexplained after discussions with
Universitetsforlaget. A theory could be that the behavior is due to technical issues; institutions
may have changed their IP range without it being registered properly in Universitetsforlaget’s
systems or search engines may not have been properly excluded, but this remains speculative.
We have not been able to explain this behavior, nor confirm that they in fact are deviations.
4.2.1. The difference in HEI-visits and non-HEI-visits
To provide statistical evidence for the difference in HEI-visits and non-HEI-visits, the mean in-
crease in percentage per year for each journal was calculated. This was done by stepwise cal-
culating the increase in each journal from the previous year in the period 2014–2019 for both
groups. This procedure does not take into account the effect of conversion itself, but was deemed
sufficient as the aim is to provide evidence for a difference in increase between the two groups.
The mean increase for the two groups per journal is presented in Table 5.
Statistics are given by the sign-test, and the analysis was run in R using the rstatix library. The
sign test was chosen because the distribution of differences between paired data values were not
normally distributed, which is the assumption of the paired t-test. Transformation of data was not
sufficient to satisfy this assumption. Further, the data are not symmetrical around the median,
which is a requirement for the paired samples Wilcoxon test.
The null hypothesis for the test is that the median of the paired differences equals zero. The
median increase in percentage per year for HEI-visits is significantly different from the median
increase for non-HEI-visits, p < .001 (Figure 6).
4.3. Summary of Findings
The group of converted journals with a closed back list shows a statistically significant increase
in visiting numbers following flipping. Collectively, the increase is estimated to be a 64% in-
crease in the total number of visits. The effect is confirmed when using relative increase as
Table 5. Mean increase for journals 2014–2019, differentiated on HEIs and non-HEIs
Journal
Edda
Heimen
Kunst og Kultur
Norsk Antropologisk Tidsskrift
Norsk Filosofisk Tidsskrift
Norsk Litteraturvitenskapelig Tidsskrift
Norsk Statsvitenskapelig Tidsskrift
Studia Musicologica Norvegica
Teologisk Tidsskrift
Tidsskrift for Kjønnsforskning
Tidsskrift for Samfunnsforskning
Mean increase in % per year
(2014–2019) for HE institutions
19.4
Mean increase in % per year
(2014–2019) for non-HE institutions
48.1
39.0
20.9
31.1
15.6
19.1
7.6
12.0
25.1
28.4
16.49
85.6
35.6
36.0
45.1
48.1
38.1
41.7
93.7
44.2
59.04
Quantitative Science Studies
490
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Figure 6. The plot shows the mean yearly increase in visits grouped by HEIs and non-HEIs. The group of non-HEIs has a higher increase per
year than the group of HEIs.
the dependent variable and the method was validated by applying the same method to a limited
data set with fictitious intervention data.
The group of converted journals with a closed back list limited to articles published before the
time of conversion showed no significant increase in visiting numbers. There was no statistically
significant effect of conversion.
The identified general increase in visiting numbers is not accounted for by previously sub-
scribing higher education institutions.
Hypotheses 1, 2, and 3 are therefore confirmed.
5. DISCUSSION
In essence, this study provides evidence that the conversion of journals to open access causes an
increase in visits for journals, and further that this increase takes place outside previously sub-
scribing HEIs. Although the effect was expected, it is confirmed under conditions where local
journals are in need of funds to secure continued operation and where policy mandates exercise
heavy pressure. These results correspond with some of the main open access arguments: Open
access generates more readership, partly by granting access to users outside academia (Tennant
et al., 2016).
The importance of the results lies partly in the strategic choices made to accommodate open
access and partly in the consideration of the arguments for the alleged benefits of open access.
First, open access has become an important part of research policies, with many strategies to
accommodate access to the literature for the wider research community and for society in gen-
eral. Strategies range from local initiatives at institutional libraries up to national mandates and
international collaboration between research councils. Strategies for changing the profile of the
journal, rather than changing the researcher’s behavior, therefore serve as interesting additions
to the policymaking toolbox. This is the case with Plan S, which has issued a report on conver-
sion of journals (Wise & Estelle, 2019) or recent initiatives in Finland on sustainable transitions to
Quantitative Science Studies
491
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
open access (Ilva, 2018). This study has the ambition of making a contribution to our under-
standing of this toolbox.
Second, the study has in general been motivated by claims about the economic, societal, and
academic benefits of open access and whether these can be justified. Clearly, any economic
benefits of open access are less relevant in this context. The journals are depended on baseline
funding and thus at the other end of the income scale compared to large international publishers
(Larivière et al., 2015). Hence, primarily this study contributes to the discussion about whether it
is likely that open access facilitates societal and academic impact by increased readership and
new user groups.
The results are consistent with and confirm the general visits and downloads advantages
found in previous studies (Davis, 2011; Davis & Walters, 2011; Piwowar et al., 2019) and ex-
tends this research with knowledge about where the effect takes place. The results also expand
on the flipping scenarios described in the review on journal conversion, particularly the scenar-
ios of “National Journal Subsidies” and “Consortium or Library Partnership Subsidy” (Laakso
et al., 2016; Solomon et al., 2016) which has motivated this study. The report by Solomon
et al. was not intended as an analysis of the effects of flipping, but rather on how conversion
takes place. This study connects the general visits and downloads advantage with the review’s
scenarios and methods for journal conversion.
There are several limitations to the study. The unit of data in this study is “visits” but it remains
a challenge to understand what “visits” is ultimately a proxy for. What is sought by authors and
journals is readership and impact, but there is at best an unclear link between visits and reader-
ship, let alone impact. Visits is a volatile concept and may include everything from a 2-second
glance at a webpage to downloading a PDF and thorough reading. If converting a journal to
open access and a subsequent rise in visits primarily results in the former type of behavior, then
the advantages of conversion are clearly diminished. However, one can assume a correlation
between the number of visits and actual readership, albeit not a one-to-one relationship.
Further, visits are essentially counts of requests from computers identified by their IP address,
so there is need for additional assumptions to conclude on the origin of visits. The Counter pro-
tocol is the de facto standard for counting visits in academic journals, but, naturally it does not
reveal who sits in front of the computer or how many users the computer serves. Before journal
conversion, HEI users had access to the journals exclusively by institutional subscription
schemes. After conversion, the users could also access the literature from other computers, in-
cluding at home. Likewise, nonacademic users had the opportunity to access the journals from
campus computers by visiting the library physically and thus be counted as HEI-visits. In addi-
tion, academic users in other Scandinavian countries are not accounted for in the non-HEI-visits
group; a small part of the increase can most likely be attributed to them, because language sim-
ilarities make it relatively easy to read Norwegian literature.
These factors are probable sources of error and make it difficult to translate visits directly into
readership, to draw conclusions about the effect-size of visits, or to estimate precisely the types of
users, their numbers, and the share of visits. However, even if the extent of these factors is
unknown, it can be argued that it is highly unlikely that they account for the increase per se:
The 64% increase is too large to be explained by these factors alone.
The investigation has neither been able to identify whether as a general rule all open articles
in a journal are visited more often, or if open access accommodates a kind of Matthew effect. The
increase in visits is measured at the journal level only, but could be skewed at the article level.
Articles attracting visits could attract more visits, while articles that do not get attention in the first
place remain largely untouched.
Quantitative Science Studies
492
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
A final issue is to what extent the effects found in the study can be extended to other journals.
The journals in the study are all local journals in the same disciplinary fields and in the same
subsidiary program; as such, they belong to a rather exclusive selection of journals. The strength
of the study design is, in my view, that it accounts for all other variables under the assumption
that they influence the journals in the control group and treatment group equally. However, this
assumption can also be considered a limitation, as the study design does not reveal unknown
influencing variables. These variables may be both properties of the journal or external factors,
and as such a generalization to journals outside the selection is not automatically appropriate.
For example, the potential of increased readership by converting a journal is most likely inti-
mately connected to the topics of the journals. Very specialized journals in a local language will
likely have a much smaller effect.
The limitations of the study are also a good starting point for some of the additional work that
needs to be done. Later research might want to look more specifically into users in the private
and public sector and whether they represent research organizations or other types of users. New
user groups are clearly a part of the motivation for the NOJ-SSH arrangement, which was
launched with the vision that a “significant part of research in the Norwegian language would
be immediately available for relevant user groups both within and outside academia” (Ministry
of Education, 2017, p. 54). This particular policy rests on the concept of public value. Perhaps
the most interesting aspect of open access for later research is the usefulness of the literature for
the users. How do new users engage with the literature and for what purpose? Do users actively
pursue the scientific literature or do they accidently stumble upon it? Later research could also
investigate if the literature plays a more prominent role in the public sphere with open access, in
terms of not only increased readership in new user groups but increased public value.
ACKNOWLEDGMENTS
I would like to extend my gratitude to Nils Olav Lahlum and Håkon Pagander at
Universitetsforlaget for supplying data and offering their expertise. The study has also benefited
from insightful advice and comments from Mikael Laakso and from colleagues at the TIK-center
and the Osiris research group, particularly Magnus Gulbrandsen. I also thank Henrik Karlstrøm
for valuable input regarding coding and two anonymous reviewers for valuable feedback.
Nevertheless, the author remains responsible for any mistakes still present.
COMPETING INTERESTS
The author is a PhD candidate at the TIK-center at the University of Oslo while holding a position
at UNIT, a governmental body reporting to the Ministry of Education and Research. The position
is a part of the department with responsibility for coordinating open access affairs in Norway,
which includes the administration of the NOJ-SSH project.
FUNDING INFORMATION
The PhD project is financed by the Research Council Norway (RCN-project: 272456) and is
also a part of the Oslo Institute for Research on the Impact of Science (OSIRIS, RCN-project:
256240/O30).
DATA AVAILABILITY
The data set has been made available by agreement with Universitetsforlaget. Data and repro-
ducible code: Wenaas (2021).
Quantitative Science Studies
493
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
.
/
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
REFERENCES
Benneworth, P., Gulbrandsen, M., & Hazelkorn, E. (2016). The im-
pact and future of arts and humanities research. Cham: Springer.
DOI: https://doi.org/10.1057/978-1-137-40899-0
Bird, C. (2008). Oxford Journals’ adventures in open access.
Learned Publishing, 21(3), 200–208. DOI: https://doi.org/10
.1087/095315108X288910
Bird, C. (2010). Continued adventures in open access: 2009 perspec-
tive. Learned Publishing, 23(2), 107–116. DOI: https://doi.org/10
.1087/20100205
Björk, B.-C. (2017). Open access to scientific articles: A review of
benefits and challenges. Internal and Emergency Medicine, 12,
247–253. DOI: https://doi.org/10.1007/s11739-017-1603-2,
PMID: 28101848
Björk, B.-C., Laakso, M., Welling, P., & Paetau, P. (2014). Anatomy
of green open access. Journal of the Association for Information
Science and Technology, 65, 237–250. DOI: https://doi.org/10
.1002/asi.22963
BOAI. (2002). Budapest Open Access Initiative | Read the Budapest
Open Access Initiative. https://www.budapestopenaccessinitiative
.org/read
Bozeman, B., & Sarewitz, D. (2011). Public value mapping and
science policy evaluation. Minerva, 49(1), 1–23. DOI: https://doi
.org/10.1007/s11024-011-9161-7
Brewer, J. D. (2013). The public value of the social sciences: An
interpretive essay. London: A&C Black.
Brodersen, K. H., Gallusser, F., Koehler, J., Remy, N., & Scott, S. L.
(2015). Inferring causal impact using Bayesian structural time-
series models. Annals of Applied Statistics, 9(1), 247–274. DOI:
https://doi.org/10.1214/14-AOAS788
Busch, S. (2014a). The careers of converts—How a transfer to
BioMed Central affects the Impact Factors of established journals.
Research in Progress Blog, January 15. http://blogs.biomedcentral
.com/bmcblog/2014/01/15/the-careers-of-converts-how-a-transfer
-to-biomed-central-affects-the-impact-factors-of-established
-journals/
Busch, S. (2014b). The Impact Factor of journals converting from sub-
scription to open access. Research in Progress Blog, November 6.
https://blogs.biomedcentral.com/bmcblog/2014/11/06/the-impact
-factor-of-journals-converting-from-subscription-to-open-access/
cOAlition S. (2018). “Plan S” and “cOAlition S”—Accelerating the
transition to full and immediate Open Access to scientific publi-
cations. https://www.coalition-s.org/
Dalrymple, D. (2003). Scientific knowledge as a global public good:
Contributions to innovation and the economy. The Role of Scientific
and Technical Data and Information in the Public Domain:
Proceedings of a Symposium (pp. 35–49). http://hdl.handle.net
/10919/66532
Davis, P. M. (2011). Open access, readership, citations: A randomized
controlled trial of scientific journal publishing. FASEB Journal,
25(7), 2129–2134. DOI: https://doi.org/10.1096/fj.11-183988,
PMID: 21450907
Davis, P. M., & Walters, W. H. (2011). The impact of free access to
the scientific literature: A review of recent research. Journal of the
Medical Library Association, 99(3), 208–217. DOI: https://doi
.org/10.3163/1536-5050.99.3.008, PMID: 21753913, PMCID:
PMC3133904
Federation of Finnish Learned Societies, The Committee for Public
Information, The Finnish Association for Scholarly Publishing,
Universities Norway, & European Network for Research Evaluation
in the Social Sciences and the Humanities. (2019). Helsinki Initiative
on Multilingualism in Scholarly Communication. https://figshare
.com/articles/Helsinki_Initiative_on_Multilingualism_in_Scholarly
_Communication/7887059
Fell, M. J. (2019). The economic impacts of open science: A rapid
evidence assessment. Publications, 7(3), 46. DOI: https://doi.org
/10.3390/publications7030046
Ilva, J. (2018). Looking for commitment: Finnish open access jour-
nals, infrastructure and funding. Insights, 31, 25. DOI: https://doi
.org/10.1629/uksg.414
Kulczycki, E., Guns, R., Pölönen, J., Engels, T. C. E., Rozkosz, E. A.,
… Sivertsen, G. (2020). Multilingual publishing in the social sci-
ences and humanities: A seven-country European study. Journal of
the Association for Information Science and Technology, 71(11),
1371–1385. DOI: https://doi.org/10.1002/asi.24336, PMID:
33288998, PMCID: PMC7687152
Kurz, C. F., Rehm, M., Holle, R., Teuner, C., Laxy, M., &
Schwarzkopf, L. (2019). The effect of bariatric surgery on health
care costs: A synthetic control approach using Bayesian structural
time series. Health Economics, 28(11), 1293–1307. DOI: https://
doi.org/10.1002/hec.3941, PMID: 31489749
Laakso, M., & Björk, B.-C. (2013). Delayed open access: An over-
looked high-impact category of openly available scientific litera-
ture. Journal of the American Society for Information Science and
Technology, 64(7), 1323–1329. DOI: https://doi.org/10.1002
/asi.22856
Laakso, M., Solomon, D., & Björk, B.-C. (2016). How subscription-
based scholarly journals can convert to open access: A review of
approaches. Learned Publishing, 29(4), 259–269. DOI: https://
doi.org/10.1002/leap.1056
Larivière, V., Haustein, S., & Mongeon, P. (2015). The oligopoly of
academic publishers in the digital era. PLOS ONE, 10(6),
e0127502. DOI: https://doi.org/10.1371/journal.pone.0127502,
PMID: 26061978, PMCID: PMC4465327
Matthias, L., Jahn, N., & Laakso, M. (2019). The two-way street of open
access journal publishing: Flip it and reverse it. Publications, 7(2),
23. DOI: https://doi.org/10.3390/publications7020023
Ministry of Education. (2009). St.meld. Nr. 30 (2008–2009). https://
www.regjeringen.no/no/dokumenter/stmeld-nr-30-2008-2009
-/id556563/
Ministry of Education. (2013). Meld. St. 18 (2012–2013). https://
www.regjeringen.no/no/dokumenter/meld-st-18-20122013
/id716040/
Ministry of Education. (2017). Meld. St. 25 (2016–2017). https://
www.regjeringen.no/no/dokumenter/meld.-st.-25-20162017
/id2545646/
Ministry of Education and Research. (2017). National goals and
guidelines for open access to research articles [Retningslinjer].
https://www.regjeringen.no/en/dokumenter/national-goals-and
-guidelines-for-open-access-to-research-articles/id2567591/
Momeni, F., Fraser, N., Peters, I., & Mayr, P. (2019). From closed to
open access: A case study of flipped journals. ArXiv:1903.11682
[Cs]. http://arxiv.org/abs/1903.11682
Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., …
Haustein, S. (2018). The state of OA: A large-scale analysis of the
prevalence and impact of Open Access articles. PeerJ, 6, e4375.
DOI: https://doi.org/10.7717/peerj.4375, PMID: 29456894,
PMCID: PMC5815332
Piwowar, H., Priem, J., & Orr, R. (2019). The future of OA: A large-
scale analysis projecting Open Access publication and reader-
ship. BioRxiv, 795310. DOI: https://doi.org/10.1101/795310
Poyser, O. (2019). Exploring the dynamics of Bitcoin’s price: A
Bayesian structural time series approach. Eurasian Economic
Quantitative Science Studies
494
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Attracting new users or business as usual?
Review, 9(1), 29–60. DOI: https://doi.org/10.1007/s40822-018
-0108-2
Project Counter. (n.d.). Project Counter. Retrieved January 31,
2020, from https://www.projectcounter.org/about/
Rooryck, J. (2017). Lingua to Glossa. Rooryck.Org. https://www
.rooryck.org/lingua-to-glossa
Sarewitz, D., & Pielke, R. A. (2007). The neglected heart of science
policy: Reconciling supply of and demand for science. Environmental
Science & Policy, 10, 5–16. DOI: https://doi.org/10.1016/j.envsci
.2006.10.001
Schimmer, R., & Geshunhn, K. (2015). Disrupting the subscription
journals’ business model for the necessary large-scale transforma-
tion to open access: A Max Planck Digital Library Open Access
Policy White Paper. https://www.scienceopen.com/document?
id=b2341b73-1e0e-4b6f-8ef0-15620638e1ba
Sivertsen, G. (2013). Norskspråklige vitenskapelige tidsskrifter
i humaniora og samfunnsvitenskap: Forfatterkrets, formål og
mulighet for åpen tilgang (p. 36). https://www.nifu.no/publications
/1055091/
Solomon, D., Laakso, M., Björk, B.-C., & Suber, P. (2016). Converting
scholarly journals to open access: A review of approaches and
experiences. https://digitalcommons.unl.edu/scholcom/27
SPARC Europe. (n.d.). The Open Access Citation Advantage
Service (OACA). SPARC Europe. Retrieved May 12, 2018 from
https://sparceurope.org/what-we-do/open-access/sparc-europe
-open-access-resources/open-access-citation-advantage-service
-oaca/
Suber, P. (2003). The taxpayer argument for open access. https://
dash.harvard.edu/handle/1/4725013
Tennant, J. P., Waldner, F., Jacques, D. C., Masuzzo, P., Collister,
L. B., & Hartgerink, C. H. (2016). The academic, economic and
societal impacts of Open Access: An evidence-based review.
F1000Res, 5, 632. DOI: https://doi.org/10.12688/f1000research
.8460.3, PMID: 27158456, PMCID: PMC4837983
Vocht, F. de, Tilling, K., Pliakas, T., Angus, C., Egan, M., …
Hickman, M. (2017). The intervention effect of local alcohol
licensing policies on hospital admission and crime: A natural
experiment using a novel Bayesian synthetic time-series method.
Journal of Epidemiol & Community Health, 71(9), 912–918. DOI:
https://doi.org/10.1136/jech-2017-208931, PMID: 28679538,
PMCID: PMC5561361
Waltman, L., Larivière, V., Milojevic(cid:1), S., & Sugimoto, C. R. (2020).
Opening science: The rebirth of a scholarly journal. Quantitative
Science Studies, 1(1), 1–3. DOI: https://doi.org/10.1162/qss_e_00025
Wenaas, L. (2021). Replication data for: Attracting new users or
business as usual? A case study of converting academic subscrip-
tion based journals to open access [Data set]. DataverseNO.
DOI: https://doi.org/10.18710/FUPZIY
Wise, A., & Estelle, L. (2019). Society publishers accelerating open
access and Plan S (SPA OPS) (p. 83). DOI: https://doi.org/10
.6084/m9.figshare.c.4561397.v3
Zuccala, A. (2009). The layperson and open access. Annual Review
of Information Science and Technology, 43, 359–396. DOI:
https://doi.org/10.1002/aris.2009.1440430115
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
q
s
s
/
a
r
t
i
c
e
-
p
d
l
f
/
/
/
/
2
2
4
7
4
1
9
3
0
7
5
0
q
s
s
_
a
_
0
0
1
2
6
p
d
/
.
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Quantitative Science Studies
495
