Subjectivity Predicts Adjective
Ordering Preferences
Gregory Scontras1, Judith Degen2, and Noah D. Goodman2
1语言学系, 加州大学, 尔湾
2心理学系, 斯坦福大学
关键词: adjective ordering, 主观性, 语义学, 语言, 认识
开放访问
杂志
抽象的
From English to Hungarian to Mokilese, speakers exhibit strong ordering preferences in
multi-adjective strings: “the big blue box” sounds far more natural than “the blue big box.”
We show that an adjective’s distance from the modified noun is predicted not by a rigid
syntax, but by the adjective’s meaning: less subjective adjectives occur closer to the nouns
they modify. This finding provides an example of a broad linguistic universal—adjective
ordering preferences—emerging from general properties of cognition.
介绍
Regularities in the behavior of speakers and speech communities provide a window onto the
psychology of language. Here we take up one such regularity: adjective ordering. Speakers
and listeners exhibit strong ordering preferences when two or more adjectives are used to
modify a noun, as in “the big blue box” or “the good smooth purple plastic chair.” Deviate
from the preferred order, and the construction becomes odd. Something feels particularly
unwieldy about “the blue big box,” even more so with “the plastic good purple smooth chair.”
Why do most strings of adjectives have tightly constrained order? We investigate the role of
adjective meaning, specifically the subjectivity of the properties that the adjectives name, 在
predicting ordering preferences.
Adjective ordering preferences stand as a particularly striking case of regularity in
语言. More remarkable than their robustness in English is their cross-linguistic system-
aticity: we continually find the same preferences across the world’s languages. Hungarian
(Uralic), Telugu (Dravidian), Mandarin Chinese, and Dutch are just a handful of languages
with pre-nominal adjectives (IE。, languages where adjectives precede nouns) reported to
have the same ordering preferences as English (狄克逊, 1982; Hetzron, 1978; LaPolla &
黄, 2004; 马丁, 1969乙; Sproat & Shih, 1991). In languages like Selepet (Papuan)
and Mokilese (Micronesian) with post-nominal adjectives (IE。, where adjectives follow
nouns), these preferences are preserved in the reverse (狄克逊, 1982; Hetzron, 1978; Sproat
& Shih, 1991)—stable preferences determine the linear distance of an adjective from the
noun it modifies.
There have been two general approaches to the investigation of adjective ordering pref-
erences. As part of a larger project mapping the syntax and semantics of adjectives, the lin-
guistics literature advances a universal hierarchy of semantic classes of adjectives. Leading
the charge, 狄克逊 (1982) set out to uncover language-internal structure by which to organize
ordering preferences. The preferences were assumed to be hard-coded in the grammar; 这
researcher’s job was simply to uncover them. Building on the ordering of semantic classes
引文: Scontras, G。, Degen, J。, &
古德曼, 氮. D. (2017). Subjectivity
predicts adjective ordering
优先. 开放的心态: Discoveries
in Cognitive Science, 1(1), 53–65.
土井:10.1162/opmi_a_00005
DOI:
http://doi.org/10.1162/opmi_a_00005
补充材料:
http://www.mitpressjournals.org/doi/
suppl/10.1162/opmi_a_00005
已收到: 26 二月 2016
公认: 11 十月 2016
利益争夺: 作者
declared that they have no competing
兴趣.
通讯作者:
Gregory Scontras
g.scontras@uci.edu
版权: © 2017
麻省理工学院
在知识共享下发布
归因 4.0 国际的
(抄送 4.0) 执照
麻省理工学院出版社
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
proposed by Dixon, Cinque (1994) advanced a fully syntactic account of the conventional-
ization of ordering preferences under which different classes of adjectives populate dedicated
syntactic categories which inhabit specialized projections in the syntactic tree. 例如,
color adjectives project a Color Phrase, shape adjectives project a Shape Phrase. The Shape
Phrase syntactically dominates the Color Phrase; with left-branching structure, hierarchical
dominance results in linear precedence. The ultimate source of this rigid structure was imma-
terial; at issue was a comprehensive and deterministic account of the facts (see Scott, 2002,
and Laenzlinger, 2005, for similar proposals).
Before the grammatical approaches, which map, 可以说, the terrain of adjective struc-
真实, psychological approaches advanced the idea that aspects of adjectives’ meaning explain
their relative order. The trouble lies in deciding precisely which aspects of meaning are rel-
埃万特. Kicking off the enterprise in 1898, Sweet proposed that adjectives which are more
closely connected with the noun in meaning occur closer to the noun, and that adjectives
with a more specialized meaning occur closer to the noun. 相似地, Whorf (1945) proposed
(1960)
that adjectives describing more “inherent” properties occur closer to the noun. Ziff
proposed that adjectives with less context-dependent meaning occur closer to the noun, 和
that adjectives that felicitously describe a narrower set of nouns occur closer to the noun.
Recent compositional approaches have argued that the fundamental factor in predicting ad-
jective ordering is whether or not an adjective forms a new concept with the noun it modifies
(McNally & Boleda, 2004; Svenonius, 2008): first you form the concepts (例如, “wild rice”
or “bad apple”), then you modify them (例如, “Minnesotan wild rice”). 相似地, Truswell
(2009) argues that the type of composition an adjective invokes (IE。, intersective vs. subsec-
主动的) determines its relative order (比照. the “absoluteness” proposal from Sproat & Shih, 1991).
These proposals and others like them circle around similar aspects of adjective meaning in
their account of ordering preferences; unfortunately, operationalizing metrics like meaning
distance, 特异性, inherence, and context-dependence is not a trivial task (but see the at-
tempt in Martin, 1969A, as well as our “Comparing subjectivity with alternative accounts
of adjective order” in our Supplemental Materials; Scontras, Degen, & 古德曼, 2017).
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
We revisit the idea that ordering preferences emerge from aspects of adjective meaning,
attempting to provide more thorough empirical grounding to these notions; from the gram-
matical approach we adopt the strategy of using semantic classes of adjectives to structure our
investigation and smooth our data. Distilling the psychological proposals that precede us into
a single feature, we advance the hypothesis that it is the subjectivity of the property named that
determines ordering preferences, such that less subjective adjectives occur linearly closer to
the nouns they modify (Hetzron, 1978; F. 爬坡道, 2012; Quirk, Greenbaum, 水蛭, & Svartvik,
1985). In “the big blue box,” judgments about bigness are likely less consistent than judgments
about blueness; “blue” is less subjective than “big,” and so, according to this theory, it occurs
closer to the noun “box.”
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
We believe that subjectivity synthesizes—rather than supplants—many of the previ-
ous psychological approaches, incorporating notions like “inherentness” and “context depen-
dence” into an intuitive psychological construct that readily operationalizes as a behavioral
措施. To test the hypothesis that adjective subjectivity predicts ordering preferences, 我们
created and validated empirical measures of the ordering preferences themselves and of an
adjective’s subjectivity. With reliable estimates of both, we then evaluated the predictive
power of subjectivity in adjective ordering preferences. To evaluate the relative success of
our subjectivity hypothesis, in “Comparing subjectivity with alternative accounts of adjective
order” in our Supplemental Materials (Scontras et al., 2017), we operationalized three of the
开放的心态: 认知科学的发现
54
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
previous accounts (inherentness, intersective vs. subsective modification, and complex con-
cept formation) and compared their predictions with those of subjectivity.
EXPERIMENT 1: ESTABLISHING THE MEASURES
Ordering preferences
We began by measuring preferences in adjective ordering. We selected a sample of 26 rela-
tively frequent, imageable adjectives from seven different semantic classes (年龄, 颜色, dimen-
锡安, 材料, physical, shape, 价值). We then elicited naturalness judgments on adjective-
adjective-noun object descriptions.
Participants We recruited 50 participants through Amazon.com’s Mechanical Turk crowd-
sourcing service. Participants were compensated for their participation.
Participants were asked to indicate which of two descriptions of an ob-
Design and methods
ject sounded more natural. Each description featured a noun modified by two adjectives,
for example “the red small chair” or “the small red chair.” Descriptions were random com-
binations of two adjectives and a noun from the list in Table 1, with the constraint that no
description contained adjectives from the same semantic class. Description pairs contained
the same words, with relative adjective order reversed. On each trial, participants indicated
their choice by adjusting a slider with endpoints labeled with the competing descriptions; 一个
example trial appears in Figure 1. Participants completed 26 试验. On each trial, we measured
the distance of the slider from each endpoint; values ranged between 0 和 1. Only native
speakers of English were included in the analyses; we analyzed data from 45 参与者.
对于每个形容词, we computed its mean naturalness score by averaging ratings of
结果
configurations in which it appeared in first position, farthest from the noun. 数字 2 (natural-
内斯) plots these mean naturalness scores by adjective class; greater values signal that a class’s
adjectives are preferred in first position, farther from the noun. This preferred distance measure
桌子 1. Adjectives, 名词, and their semantic classes.
Adjective Class
Adjective Class
Noun
班级
老的
新的
rotten
fresh
红色的
黄色的
绿色的
蓝色的
purple
棕色的
wooden
plastic
metal
年龄
年龄
年龄
年龄
颜色
颜色
颜色
颜色
颜色
颜色
材料
材料
材料
好的
坏的
round
square
大的
小的
huge
微小的
short
长的
smooth
难的
soft
价值
价值
shape
shape
dimension
dimension
dimension
dimension
dimension
dimension
physical
physical
physical
apple
banana
carrot
cheese
tomato
chair
couch
fan
TV
desk
食物
食物
食物
食物
食物
furniture
furniture
furniture
furniture
furniture
开放的心态: 认知科学的发现
55
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
数字 1. Example trial from Expt. 1: Ordering preferences. Participants indicated the more natural
of two adjective-adjective-noun descriptions on a sliding scale.
closely tracks class-level ordering hierarchies reported in the literature (狄克逊, 1982; Sproat
& Shih, 1991).
Corpus Validation
To validate our behavioral measure of ordering preferences, we conducted a corpus study on
相同 26 adjectives and measured their mean distance from the noun in phrases with two
形容词. We used TGrep2 (Rohde, 2005) and the TGrep2 Database Tools (Degen & Jaeger,
2011) to extract all “A A N” NPs that contained one of the 26 adjectives in Table 1 从
Penn Treebank subset of the Switchboard corpus of telephone dialogues (Godfrey, Holliman,
& McDaniel, 1992), as well as from the spoken and the written portions of the British National
语料库 (BNC, 参见http://www.natcorp.ox.ac.uk/). For these cases, we computed the distance of
each occurrence of our 26 target adjectives from the modified noun, yielding results for a total
的 38,418 adjective tokens. 对于每个形容词, mean distance from the noun was computed
(where the position directly preceding the noun was coded as 0, and the position preceding
that was coded as 1).
Mean distance from the noun for each adjective class is shown in Figure 2 (语料库). 这
corpus measure closely tracks the qualitative pattern we measured in our naturalness experi-
蒙特; quantitatively, the two measures are highly correlated (r2 = .83, 95% CI [.63, .90]), 在
spite of the fact that the corpus measure includes cases from a superset of the nouns tested
in our naturalness experiment. Our naturalness ratings thus operationalize both immediate
ordering preferences and speakers’ preferences in natural usage.
数字 2. Mean distance from noun inferred from naturalness ratings (preference), mean distance
from noun calculated from corpus counts (语料库), mean subjectivity ratings (主观性), 和
mean faultless disagreement ratings (faultless) for adjectives grouped by their semantic class. Error
bars represent bootstrapped 95% confidence intervals (DiCiccio & Efron, 1996).
开放的心态: 认知科学的发现
56
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
数字 3. Example trial from Expt.
形容词.
1: Subjectivity.
Participants rated the subjectivity of
Subjectivity
With clear estimates of ordering preferences, we then measured the subjectivity of the adjec-
tives that were tested in the ordering preferences experiment. We started with a direct measure
of “subjectivity.”
Participants We recruited 30 participants through Amazon.com’s Mechanical Turk crowd-
sourcing service. Participants were compensated for their participation.
Participants were shown a series of adjectives and asked to indicate how
Design and methods
“subjective” each one was on a sliding scale with endpoints labeled as “completely objective”
(coded as 0) and “completely subjective” (coded as 1; 数字 3). Participants completed a
total of 26 试验, one for each adjective in Table 1. The order was randomized for each par-
ticipant. Only native English speakers were included in the analyses; we analyzed data from
28 参与者.
Results We averaged the subjectivity scores for each adjective; greater values indicate greater
主观性. These averages were used in the analyses reported below. 数字 2 (主观性)
shows these scores by adjective class.
Faultless Disagreement Validation
Because subjectivity may be an ambiguous, or even subjective, 财产, we explored a sec-
ond measure that may have greater ecological validity. We operationalized subjectivity as the
potential for faultless disagreement between two speakers, which captures potential uncer-
tainty about assessment criteria and assessment outcomes (Barker, 2013; 肯尼迪, 2013;
Kölbel, 2004).1 We had participants (n = 40) evaluate whether two speakers could both be
right while the speakers produced conflicting object descriptions. 例如, an experimen-
tal trial would have Mary assert, “That apple is old,” then have Bob counter with “That apple is
not old”; participants rated whether both Mary and Bob could be right, or whether one of them
must be wrong. This measure, the faultless disagreement potential for the adjective at issue,
serves as an empirical estimate of adjective subjectivity. 数字 2 (faultless) plots these scores
1 See MacFarlane (2014) for more discussion of the many factors, both “semantic” and “pragmatic,” that
contribute to faultless disagreement effects. For a different approach, see F. 爬坡道 (2012), who builds on previous
corpus work (Wulff, 2003) to infer adjective subjectivity from surface features of strings.
开放的心态: 认知科学的发现
57
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
数字 4. Mean naturalness ratings plotted against mean subjectivity scores for each of the 26
adjectives tested in Expt. 1.
by adjective class, where a value of 1 signals that a class’s adjectives are always amenable
to faultless disagreement (IE。, maximally subjective). The results of this method were highly
correlated with our direct “subjectivity” scores (r2 = .91, 95% CI [.86, .94]), suggesting that
they measure a common underlying value: adjective subjectivity.
Predicting adjective order
To evaluate the power of subjectivity in predicting adjective ordering preferences, 数字 4 plots
mean naturalness ratings (Expt. 1: Ordering preferences) against mean adjective subjectivity
scores (Expt. 1: Subjectivity). Adjective subjectivity scores account for 85% of the variance
in the naturalness ratings (r2 = .85, 95% CI [.75, .90]). The faultless disagreement scores also
perform well, accounting for 88% of the variance (r2 = .88, 95% CI [.77, .95]). Using either
措施, more subjective adjectives are preferred farther from the noun; subjectivity indeed
predicts adjective ordering preferences.
One might worry that conducting our analysis at the level of individual adjectives ob-
scures information about the specific adjective-adjective configurations that participants rated
in our naturalness experiment. We therefore computed a subjectivity difference score for each
adjective class configuration (IE。, an ordered pairing of two adjective classes, CLASS1-CLASS2) 经过
subtracting the mean subjectivity score for CLASS2 from the mean subjectivity score for CLASS1.
Higher difference scores indicate that the adjective class closer to the noun is less subjective
than the class farther away. 数字 5 plots mean naturalness ratings for adjective class con-
figurations against these subjectivity difference scores; the two measures are highly correlated
(r2 = .80, 95% CI [.68, .88]). We also see that as the difference in subjectivity approaches zero,
the naturalness ratings approach 0.5 (IE。, 机会): ordering preferences weaken for adjectives
of similar subjectivity (例如, “yellow square” or “fresh soft”).
讨论
We found that adjective subjectivity scores account for almost all of the variance in naturalness
ratings, for several different analyses, strongly supporting our hypothesis that less subjective
开放的心态: 认知科学的发现
58
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
数字 5. Mean configuration naturalness ratings plotted against subjectivity difference scores for
each pair of adjective classes tested in Expt. 1.
adjectives occur closer to the noun. In “Comparing Subjectivity With Alternative Accounts of
Adjective Order” in our Supplemental Materials (Scontras et al., 2017), we compare these
results with the predictions made by other accounts. We found that subjectivity vastly out-
performs adjective inherentness (r2 = .00, 95% CI [.00, .02]) and concept-formability (r2 =
.00, 95% CI [.00, .00]) in accounting for ordering preferences. 的确, we failed to find any
evidence that ordering preferences depend on the modified noun. For subsective versus inter-
sective modification, we found that subjectivity explains independent variance in the observed
preferences within the different modification classes.
One might worry that the observed success of subjectivity in predicting ordering prefer-
ences is an artifact of the set of 26 adjectives we tested, and might not generalize to a broader
set of adjectives. 所以, we next consider a much larger set of adjectives.
EXPERIMENT 2: GENERALIZING OUR FINDINGS
To test the generalizability of the findings from Expt. 1, we aimed to construct a set of
adjectives that are attested in multi-adjective constructions and that span both semantic classes
and a broad spectrum of frequencies and lengths. The set of 78 adjectives we ultimately used
includes many adjectives that are traditionally overlooked in investigations of order-
ing preferences.
Ordering preferences
Participants We recruited 495 participants through Amazon.com’s Mechanical Turk. Partic-
ipants were compensated for their participation.
Starting with naturally-occurring examples of double adjective modification from
Materials
the Switchboard corpus, we chose 196 unique adjectives (从 13 different classes; 桌子 2)
和 166 unique nouns. Details of our selection process can be found in “Materials Selection
for Expt. 2” in our Supplemental Materials (Scontras et al., 2017).
开放的心态: 认知科学的发现
59
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
The design was similar to our previous naturalness rating experiments
Design and methods
(Expt. 1: Ordering preferences): participants indicated which of two object descriptions sounded
more natural, choosing between adjective-adjective-noun permutations that varied the relative
order of the adjectives. Adjectives were chosen at random from the set in Table 2, 与
constraint that adjectives from the same class were not paired together. Participants completed
30 试验. On each trial, participants indicated their choice by adjusting the slider between
endpoints labeled with the competing descriptions. 此外, participants were able to
indicate if a particular description did not make sense by checking a box labeled “Neither op-
tion makes sense.” Only native speakers of English were included in the analyses; we analyzed
数据来自 473 参与者.
对于每个形容词, we computed its mean naturalness score by averaging ratings
结果
of configurations in which it appeared in first position, farthest from the noun. 参加者
demonstrated little preference for adjective order when the descriptions were nonsense. 为了
this reason, we excluded responses to nonsensical descriptions from the analyses of subjectiv-
ity below; this exclusion process removed 2,295 observations (16% of the total 14,190).
Subjectivity
下一个, we evaluated the subjectivity of our new set of adjectives using the direct “subjectivity”
task from Expt. 1: Subjectivity.
Participants We recruited 198 participants from Amazon.com’s Mechanical Turk. Partici-
pants were compensated for their participation.
The design was identical to our previous direct “subjectivity” experi-
Design and methods
蒙特. Participants completed a total of 30 试验. On each trial an adjective was chosen at
random from the set of 78 表中 2. Only native speakers of English were included in the
分析; we analyzed data from 189 参与者.
Results We averaged the subjectivity scores for each adjective; greater values indicate
greater subjectivity. To evaluate the power of subjectivity in predicting adjective ordering
优先, we compared subjectivity scores with the naturalness ratings (数字 6). Ad-
jective subjectivity scores account for 51% of the variance in the naturalness ratings (r2 =
.51, 95% CI [.32, .66]). Four observations clearly stood out in Figure 6, corresponding to the
superlatives best, biggest, closest, and last.
的确, superlatives have been observed to
eschew adjective ordering preferences, occurring farthest from the modified noun regardless
of class or subjectivity (狄克逊, 1982); our naturalness ratings reflect this fact. Removing
superlatives, subjectivity scores perform markedly better, accounting for 61% of the vari-
安斯 (r2 = .61, 95% CI [.47, .71]). At the level of adjective class configurations, subjectiv-
ity difference scores account for 74% of the variance in the configuration ratings (r2 = .74,
95% CI [.66, .79]; 数字 7).2
2 This analysis and the plot in Figure 7 exclude superlatives. If we include superlatives in the class configuration
分析, subjectivity difference scores account for 69% of the variance in the naturalness ratings (r2 = .69, 95%
CI [.60, .76]).
开放的心态: 认知科学的发现
60
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, 古德曼
桌子 2. Adjectives used in Expt. 2.
Adjective
班级
Adjective
班级
Adjective
班级
junior
新的
老的
old-time
senior
young
黑色的
blonde
蓝色的
绿色的
purple
红色的
白色的
黄色的
biggest
大的
长的
mini
狭窄的
打开
thick
thin
文明的
creative
entrepreneurial
playful
年龄
年龄
年龄
年龄
年龄
年龄
颜色
颜色
颜色
颜色
颜色
颜色
颜色
颜色
dimension
dimension
dimension
dimension
dimension
dimension
dimension
dimension
人类
人类
人类
人类
专业的
sad
selfish
严格的
closest
内部的
overhead
corduroy
crocheted
gold
wooden
brazilian
english
european
hispanic
international
japanese
national
vietnamese
creamy
curly
frozen
lacy
smooth
solid
spicy
人类
人类
人类
人类
地点
地点
地点
材料
材料
材料
材料
nationality
nationality
nationality
nationality
nationality
nationality
nationality
nationality
physical
physical
physical
physical
physical
physical
physical
sweet
circular
square
快速地
slow
speedy
current
daily
everyday
historical
最好的
exciting
favorite
lavish
清楚的
pleasant
prestigious
strange
designated
不同的
个人
最后的
mixed
潜在的
代币
独特的
physical
shape
shape
速度
速度
速度
颞
颞
颞
颞
价值
价值
价值
价值
价值
价值
价值
价值
X
X
X
X
X
X
X
X
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
/
e
d
你
哦
p
米
我
/
我
A
r
t
我
C
e
–
p
d
F
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
哦
p
米
_
A
_
0
0
0
0
5
p
d
.
我
F
乙
y
G
你
e
s
t
t
哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3
A post-hoc look at our data revealed a small number of outlier adjectives (此外
the four superlatives). To systematically detect these outlier adjectives, we fit a linear regression
predicting naturalness ratings by subjectivity scores, then calculated the absolute difference
between the actual naturalness ratings and the model’s predicted values. Setting the cutoff
for this difference score at 3 × standard deviation, four adjectives stood apart as outliers:
entrepreneurial, solid, current, and daily (labeled in blue in Figure 6). Without the four outlier
形容词 (and the four superlatives), adjective subjectivity scores account for 70% 的
variance in the naturalness ratings (r2 = 0.70, 95% CI [0.58, 0.78]).
We also looked at the contribution of frequency and length in predicting ordering pref-
erences. Treating subjectivity, 频率, and length as predictors in a linear regression pre-
dicting naturalness ratings (excluding superlatives), the model accounts for 70% of the vari-
安斯 (r2 = .70). Nested model comparison reveals that the subjectivity predictor explains
significant variance in the extended model, F(1, 70) = 141.38, p < .001; the frequency and
length predictors also explain significant variance, frequency: F(1, 70) = 7.71, p < .01; length:
F(1, 70) = 9.73, p < .01. If we remove outlier adjectives that fall more than three standard
deviations away from the predicted value of the extended model (there were six: mini, frozen,
solid, current, daily, designated), the model performs better, accounting for 76% of the vari-
ance (r2 = .76).
OPEN MIND: Discoveries in Cognitive Science
61
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, Goodman
Figure 6. Mean naturalness ratings plotted against mean subjectivity scores for each of the 78
adjectives tested in Expt. 2. Superlatives are labeled in green; outlier adjectives are labeled in blue.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
o
p
m
i
/
l
a
r
t
i
c
e
-
p
d
f
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
o
p
m
_
a
_
0
0
0
0
5
p
d
.
i
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Figure 7. Mean configuration naturalness ratings plotted against subjectivity difference scores for
each pair of adjective classes tested in Expt. 2.
Discussion
The results of the current experiment demonstrate that subjectivity predicts ordering prefer-
ences in a much larger set of materials drawn from naturally occurring examples. At worst,
subjectivity accounts for more than half of the variance in the naturalness ratings for our set of
78 adjectives. Once we exclude superlatives, whose semantics likely dictates their position in
strings of nominal modifiers, as well as four outlier adjectives, subjectivity accounts for 70%
of the variance in this set of 70 adjectives. While adjective frequency and length contribute to
the observed preferences, we saw that subjectivity alone accounts for the vast majority of the
variance in our data.
OPEN MIND: Discoveries in Cognitive Science
62
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, Goodman
There remains the question of precisely why the four outlier adjectives—entrepreneurial,
solid, current, and daily—performed so poorly with respect to the predictions of subjectivity.
Perhaps the most notable feature of this set of adjectives is its heterogeneity: we fail to find
clear groupings by semantic class, relative frequency, or length. However, length likely does
factor into the observed behavior of entrepreneurial, the longest adjective tested, which was
the only outlier underpredicted by its subjectivity: participants preferred entrepreneurial closer
to the noun than its subjectivity alone would predict. Indeed, relative length has long been
known to affect the order of constituents, even in the domain of adjective ordering (Wulff,
2003): longer constituents appear later. Once we factor length into the equation predicting
ordering preferences, entrepreneurial no longer stands out.
GENERAL DISCUSSION
Adjective ordering preferences have received considerable attention throughout the history of
generative grammar and cognitive psychology, owing to their remarkable stability within and
across languages. Something so robust, the reasoning goes, must evidence a deep principle
of the cognitive architecture that shapes language. Yet while descriptions of the phenomenon
abound, an explanation has proven elusive. Grammatical theories that posit a rigid syntax of
adjective classes offer little more than a codification of the facts, and psychological approaches
stumble when it comes to operationalizing the specific aspects of adjective meaning at play.
In our investigation, we established two empirical constructs:
the preferences them-
selves, which we measured using naturalness ratings and validated with corpus statistics; and
adjective subjectivity, which we measured directly and corroborated with potential for fault-
less disagreement. An adjective’s semantics predicts its distance from the modified noun, such
that less subjective adjectives occur linearly closer to nouns they modify. In our Supplemental
Materials (Scontras et al., 2017), we investigated the predictions of three other hypotheses
from the literature: adjective inherentness (i.e., how essential an adjective’s meaning is to the
noun it modifies; Sweet, 1898; Whorf, 1945), intersective versus subsective modification (i.e.,
the mode by which an adjective composes semantically with the noun it modifies; Truswell,
2009), and concept formability (i.e., whether an adjective composes with a noun to form a
complex, idiomatic concept; Bouchard, 2005; McNally & Boleda, 2004; Svenonius, 2008).
In each case, we found that subjectivity has greater predictive power.
It bears noting that the preference to place less subjective adjectives closer to nouns is
not deterministic; nonpreferred orderings of adjectives can serve a communicative purpose,
for example to establish contrastiveness in discourse (A. A. Hill, 1958; Martin, 1969a, 1970;
Vendler, 1963). This constrastiveness follows straightforwardly from a manner implicature
(Levinson, 2000): marked forms (i.e., nonpreferred orderings of adjectives) yield marked in-
terpretations (i.e., atypical modification constituency). The work lies in determining the pre-
ferred orderings from which contrastive uses depart.
Indeed, many other situational factors
are likely to influence ordering (e.g., phonological shape, noun semantics, word and bigram
frequencies; cf. Wulff, 2003, and the results of Expt. 2); it is the more general tendencies we
are concerned with here.
Adjectives are just one of many elements that may occur in complex nominal construc-
tions. Other classes of elements include demonstratives (e.g., this and that) and numerals. In
his Universal 20, Greenberg observes that the relative order of these higher-order classes is
also stable cross-linguistically (Culbertson & Adger, 2014; Greenberg, 1963), suggesting that
subjectivity interacts with additional constraints from semantic composition in the determina-
tion of word order. Indeed, we saw hints of such interactions in Expt. 2, where superlatives
OPEN MIND: Discoveries in Cognitive Science
63
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
o
p
m
i
/
l
a
r
t
i
c
e
-
p
d
f
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
o
p
m
_
a
_
0
0
0
0
5
p
d
.
i
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, Goodman
stood apart from run-of-the-mill adjectives. Beyond nominals, adverbs (e.g., honestly, prob-
ably, carefully) are reported to exhibit regular orderings cross-linguistically (Cinque, 1999;
Ernst, 2002). Understanding these orderings would likely benefit from a systematic empirical
treatment similar to the one we have advanced here.
While subjectivity accounts for the regularities we observe in adjective ordering, the
deeper explanation for how subjectivity determines the relative order of adjectives remains
unsettled. Our results suggest that ordering preferences likely emerge, at least partially, from a
desire to place less subjective content closer to the substantive head of a nominal construction
(i.e., closer to the modified noun). For now we can only speculate about the ultimate source of
this desire. Subjective content allows for miscommunication to arise if speakers and listeners
arrive at different judgments about a property description. Hence, less subjective content is
more useful at communicating about the world. An explanation along these lines, based on
pressures to facilitate successful reference resolution, would have to depend on the hierarchi-
cal, not linear, ordering of adjectives: noun phrases are built semantically outward from the
noun, and more useful, less subjective content enters earlier in this process (cf. the mirroring
of preferences in pre- vs. postnominal languages). A full explanation must examine not only
why we observe the preferences that we do, but also how and to what extent these preferences
get conventionalized via the diachronic processes that shape language—a promising direction
for future research.
Whatever its source, the success of subjectivity in predicting adjective ordering prefer-
ences provides a compelling case where linguistic universals, the regularities we observe in
adjective ordering, emerge from cognitive universals, the subjectivity of the properties that the
adjectives name.
ACKNOWLEDGMENTS
This work was supported in part by a James S. McDonnell Foundation Scholar Award (to NDG),
Office of Naval Research Grant N000141310788 (to NDG), and a Swiss National Science
Foundation Early Postdoc Mobility fellowship (to JD).
AUTHOR CONTRIBUTIONS
GS, JD, and NDG designed research, GS and JD performed research and analyzed data, GS,
JD, and NDG wrote the article.
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
o
p
m
i
/
l
a
r
t
i
c
e
-
p
d
f
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
o
p
m
_
a
_
0
0
0
0
5
p
d
.
i
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
REFERENCES
Barker, C. (2013). Negotiating taste. Inquiry, 56 (2-3), 240–257. doi.
org/10.1080/0020174X.2013.784482
Bouchard, D. (2005). Sériation des adjectifs dans le SN et for-
mation de concepts [Serialization of adjectives in the NP and
formation of concepts]. Recherches linguistiques de Vicennes,
34, 125–142.
Cinque, G.
(1994). On the evidence for partial N-movement
in the romance DP.
J. Koster,
In R. S. Kayne, G. Cinque,
J.-Y. Pollock, L. Rizzi, & R. Zanuttini (Eds.), Paths towards univer-
sal grammar: Studies in honor of Richard S. Kayne (pp. 85–110).
Washington, DC: Georgetown University Press.
Culbertson, J., & Adger, D. (2014). Language learners privilege struc-
tured meaning over surface frequency. Proceedings of the Na-
tional Academy of Sciences of the United States of America,
111(16), 5842–5847. Retrieved from http://www.pnas.org/
content/111/16/5842
Degen,
J., & Jaeger, T. F.
from
Retrieved
publications/tdt_manual.pdf
(2011). The TGrep2 database tools.
https://sites.google.com/site/judithdegen/
DiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals.
Statistical Science, 11, 189–228.
Dixon, R. (1982). Where have all the adjectives gone? And other
Cinque, G. (1999). Adverbs and functional heads. Oxford, UK:
essays in semantics and syntax. Berlin, Germany: Mouton.
Oxford University Press.
OPEN MIND: Discoveries in Cognitive Science
64
Subjectivity Predicts Adjective Ordering Preferences
Scontras, Degen, Goodman
Ernst, T. (2002). The syntax of adjuncts. Cambridge, UK: Cambridge
Martin, J. E. (1970). Adjective order and juncture. Journal of Verbal
University Press.
Godfrey, J. J., Holliman, E. C., & McDaniel, J. (1992). SWITCH-
BOARD: A telephone speech corpus for research and develop-
ment. Proceedings of ICASSP-92, 1, 517–520.
Greenberg, J. H. (1963). Some universals of grammar with par-
ticular reference to the order of meaningful elements. In Uni-
versals of human language (pp. 73–113). Cambridge, MA:
MIT Press.
Hetzron, R.
In
H. Seller (Ed.), Language universals (pp. 165–184). Tübingen,
Germany: Narr.
(1978). On the relative order of adjectives.
Hill, A. A. (1958). Introduction to linguistic structures. New York,
NY: Harcourt, Brace.
Hill, F. (2012). Beauty before age? Applying subjectivity to auto-
matic English adjective ordering. In Proceedings of the NAACL
HLT 2012 Student Research Workshop (pp. 11–16). Stroudsburg,
PA: Association for Computational Linguistics.
Kennedy, C.
(2013). Two sources of subjectivity: Qualitative
Inquiry, 56(2–3),
assessment and dimensional uncertainty.
258–277. Retrieved from http://www.tandfonline.com/doi/abs/
10.1080/0020174X.2013.784483
Kölbel, M.
(2004). Faultless disagreement. Proceedings of
the
Aristotelian Society, 104, 53–73.
Laenzlinger, C. (2005). French adjective ordering: Perspectives on
DP-internal movement types. Lingua, 115, 645–689.
LaPolla, R.
J., & Huang, C.
In
R. M. W. Dixon & A. Y. Aikenvald (Eds.), Adjective classes:
A cross-linguistic typology (pp. 306–322). Oxford, UK: Oxford
University Press.
(2004). Adjectives in Qiang.
Learning and Verbal Behavior, 9, 379–383.
McNally, L., & Boleda, G. (2004). Relational adjectives as proper-
ties of kinds. Empirical Issues in Formal Syntax and Semantics,
5, 179–196. Retrieved from http://www.cssp.cnrs.fr/eiss5/index_
en.html
Quirk, R., Greenbaum, S., Leech, G., & Svartvik,
J. (1985). A
comprehensive grammar of the English language. London, UK:
Longmans.
Rohde, D. L. T. (2005). TGrep2 user manual, Version 1.15. Retrieved
from http://tedlab.mit.edu/dr/Tgrep2/tgrep2.pdf
Scontras, G., Degen, J., & Goodman, N. D. (2017). Comparing
subjectivity with alternative accounts of adjective order. Sup-
plementary material for “Subjectivity predicts adjective ordering
preferences.” Open Mind: Discoveries in Cognitive Science, 1(1).
doi.org/doi.org/10.1162/opmi_a_00005
Scott, G.-J. (2002). Stacked adjectival modification and the struc-
ture of nominal phrases. In G. Cinque (Ed.), The cartography of
syntactic structures, Volume 1: Functional structure in the DP
and IP (pp. 91–120). Oxford, UK: Oxford University Press.
Sproat, R., & Shih, C. (1991). The cross-linguistic distribution of
adjective ordering restrictions. In C. Georgopoulos & R. Ishihara
Essays in
(Eds.),
honor of S.-Y. Kuroda (pp. 565–593). Dordrecht, Netherlands:
Kluwer Academic.
Interdisciplinary approaches to language:
Svenonius, P. (2008). The position of adjectives and other phrasal
modifiers in the decomposition of DP.
In L. McNally &
C. Kennedy (Eds.), Adjectives and adverbs: Syntax, semantics,
and discourse (pp. 16–42). Oxford, UK: Oxford University Press.
Sweet, H. (1898). A new English grammar, logical and historical.
Levinson, S. C. (2000). Presumptive meanings. Cambridge, MA:
London, UK: Clarendon Press.
MIT Press.
Truswell, R. (2009). Attributive adjectives and nominal templates.
MacFarlane,
J.
(2014). Assessment
sensitivity. Oxford, UK:
Linguistic Inquiry, 40, 525–533.
Clarendon Press.
Martin, J. E. (1969a). Semantic determinants of preferred adjec-
Journal of Verbal Learning and Verbal Behavior,
tive order.
8, 697–704.
Martin, J. E. (1969b). Some competence-process relationships in
noun phrases with prenominal and postnominal adjectives.
Journal of Verbal Learning and Verbal Behavior, 8, 471–480.
Vendler, Z. (1963). The transformational grammar of English adjec-
tives. Philadelphia: University of Pennsylvania.
Whorf, B. L. (1945). Grammatical categories. Language, 21(1), 1–11.
(2003). A multifactorial corpus analysis of adjective
Wulff, S.
order in English. International Journal of Corpus Linguistics, 8(2),
245–282. doi.org/10.1075/ijcl.8.2.04wul
Ziff, P. (1960). Semantic analysis. Ithaca, NY: Cornell University
Press.
OPEN MIND: Discoveries in Cognitive Science
65
l
D
o
w
n
o
a
d
e
d
f
r
o
m
h
t
t
p
:
/
/
d
i
r
e
c
t
.
m
i
t
.
/
e
d
u
o
p
m
i
/
l
a
r
t
i
c
e
-
p
d
f
/
/
/
/
/
1
1
5
3
1
8
6
8
2
6
2
o
p
m
_
a
_
0
0
0
0
5
p
d
.
i
f
b
y
g
u
e
s
t
t
o
n
0
8
S
e
p
e
m
b
e
r
2
0
2
3
