Understanding Social Robots:
Attribution of Intentional Agency
to Artificial and Biological Bodies

Abstract Much research in robotic artificial intelligence (人工智能) 和
Artificial Life has focused on autonomous agents as an embodied and
situated approach to AI. Such systems are commonly viewed as
overcoming many of the philosophical problems associated with
traditional computationalist AI and cognitive science, 例如
grounding problem (Harnad) or the lack of intentionality (Searle),
because they have the physical and sensorimotor grounding that
traditional AI was argued to lack. Robot lawn mowers and
self-driving cars, 例如, more or less reliably avoid obstacles,
approach charging stations, and so on—and therefore might be
considered to have some form of artificial intentionality or
intentional directedness. It should be noted, 尽管, that the fact
that robots share physical environments with people does not
necessarily mean that they are situated in the same perceptual and
social world as humans. For people encountering socially interactive
系统, such as social robots or automated vehicles, this poses the
nontrivial challenge to interpret them as intentional agents to
understand and anticipate their behavior but also to keep in mind
that the intentionality of artificial bodies is fundamentally different
from their natural counterparts. 这需要, on one hand, A
“suspension of disbelief ” but, 另一方面, also a capacity for
the “suspension of belief.” This dual nature of (attributed) artificial
intentionality has been addressed only rather superficially in
embodied AI and social robotics research. It is therefore argued that
Bourgine and Varela’s notion of Artificial Life as the practice of
autonomous systems needs to be complemented with a practice of socially
interactive autonomous systems, guided by a better understanding of the
differences between artificial and biological bodies and their
implications in the context of social interactions between people and
技术.

Tom Ziemke
Linköping University
认识 & Interaction Lab
Human-Centered Systems Division
Department of Computer and

Information Science

tom.ziemke@liu.se

关键词
归因, embodiment, grounding,
human–robot interaction, intentionality,
observer’s grounding problem, 社会的
robots

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

1 介绍

Thirty years ago, Bourgine and Varela (1992), in their editorial introduction to the proceedings
of the first European Conference on Artificial Life, referred to Artificial Life research as “the practice
of autonomous systems” as well as “the most recent expression of a relatively long tradition of

© 2023 麻省理工学院.
根据知识共享署名发布
4.0 国际的 (抄送 4.0) 执照.

Artificial Life 29: 351–366 (2023) https://doi.org/10.1162/artl_a_00404

时间. Ziemke

Understanding Social Robots

thought which searches the core of basic cognitive and intelligent abilities in the very capacity for
being alive” (p. 希). Their characterization of autonomy is worth quoting at length:

The guiding intuition to both cybernetic forerunners and current proponents of artificial
life is quite similar: the need to understand the class of processes that endow living
creatures with their characteristic autonomy. . . . Autonomy in this context refers to their
basic and fundamental capacity to assert their existence and to bring forth a world that is
significant and pertinent without be[英] pre-digested in advance. Thus the autonomy of
the living is understood here in regards to its actions and to the way it shapes a world
into significance. This conceptual exploration goes hand in hand with the design and
construction of autonomous agents and suggests an enormous range of applications at all
scales, from cells to societies. (布尔吉讷 & 瓦雷拉, 1992, p. 希)

In the context of this article, the twofold use of the term autonomy in this quotation deserves
further attention. Bourgine and Varela’s reference to the design and construction of autonomous
agents implies a distinction between (A) biological autonomy as the phenomenon that we seek to under-
stand when using Artificial Life—or computational and robotic models in general—for the purpose
of scientific modeling and (乙) artificial autonomy as a capacity with which we seek to endow robotic
技术, based on our scientific understanding of its biological counterpart, or rough abstrac-
tions thereof.

Nowadays, 30 多年后, there are many examples of robotic real-world technologies with
some degree of autonomy in the artificial sense, ranging from robot vacuum cleaners to automated
vehicles. Robotic lawn mowers, 例如, can relatively reliably mow one’s lawn, avoid obstacles
while doing so, detect when they are about to run out of energy, navigate to their charging stations
for recharging, 等等. Although at some abstract level, these behavioral capacities could be
likened to those of living systems, the artificial autonomy with which they operate is, 当然,
fundamentally different from the biological autonomy of even the simplest living systems. 对于一个
事物, robotic lawn mowers, as engineered products with limited adaptivity, can only to a small
degree—if any—“bring forth a world that is significant and pertinent without be[英] pre-digested
in advance,” which Bourgine and Varela (1992, p. 希) considered characteristic for living systems
(比照. earlier discussion and Varela et al., 1991). Your robotic lawn mower might, 例如, 学习
to adapt to the specific layout of your garden and, in that sense, go beyond its preprogramming,
but clearly its overall functionality is largely determined by its designers and its owner. 毕竟, 它
is mowing your lawn, and you are the one who cares if the robot runs out of energy before reaching
its charging station. This seems to imply that interpretations of robot lawn mowers—or similar
artificial autonomous agents—as intentional agents might be a modern case of what Searle (1980)
called “extend[英] our own intentionality; our tools are extensions of our purposes, and so we find
it natural to make metaphorical attributions of intentionality to them” (p. 419). This would also seem
to be in line with Varela’s notion of intentionality as fundamentally rooted in biological autonomy
(例如, 瓦雷拉, 1992, 1997).

另一方面, much recent research in human–robot interaction (HRI) has shown that
people tend to interpret and predict the behavior of robots—or other autonomous artifacts,
such as automated vehicles—in folk psychological terms (例如, Perez-Osorio & Wykowska, 2020;
Schellen & Wykowska, 2019; Sciutti et al., 2013; Thellman et al., 2022; Thellman et al., 2017). 那
means that people tend to attribute intentional mental states to robotic systems, such as beliefs (例如,
in the lawn mower’s case, that the battery level is low), desires or goals (例如, to keep the battery level
above some threshold), 和意图 (例如, to approach the charging station to recharge). It should
be noted, 尽管, that this does not mean that people necessarily really believe that the robots in
question actually have such folk psychological, humanlike or animallike mental states (Thellman &
Ziemke, 2019). 实际上, in one of the earliest empirical studies of such attributions, Heider and
Simmel (1944) showed that people even attributed mental states to simple, moving geometric
形状, such as squares and triangles—even though presumably nobody thinks that triangles have

352

Artificial Life Volume 29, 数字 3

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

such states. Our recent systematic review of 155 empirical studies (Thellman et al., 2022) 有
shown that mental state attribution to robots is a common phenomenon and that the known con-
sequences include increased predictability, explainability, and trust. 换句话说, and somewhat
oversimplified, people seem to interpret robots as intentional agents because it makes them easier
to interact with.

Needless to say, there is an interesting tension here—which this article seeks to explore
further—between the view of intentionality as a biological phenomenon and the fact that people
frequently attribute intentionality to robotic technologies anyway. Hence the intended contribution
of this article to the theme of the special issue of which it is a part—“Biology in AI”—can be
described as follows: The article does not address the biological understanding or inspiration that in
many cases contributes to the development of artificial autonomous agents; 反而, it addresses the
fact that most of the human cognitive mechanisms used to interpret intentional agency in the world
around us of course stem from our interactions with a broad spectrum of biological autonomous
代理人, 那是, humans and other animals (比照. Urquiza-Haas & Kotrschal, 2015). We therefore tend
to interpret robots, virtual agents, and computer game or cartoon characters—but also, in many
案例, inanimate moving objects, such as Heider and Simmel’s (1944) geometric shapes—as if they
were living systems. 或者, as Black (2014) 把它, we “seem to have an innate propensity to see
bodies wherever we look” (p. 16), and therefore “consistently anthropomorphise machines, 我们的
attempts to conceptualise unfamiliar new artefacts falling back on the most fundamental and
sophisticated frameworks for understanding animation we have—those related to the human body”
(p. 38).

It should be noted, 尽管, that most of the time, people are of course aware of their attribu-
tions being attributions. For cartoon characters like Donald Duck, 例如, which are usually
not interactive and only appear on a screen or paper—and therefore are easy to consider as “not
real”—it might be relatively easy to shake off this intentional interpretation, 那是, to recognize
that we are attributing intentional agency where maybe there is none. For social robots, 在另一
手, which are interactive, physical, and therefore part of the “real world” in some intuitive sense,
things might be more complicated. H. Clark and Fischer (2022) have recently referred to this as the
social artifact puzzle, which they characterize as follows: “It seems self-contradictory, even irrational,
for people to hold these two attitudes simultaneously: (A) a willingness to interact with social robots
as real people or animals; 和 (乙) a recognition that they are mechanical artifacts” (部分 1, 为了 2).
This article therefore argues that today, when social robots, automated vehicles, and similar
interactive autonomous technologies are starting to become a real-world reality for many peo-
普莱, the practice of autonomous systems, as Bourgine and Varela (1992) envisioned, with a focus on
understanding and modeling biological autonomy, needs to be complemented with what might
be called a practice of socially interactive autonomous systems, with a focus on the human interpretation
of artificial autonomy. 重要的, this needs to be guided by a better understanding of the
differences between artificial and biological bodies and their implications in the context of
social—or quasi-social—interactions between people and autonomous technologies.

本文其余部分的结构如下. 部分 2 provides further background
by discussing intentionality and its relevance to the grounding problem and related criticisms of tradi-
tional artificial intelligence (人工智能) as well as embodied AI approaches to overcoming these problems.
部分 3 zooms in on current research on social robots and their interpretation in terms of inten-
tional agency, including the social artifact puzzle, discussions of deception in social robotics, and what I
refer to as the observer’s grounding problem. 部分 4, finally, concludes by getting back to the proposal
that we need to develop a practice of socially interactive autonomous systems, outlining what the
key elements and requirements of such a practice might be.

2 Background: Intentionality and Grounding

Before we move on to social robots in section 3, let me try to quickly recap the (象征) grounding
problem and related criticisms of traditional AI and cognitive science, 尤其, their view of

Artificial Life Volume 29, 数字 3

353

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

cognition as symbolic computation. There are older criticisms of AI (例如, Dreyfus, 1972/1979;
Taube, 1961), but I will here start with Searle’s much-discussed Chinese Room argument from 1980.
If you wonder why we need to recap this—after all, aren’t these discussions 30–50 years old, and isn’t
the symbol grounding problem solved anyway? (例如, Steels, 2008)—then the answer simply is that
some of the issues discussed in the context of the human interpretation of social robot behavior are
in fact closely related to those “old” arguments. 而且, Searle’s argument might be considered
particularly interesting because it addresses the role of intentionality in the human interpretation of
人工智能, which is still highly relevant to the human interpretation of social robot behavior. Anyway, I will
try to keep this section brief. Before we dive into the discussion of intentionality in AI in section 2.2,
尽管, I will try in section 2.1 to set the scene and structure the discussion by clarifying different
notions of intentionality.

2.1 Intentionality
The first thing to note about intentionality is that it is too complex and too controversial a phe-
nomenon to address in much detail here. 实际上, one might be tempted to avoid the concept
altogether, were it not for the fact that intentionality, independent of philosophical baggage and
scientific complexity, simply might be fundamental to the folk psychology of how people interpret the
world around them and, 尤其, how they interpret behavior in terms of intentional agency.1
As Malle et al. (2004) point out, the concept of intentionality

brings order to the perception of behavior in that it allows the perceiver to detect
structure—intentions and actions—in humans’ complex stream of movement . . . [和]
supports coordinated social interaction by helping people explain their own and others’
behavior in terms of its underlying mental causes. (p. 1)

更具体地说, we can roughly distinguish three overlapping aspects or notions of intention-
ality, corresponding to different ways in which agents can be directed at their environment. 第一的,
the aspect of intentionality discussed most in philosophy of mind is the aboutness of internal states (或者
“representations,” for lack of a better term). When your robot lawn mower, 例如, turns away
from objects it detects, or when it approaches its charging station, this could very well be ascribed to
the aboutness of the corresponding internal states. In more philosophical and human-centric terms,
Searle (1999) says the following:

The primary evolutionary role of the mind is to relate us in certain ways to the
环境, and especially to other people. My subjective states relate me to the rest of
世界, and the general name of that relationship is “intentionality.” These subjective
states include beliefs and desires, intentions and perceptions. . . . “Intentionality,” to
repeat, is the general term for all the various forms by which the mind can be directed at,
or be about, or of, objects and states of affairs in the world. (p. 85)

第二, central to discussion of intentionality in (社会的) psychology is the intentionality of behavior,
which can be illustrated by the fact that in the preceding example, you might say that your robot
lawn mower did not approach its charging station by chance but did so intentionally, 即, 为了
purpose of recharging. According to Malle and Knobe (1997), the folk concept of intentionality involves
several aspects: a desire for a particular outcome, beliefs about an action leading to that outcome,
an intention to perform the action, the skill to perform the action, and awareness of fulfilling the
intention while performing the action.

1 In Malle et al.’s (2004) 字, “if one took a Kantian approach to social cognition, searching for the fundamental concepts without
which such cognition is impossible, intentionality would be one of those concepts, on par with space, time and causality in the realm
of non-social cognition” (p. 1).

354

Artificial Life Volume 29, 数字 3

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

第三, there is what Varela (1992, 1997) referred to as the biological roots of intentionality, 即,
that organisms as a whole are part of and directed toward their environment in the sense that they need
to interact with it for the purpose of self-maintenance.

These distinctions also allow us to get a better grip on the nature of the “interesting ten-
sion” mentioned in the previous section. From the philosophical perspective of intentionality as
aboutness, it is easy to see why one might consider robot lawn mowers—which appear to sense
物体, avoid some of them, and approach others—as intentional agents. 从 (社会的) psy-
chology perspective, it might also seem natural to view behaviors like approaching a charging sta-
tion when energy levels are low as intentional—although there might be disagreement regarding
to what extent such systems really fulfill Malle and Knobe’s (1997) rather human-centric criteria
(例如, desire or awareness). That current robot lawn mowers lack what Varela (1992, 1997) referred
to as the biological roots of intentionality seems relatively clear, but it is less obvious if this does
or does not outweigh the other perspectives. 而且, as discussed in the beginning of the ar-
ticle, Varela seemed perfectly willing to use the term autonomy in both the biological sense and an
artificial sense, so it might not be far-fetched to think that similar arguments could be made in the
case of intentionality. Let us therefore move on to the next subsection, which discusses the role
of intentionality in discussions of AI.

2.2 Intentionality and AI
In his Chinese Room argument, originally formulated before robots became a focus of attention
in much AI research, Searle (1980) imagined himself sitting in a room with the only connections
to the outside world being strings of Chinese symbols that he receives as “input” and generates as
“output” (例如, as messages on pieces of paper passed under the door). Searle does not understand
any Chinese, so he does not know what the symbols refer to, but he is able to manipulate them
with the help of a set of rules (written in some language that Searle does understand), which help
him generate output based on the input he receives over time. Searle’s argument, intended as a
criticism of traditional views of cognition as symbolic computation, was that he, sitting in the room,
pretty much does exactly what a computer does when it transforms inputs into outputs according
to some program. His point, 当然, was that clearly he does not understand any Chinese, 在里面
sense that he does not know to what objects or events in the outside world any of the symbols refer.
尽管如此, people outside the room who speak Chinese might very well be able to attach meaning
to those inputs and outputs and therefore might get the impression that whoever or whatever is
in the room does understand Chinese—assuming that the program Searle follows is sufficiently
好的. Searle’s conclusion was that the computational view of human cognition is flawed, or must
at least be incomplete, because “the operation of such a machine is defined solely in terms of
computational processes over formally defined elements” (p. 422), and such “formal properties are
not by themselves constitutive of intentionality” (p. 422; in the philosophical sense of aboutness; 比照.
previous subsection). In Searle’s (1990) Scientific American article, this was illustrated with a cartoon
of a man and a computer facing a symbol on the wall, with the man imagining a horse, whereas the
computer only “sees” the symbol.

Harnad’s (1990) formulation of the symbol grounding problem was based on Searle’s arguments
but referred to the problem of intentionality as a lack of “intrinsic meaning” in purely computa-
tional systems. 这, 他争辩说, could be resolved by what he termed symbol grounding, 那是, 这
grounding of internal symbolic representations in sensorimotor interactions with the environment.
Others have pointed out that the problem applies not only to symbolic AI but also to sub-
symbolic neural-network approaches or any other computational formalism and have therefore
used terms like representation grounding (Chalmers, 1992), concept grounding (Dorffner & Prem, 1993),
and simply the grounding problem (例如, Ziemke, 1997, 1999). 布鲁克斯 (1993), one of the pioneers
of the embodied AI approach (例如, 布鲁克斯, 1991A, 1991乙), used the term physical grounding
and argued that in robots, unlike with the computer programs of traditional AI, “everything
is grounded in primitive sensor motor patterns of activation” (p. 154). Other roboticists, 更多的
interested in symbolic representation than Brooks, have also formulated the perceptual anchoring

Artificial Life Volume 29, 数字 3

355

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

问题 (Coradeschi & Saffiotti, 2003), which they describe as the intersection of symbol
grounding and pattern recognition, or “the problem of connecting, inside an artificial system,
symbols and sensor data that refer to the same physical objects in the external world” (p. 85).
The general approach of using simulated or robotic autonomous agents has also been referred to as
the “artificial life roots” of AI (Steels, 1993) or the “artificial life route” (Steels & 布鲁克斯, 1995) 到
人工智能, and this is of course what Bourgine and Varela (1992) referred to as the design and construc-
tion of autonomous agents in the quotation discussed at the beginning of this article.

因此, from the embodied AI perspective, you might think that by putting AI into robots, 这
grounding problem has been resolved. Proponents of this approach have argued that robots are
embodied and situated in roughly the same sense that humans and other animals are and therefore
should be able to overcome traditional AI’s problems with intentionality or intrinsic meaning. 这
problem of ungrounded representations, it might be argued, is solved through physical grounding
and either not having any representations at all (例如, 布鲁克斯, 1991乙; Varela et al., 1991) or acquiring
grounded internal representations, as Harnad (1990) proposed, in the course of sensorimotor inter-
action with the external world. Steels (2008), 例如, made the explicit claim that “the symbol
grounding has been solved,” based on his own large-scale Talking Heads experiments, 其中
populations of robotic agents, in interaction with people, evolved a shared symbolic vocabulary
through a large number of one-to-one—robot–robot as well as robot–human—language games in-
volving labeling objects in a shared physical environment. Steels also explicitly stated that Searle was
wrong to consider intentionality a biological property, arguing that the key to intentionality instead
lies in adaptation in the course of agent–environment and agent–agent interactions. In Harnad’s
(1989) 条款, this type of embodied AI has gone from a computational functionalism to a robotic function-
救世主. Zlatev (2001), 例如, formulated the latter position explicitly by stating that there is “no
good reason to assume that intentionality is an exclusively biological property (pace e.g., Searle)”
and that therefore “a robot with bodily structures, interaction patterns and development similar to
those of human beings . . . could possibly recapitulate ontogenesis, leading to the emergence of
intentionality, consciousness and meaning” (p. 155).

It should be noted, 尽管, that the robotic functionalism underlying much current research
in robotics and AI is of course far from uncontroversial
(例如, Froese & Ziemke, 2009;
Sharkey & Ziemke, 2001; Ziemke, 2022; Zlatev, 2002), and consensus is not likely to be achieved
any time soon. For the discussion in this article, 然而, it suffices to acknowledge that there are
different positions on this. One way to move the discussion forward—and toward the discussion
of human interpretation of social robots in the next section—is to think about this in terms of
situatedness. As discussed earlier, robots are commonly referred to as situated agents. The fact that
robots share physical environments with people, 然而, does not necessarily mean that they are
situated in the same perceptual and social world as humans (Ziemke, 2020). This is obvious to anyone
whose robot lawn mower has run over hedgehogs or small toys left on the lawn or who has had
their kids’ LEGO bricks sucked up by a robot vacuum cleaner. Although these robots are physically
situated in the human world, many of them are not equipped to detect hedgehogs, LEGO bricks,
and so on—or do not attach the same meaning to them as people do. This brings us to the discus-
sion of socially interactive robots, for which a certain degree of social situatedness and shared meaning
is crucial.

3 讨论: Social Robots as a Challenge

可以说, a crucial prerequisite for individual and social trust in socially interactive robots—once
they have entered the real world, where they must be able to interact with people of different ages
as well as educational and cultural backgrounds—is that people can interpret and anticipate the
behavior of such systems sufficiently reliably to safely interact with them. Hence the design of in-
teractive autonomous systems needs to be informed by a thorough understanding of the mecha-
nisms underlying human social interaction with such systems (Ziemke, 2020). This section argues

356

Artificial Life Volume 29, 数字 3

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

数字 1. Social robots as intentional agents. (左边) An experiment on human mental state attribution to robots
(Thellman & Ziemke, 2020), 在哪里 (1) a humanoid robot observes somebody putting a ball into one of two bas-
kets and (2) a human observer is asked to predict the robot’s response to the question of where the ball is. (正确的) A
pedestrian encounters a self-driving car at a crosswalk and wonders if it will let him cross the road (Ziemke, 2020).
Reprinted with permission from Ziemke, Science Robotics, 5, eabe2987 (2020). Credit: A. Kitterman/Science Robotics.

that this poses significant challenges for the research communities involved as well as for HRI
practitioners trying to design social robots and human–robot interactions. Some have even argued
that what is needed is a “new ontological category” for “artifacts that appear to think and feel, 能
be friends, and at least potentially lay some moral claims for kind, fair and just treatment” (Melson
等人。, 2006, p. 4; 比照. H. 克拉克 & Fischer, 2022).

Before we dive deeper into discussions of how people interpret social robots, 尽管,
let me try to clarify what exactly I mean by the term social robot here. In Thellman and Ziemke (2021),
we argued in more detail for the position that what makes a robot or any other interactive
autonomous system “social” is that its behavior is most appropriately interpreted in terms of
intentional states. This is based on the notion that human social interaction is fundamentally
connected to people’s understanding of others as intentional agents, as pointed out earlier with
reference to Malle et al.’s (2004) view of intentionality as a central component of human social
认识. The term robot here is used in a broad sense to refer to any physical artifact that inter-
acts with its environment by means of sensors and motors. 数字 1 provides two examples of
what is meant by a social robot. The humanoid robot to the left might be an example of what
most readers consider a “social robot,” whereas some would probably not necessarily apply that
term to the self-driving car on the right. The important point here, 尽管, is that in interac-
tions with people, they are both usually easily—and maybe most appropriately—interpreted in folk
psychological terms as intentional agents with beliefs, 目标, 和意图. The humanoid example
illustrates a robotic version of a classical false-belief task from developmental child psychology
(比照. Thellman & Ziemke, 2020), where the human observer’s task is to predict what beliefs some
other agent (在这种情况下, the robot) holds regarding the position of an object. The second example
is a scenario in which a pedestrian encounters a self-driving car at a crosswalk and needs to assess
what the vehicle’s perception of the situation might be and how it might act. The latter example is
also particularly interesting because it illustrates a real-world scenario in which (A) there is something
more at stake than a mere misinterpretation of observed behavior and (乙) the pedestrian is under a
certain time pressure.

Given what I have said about people’s attribution of intentional agency to, 例如, Heider
and Simmel’s (1944) moving geometric shapes or cartoon characters like Donald Duck, it prob-
ably does not come as a surprise to anyone that social robots are of course commonly inter-
preted as intentional agents as well. 有, 然而, so far relatively little agreement about what
kinds of mental states people ascribe to robots (比照. Thellman & Ziemke, 2019): A classical study by
Gray et al. (2007), 例如, found that people tend to attribute agency to robots (例如, memory or

Artificial Life Volume 29, 数字 3

357

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

p

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
A
r
t
我
/

/

我

A
r
t
我
C
e
–
p
d

F
/

/

/

/

2
9
3
3
5
1
2
1
5
4
2
5
2
A
r
t
我

/

_
A
_
0
0
4
0
4
p
d

.

F

乙
y
G
你
e
s
t

t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

时间. Ziemke

Understanding Social Robots

规划), but not experience (例如, fear or pleasure). Sytsma and Machery (2010) also found that
people refrain from attributing subjective states that have hedonic value for the subject, 那是,
valenced states (例如, feeling pain), as opposed to unvalenced states (例如, seeing red). Buckwalter and
Phelan (2013) further showed that people’s tendency to attribute experiential or valenced states de-
pends on the described function of the robot. Fiala et al. (2014) found that, when allowed to choose
between different ways of describing the capabilities of a robot (例如, the robot “identified the lo-
cation of the box” vs. “knew the location of the box”), people preferred not to attribute mental
states at all. The authors also noted that responses to questions about the mental states of robots
are influenced by a wide variety of factors, including the apparent function of the robot, 道路
in which the question is asked, and cultural platitudes about robots. 总共, it is difficult to iden-
tify what kinds of mental states people attribute to robots by asking them directly. This is at least
partly because such questions are open to interpretation regarding the reality of the mental states
of robots (比照. Thellman & Ziemke, 2019). Fussell et al. (2008), 例如, found that people
might deny that a robot has a mind, despite having previously attributed mind (mental states) 什么时候
describing its behavior.

Owing to the growing research activity in social robotics in recent years, there also has been a
growing interest in both conceptual discussions (例如, Papagni & Koeszegi, 2021; Perez-Osorio &
Wykowska, 2020; Schellen & Wykowska, 2019; Thellman et al., 2022; Thellman & Ziemke, 2019)
and empirical investigations of why, 什么时候, and how people attribute intentional agency and men-
tal states to robots. Most of the theoretical discourse indicates that people commonly interpret
the behavior of robots following similar strategies as when interpreting human or animal behav-
ior. Dennett (1971, 1989) referred to this as the intentional stance, which he contrasted with design
stance and physical stance as alternative approaches to interpreting observed behavior. 然而,
according to the cognitive default hypothesis in studies of human interpretation of animal behavior
(Caporael & Heyes, 1997; 格思里, 1997; Urquiza-Haas & Kotrschal, 2015), anthropomorphism
in fact emerges as an automatic response to any sufficiently humanlike behavior or feature, 英语-
pecially when a swift response is required and/or other modes of interpretation are not readily
可用的. Urquiza-Haas and Kotrschal have developed a (神经的)psychological model of how
automatic/bottom-up and reflective/top-down mechanisms of physical/embodied and social cog-
nition interact in the human anthropomorphic interpretation of animal behavior. Their hypothesis
is that the balance of automatic and reflective cognitive processes depends on phylogenetic distance
and on shared morphological and behavioral features.

Regarding empirical investigations of mental state attribution to robots, my coauthors and I
recently published a systematic review of 155 学习 (Thellman et al., 2022). Most empirical re-
search so far has been concerned with determinants and consequences, 那是, the questions of when
people attribute mental states to robots, and why. Determinants include human factors, such as age
and motivation, as well as robot factors, such as appearance and behavior. Consequences include in-
creased predictability, explainability, and trust but also increases in cognitive drain and moral con-
欧洲核子研究组织. Relatively little, 然而, is known about the how, 那是, the mechanisms underlying such
mental state attributions. A multidimensional account similar to Urquiza-Haas and Kotrschal’s
(2015) model might also be a promising starting point for investigations of the neural and
psychological mechanisms involved in social human–robot interactions, especially given that one
of the findings of our review (Thellman et al., 2022) is that there is a “computer < robot < human” pattern in the tendency to attribute mental states2 that appears to be moderated by the presence of socially interactive behavior. However, an account of the human interpretation of autonomous technologies is likely to require additional dimensions and complexity, given that (a) autonomous technologies are a “moving target” (i.e., unlike cats or cows, robots today are not what they were 10 years ago) and (b) most people presumably do not yet have firmly established categories. 2 That means that people are more likely to attribute mental states to humans than to robots and more likely to attribute them to robots than to computers, which is in line with Urquiza-Haas and Kotrschal’s (2015) arguments regarding the role of shared morphological and behavioral features. 358 Artificial Life Volume 29, Number 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots As briefly mentioned earlier, H. Clark and Fischer (2022) have recently characterized the state of the art regarding human interpretation of social robots as puzzling: Social robots are a puzzle. On the one hand, people interact with them as if they were humans or pets. They talk with them, show them things, and engage with them in joint activities. At the same time, people know that social robots are mechanical artifacts. They have metal and plastic parts, sensors for vision and hearing, and speech that sounds artificial. It seems self-contradictory, even irrational, for people to hold these two attitudes simultaneously: (a) a willingness to interact with social robots as real people or animals; and (b) a recognition that they are mechanical artifacts. The puzzle is not only theoretical but practical. When a robot stops moving, people must decide “Did the social agent fall asleep, or did the artifact’s battery die?” And when its finger breaks off, “Am I sad because the social agent is in pain, or because the artifact needs repairing?” Call this the social artifact puzzle. (section 1, para 2, emphasis original) As has been argued in more detail elsewhere (Ziemke & Thellman, 2023), the claim that there is something “self-contradictory, even irrational” about how people interpret social robots might be overstated. Let us take the crosswalk scenario illustrated in Figure 1 as an example. Taking the intentional stance, the pedestrian might wonder, “Has that car seen me?,” “Does it understand that I want to cross the road?,” or “Is it planning to stop for me?” (cf. Ziemke, 2020). Alternatively, the pedestrian could of course take what Dennett refers to as the design stance and try to predict the car’s behavior based on the general assumption that such vehicles are designed to detect people and not harm them. That might seem more appropriate to some readers—and safer to pedestrians—but this would still require the pedestrian to make additional, more situation-specific assumptions about whether the car has actually detected him in that particular situation (Thellman & Ziemke, 2021; Ziemke, 2020). This brings us back to what I said earlier about review findings regarding the conse- quences of mental state attribution to robots (Thellman et al., 2022): In a nutshell, such attributions seem to lead to increased predictability, explainability, and trust, which means that treating such artifacts as intentional, social agents might simply make them easier to interact with. In that sense, H. Clark and Fischer’s (2022) “social artifact puzzle” is less puzzling than it might seem at first (Ziemke & Thellman, 2023). Moreover, we could speculate that in safety-critical situations like the crosswalk scenario, it might in fact be more rational to view the driverless car as both an intentional agent and a mechanical artifact, instead of picking only one of them.3 Related to the question of what is rational or irrational, in discussions of human interpretations of robot behavior, there is sometimes a tendency to view folk psychological interpretations of robots as intentional agents as problematic (e.g., Fuchs, 2022; Sharkey & Sharkey, 2021). This is closely related to similar discussions of anthropomorphism, which generally can be characterized as “the human tendency to attribute human traits to non-human entities” (Damiano & Dumouchel, 2018, p. 2), including both animals and artifacts (cf. earlier discussion). As Damiano and Dumouchel point out, anthropomorphism traditionally “has been viewed as a bias, a category mistake, an obstacle to the advancement of knowledge, and as a psychological disposition typical of those who are immature and unenlightened, i.e., young children and ‘primitive people’ ” (p. 2; see, e.g., Caporael, 1986; Fisher, 1996; S. D. Mitchell, 2005). Related negative views can be found in discussions of “deception” in social robotics. Sharkey and Sharkey, for example, have argued that “efforts to develop features that promote the illusion of mental life in robots could be viewed as forms of deception” (Sharkey & Sharkey, 2011, p. 34) because “current robots have neither minds nor experiences” (Sharkey & Sharkey, 2021, p. 309). Their argument that the “appearance and behaviour of a robot can lead to an overestimation of its functionality or to an illusion of sentience or cognition that can promote misplaced trust and 3 This might, admittedly, not be in line with the “swift response” aspect discussed earlier. Artificial Life Volume 29, Number 3 359 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots inappropriate uses such as care and companionship of the vulnerable” (Sharkey & Sharkey, 2021, p. 309) is in fact much in line with my own suggestion that social robot design needs to “better guide users of interactive autonomous systems by encouraging appropriate attributions of intentionality and mental states and discouraging inappropriate ones, thereby reducing unrealistic expectations on such systems” (Ziemke, 2020, p. 2). Following Coeckelbergh’s (2018) discussion of deception by information technologies in terms of magic and performance, Sharkey and Sharkey (2021) elaborate their position on deception in social robotics as follows: As in the case of a magic show, the users/audience may not have been fooled into thinking that a robot is sentient, or has emotions, and may know that it is a trick. This is likely to be the case when a social robot is displayed to an audience with a reasonable knowledge of robotics. The audience members could enjoy the performance, at the same time as looking for clues or asking questions about how the performance was accomplished. This is less likely to be the case with naïve audiences, or vulnerable groups of people such as the very young, or older people with cognitive limitations. (p. 311) The concern about naive audiences and vulnerable user groups is of course clearly warranted, especially where people are involuntarily exposed to robots they might not be prepared to deal with. The crosswalk scenario illustrated in Figure 1, for example, would be rather different if we were to consider very young or very old pedestrians and the potential cognitive limitations that are charac- teristic for different age groups. The notion of “deception” in social robotics is not unproblematic, though, owing to the difficulties discussed earlier in assessing what mental states people attribute to robots for the purpose of making them interpretable and interactable versus what states they think those robots really have (cf. Thellman & Ziemke, 2019). In a similar vein, Damiano and Dumouchel (2018) point out that anthropomorphic projections do not [necessarily] rest on the prior belief that an object or animal has human like mental states. It rests on the recognition that one is dealing with an entity that acts . . . and that the relation has changed, from, say, a relation of use to a form of interaction. That is: to a relation that requires the coordination of the actions of two “agents” for any one of them to be able to achieve his, her or its goal. (p. 6) This leads us to what might be called the observer’s grounding problem. In the original symbol grounding problem (Harnad, 1990), as illustrated in the discussion of Searle’s (1980, 1990) Chinese Room argument, the problem was that the AI system/the computer/Searle-inside-the-room lacked original intentionality or intrinsic meaning because of an inability to make the connection be- tween the symbols manipulated in the room and the objects or events in the real world (outside the room) that observers outside the room might consider the symbols to refer to. In robotic autonomous agents that are physically grounded and situated in the real world (i.e., in the same envi- ronment/room as the human observer), the problem is “solved” (cf. Steels, 2008) in the sense that the robot’s internal mechanisms are grounded in sensorimotor interaction with the environment. Whatever internal mechanisms, representations, and so on your robot lawn mower may use, for example, to avoid obstacles, keep track of its battery level, navigate to the charging station, and so on, it clearly works, at least most of the time. As I have pointed out, though, the fact that robots share physical environments with people does not necessarily mean that they are situated in the same perceptual and social world as humans. This was earlier exemplified with hedgehogs and small toys left on the lawn, which robot lawn mowers simply might not be equipped to perceive and/or to which they might not attach the same meaning as you or your kids do. This is relatively obvious but becomes both more problematic and more significant in cases when one actually needs to interact with a robot, instead of just observing 360 Artificial Life Volume 29, Number 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots it. If we, for example, again consider the crosswalk scenario illustrated in Figure 1, the cognitive burden on the pedestrian as the human observer interpreting and anticipating the car’s behavior is rather high. He needs to improvise—in “real time,” and possibly without having encountered that particular car before—an appropriate understanding of the situation that allows, in Damiano and Dumouchel’s (2018) terms, both agents to achieve their goals. What I call the observer’s grounding problem, then, is the fact that human observers, for the reasons discussed, are of course likely to interpret the situation through the lens of their own folk psycho- logical interpretations and anthropomorphic projections; the observers use their own grounding, based on their own biological autonomy and intentional agency, plus their experiences of interac- tions with other living systems, to interpret the behavior of a robotic system, whose physical and sensorimotor grounding is likely to be very different. Moreover, although the robot and its human observers are now, in terms of the Chinese Room metaphor, “in the same room,” observers are still in an important sense in the same situation as the observers outside the room in Searle’s original argument—the human observers have no direct access to what goes inside the robot’s “head” (the software controlling the car, in the crosswalk scenario) but have to use their own groundings (perception, experience, etc.) to interpret how the robot might perceive the situation.4 For people encountering socially interactive systems, this poses the nontrivial challenge to inter- pret them as intentional agents to understand and predict their behavior, but also keep in mind that the intentionality of artificial, robotic bodies is fundamentally different from that of their natural counterparts. This requires, on one hand, a “suspension of disbelief” (cf. Duffy & Zawieska, 2012) but, on the other hand, also a capacity for the “suspension of belief.” The “suspension of dis- belief ” occurs when, in our crosswalk example, the pedestrian adopts some folk psychological/ anthropomorphic interpretation like “the car has seen me, understands that I want to cross the road, and intends to slow down and let me do that”—as if the car had a human driver. The “sus- pension of belief ” occurs, or might occur, when the observer reminds himself that his interpretation of the robot’s (in this case, the car’s) interpretation of the situation is not necessarily accurate—or when it turns out that the interaction does not actually unfold in the way the observer had expected. This dual nature of (attributed) artificial intentionality has been addressed only rather superfi- cially in embodied AI and social robotics research (cf. Froese & Ziemke, 2009). As discussed in more detail elsewhere (Thellman & Ziemke, 2021; Ziemke & Sharkey, 2001), research in embod- ied AI and cognitive science (e.g., Brooks, 1991b; A. Clark, 1997; Suchman, 1987; Ziemke, 1997; Ziemke & Sharkey, 2001) acknowledged early on that any robot necessarily has a different perceptual world—or Umwelt (von Uexküll, 1982)—than humans, but the implications for HRI have received relatively little attention so far. Thellman and Ziemke (2021) have referred to this as the perceptual belief attribution problem in human–robot interaction: How can people understand what robots know (and do not know) about the shared physical environment? The observer’s grounding problem, as for- mulated in this article, then, is the other side of the same coin, you might say, because it addresses the fact that people (a) tend to interpret interactive situations in terms of their own folk psycho- logical/anthropomorphic perception and understanding and (b) might not have any other choice anyway, unless they happen to have the technical expertise or interaction experience to have a suf- ficiently in-depth understanding of how a particular robot really sees the world. 4 Conclusion: Toward a Practice of Socially Interactive Autonomous Systems Let me summarize the arguments in this article. We started off by discussing the distinction between biological autonomy and artificial autonomy implied in Bourgine and Varela’s (1992) notion of Artificial 4 It might be worth noting that the term observer’s grounding problem is of course ambiguous—and intentionally so; no pun intended—because it can be parsed in two ways and thus refers to both (a) the observer having a grounding problem (it’s for him that things are at stake, e.g., the crosswalk scenario in Figure 1) and (2) the observer’s grounding being the problem (rather than the AI system’s grounding, as in the original symbol grounding problem). Artificial Life Volume 29, Number 3 361 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots Life research as constituting or moving toward a practice of autonomous systems. We also addressed Varela’s (1992, 1997) notion of intentionality—as rooted in biological autonomy—which, at least at that time, traditional AI systems were considered to lack. We then moved on to address the ground- ing problem and related criticisms of traditional AI, as well as embodied AI approaches to overcom- ing these problems. Today’s embodied AI and robots might be said to have some form of artificial intentionality, because clearly they are capable of successfully interacting with objects in the real world, most of the time. The philosophical question as to what degree such artificial intentionality consti- tutes original intentionality—or just another case of extended human intentionality—will probably have to be left to a few more decades of discussion of the Chinese Room argument. We then zoomed in on current research on social robots and people’s folk psychological interpretations in terms of intentional agency, addressing the social artifact puzzle, discussions of deception in social robotics, and what I referred to as the observer’s grounding problem. We found that the distinction—or, should we say, the relation—between biological and artificial intentional agency plays a central role in many of these discussions. On one hand, some would argue that, as a matter of fact, “current robots have neither minds nor experiences” (Sharkey & Sharkey, 2021, p. 309) and therefore also should not be attributed mental or emotional states, at least not where naive audiences and vulnerable user groups are involved. On the other hand, em- pirical studies indicate (Thellman et al., 2022) that people in many cases—but notably not in all cases!—choose to make folk psychological attributions of intentional agency and that such attribu- tions seem to increase predictability, explainability, and trust but also seem to increase cognitive drain and moral concern. Damiano and Dumouchel (2018) have also argued that instead of taking a dichotomous approach to the ethics of social robotics that considers them “a ‘cheating’ technology, as they generate in users the illusion of reciprocal social and affective relations” (p. 1), we might want to simply acknowledge anthropomorphism, folk psychology, and so on as distinctly human fundamental mechanisms of social interaction (cf. Duffy, 2003; Papagni & Koeszegi, 2021)—which would also allow us to develop what they call “a critical experimentally based ethical approach to social robotics” (Damiano & Dumouchel, 2018, p. 1). These discussions illustrate that today, when social robots are starting to become a part of peo- ple’s everyday lives, we need to complement Bourgine and Varela’s (1992) notion of a practice of autonomous systems by also developing a practice of socially interactive autonomous systems. One of the ele- ments of such a practice will need to be a concern for what Sharkey and Sharkey (2021) refer to as naive audiences and vulnerable users, that is, people who might have difficulties dealing with the cognitive complexities involved in social human–robot interactions. One way of addressing this is what Sharkey and Sharkey (2021) refer to as “allocation of responsibility for harmful deception” (p. 309), a way of holding different stakeholders accountable for people’s interpretation of what robots can and cannot do. But it should also be noted that much more can be done in terms of fundamental AI literacy (e.g., Ng et al., 2021) to make “naive audiences” less naive. The cognitive science and AI communities, large parts of which still hold on to traditional views of cognition as computation (cf. Ziemke, 2022) and “mind as machine” (e.g., Boden, 2006), certainly could and should contribute more to clarifying the differences between people and existing technologies. This would need to include a clearer acknowledgment that, at least at this point, (human) cognition and (machine) computation are still very different things. Widespread misconceptions of AI and robotic technologies are apparent in, for example, re- cent discussions of self-driving cars, which have been characterized by overly optimistic predictions (cf. M. Mitchell, 2021). Part of the problem is that such systems lack the embodied experience that allows human drivers to understand the perspectives of other drivers, cyclists, and pedestrians. Despite decades of discussion of embodied cognition and social interaction in the cognitive sciences (e.g., Varela et al., 1991; Ziemke, 2022), many people still fail to appreciate that such embodied understanding would be difficult to acquire or replicate in self-driving cars—which, after all, differ radically from people in their “embodiment.” This is illustrated by recent experiments of ours in which people tended to assume that self-driving cars (a) have humanlike percep- tual capabilities (Thellman, Pettersson, et al., 2023) and (b) have the cognitive capacity of object 362 Artificial Life Volume 29, Number 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots permanence (which usually takes children four to eight months of sensorimotor interaction to develop) (Thellman, Holmgren, et al., 2023). For similar reasons, after accidents involving auto- mated vehicles, people often wonder why such systems lack the common sense required to understand human behavior (e.g., jaywalking; Marshall & Davies, 2019). These are examples of expectations of AI and robotic technologies that are overly human centered or simply too high (cf. Ziemke, 2020). Common sense, for example, has been discussed as a problem in AI research for decades (e.g., Dreyfus, 1972/1979; Taube, 1961) and is not likely to be resolved any time soon. Many researchers in AI and cognitive science are, of course, well aware of this but have not man- aged to communicate these fundamental limitations to the general public sufficiently well. More specifically, overly high expectations would be easier to avoid if researchers in fields like cogni- tive science, AI, and social robotics could manage to develop and communicate a clearer under- standing of what humans, as evolved living systems, share with other species, in particular, the domestic animals with which we have coevolved over extended periods of biological and cultural evolution (cf. Urquiza-Haas & Kotrschal, 2015)—and what we consequently do not share with computers, robots, self-driving cars, and so on, whose cognitive and behavioral capacities lack those shared biological roots. It should be noted, though, that making a clear distinction between biological and artificial autonomy, agency, and intentionality as they currently are does not necessarily imply a dichoto- mous approach that insists on fundamental differences between biological and artificial autonomous systems that cannot possibly be overcome. Humans and social robots can, of course, also coadapt, codevelop, and coevolve (cf. Damiano & Dumouchel, 2018), in other, nonbiological ways. How this will affect the future of human–robot social interactions remains to be seen—humans are, after all, a biological, cultural, and technological species. The practice of socially interactive autonomous systems, therefore, needs to be informed and guided by a thorough understanding of the differences between artificial and biological bodies and their implications in the context of social interactions between people and autonomous technologies. Acknowledgments This work was supported by ELLIIT, the Excellence Center at Linköping-Lund in Information Technology (https://elliit.se/), and a Swedish Research Council (VR) grant (2022-04602) on “Social Cognition in Human–Robot Interaction.” The author thanks Sam Thellman, Stevan Harnad, Luisa Damiano, Pasquale Stano, and three anonymous reviewers for useful feedback on earlier versions of this article. References Black, D. (2014). Embodiment and mechanisation: Reciprocal understanding of body and machine from the Renaissance to the present. Ashgate. Boden, M. (2006). Mind as machine: A history of cognitive science (2 vols.). Oxford University Press. Bourgine, P., & Varela, F. J. (1992). Towards a practice of autonomous systems. In F. J. Varela & P. Bourgine (Eds.), Toward a practice of autonomous systems (pp. xi–xvii). MIT Press. Brooks, R. A. (1991a). Intelligence without reason. In Proceedings of the twelfth International Joint Conference on Artificial Intelligence (IJCAI-91) (pp. 569–595). Morgan Kauffmann. Brooks, R. A. (1991b). Intelligence without representation. Artificial Intelligence, 47(1–3), 139–159. https:// doi.org/10.1016/0004-3702(91)90053-M Brooks, R. A. (1993). The engineering of physical grounding. In Proceedings of the fifteenth annual conference of the Cognitive Science Society (pp. 153–154). Erlbaum. Buckwalter, W., & Phelan, M. (2013). Function and feeling machines: A defense of the philosophical conception of subjective experience. Philosophical Studies, 166(2), 349–361. https://doi.org/10.1007/s11098 -012-0039-9 Caporael, L. R. (1986). Anthropomorphism and mechanomorphism: Two faces of the human machine. Computers in Human Behavior, 2(3), 215–234. https://doi.org/10.1016/0747-5632(86)90004-X Artificial Life Volume 29, Number 3 363 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots Caporael, L. R., & Heyes, C. M. (1997). Why anthropomorphize? Folk psychology and other stories. In R. W. Mitchell, N. S. Thompson, & H. L. Miles (Eds.), Anthropomorphism, anecdotes, and animals (pp. 59–73). University of New York Press. Chalmers, D. J. (1992). Subsymbolic computation and the Chinese room. In J. Dinsmore (Ed.), The symbolic and connectionist paradigms: Closing the gap (pp. 25–48). Erlbaum. Clark, A. (1997). Being there: Putting brain, body, and world together again. MIT Press. https://doi.org/10.7551 /mitpress/1552.001.0001 Clark, H. H., & Fischer, K. (2022). Social robots as depictions of social agents. Behavioral and Brain Sciences, 28, 1–33. https://doi.org/10.1017/S0140525X22000668, PubMed: 35343422 Coeckelbergh, M. (2018). How to describe and evaluate “deception” phenomena: Recasting the metaphysics, ethics, and politics of ICTs in terms of magic and performance and taking a relational and narrative turn. Ethics and Information Technology, 20, 71–85. https://doi.org/10.1007/s10676-017-9441-5 Coradeschi, S., & Saffiotti, A. (2003). An introduction to the anchoring problem. Robotics and Autonomous Systems, 43(2–3), 85–96. https://doi.org/10.1016/S0921-8890(03)00021-6 Damiano, L., & Dumouchel, P. (2018). Anthropomorphism in human–robot co-evolution. Frontiers in Psychology, 9, 468. https://doi.org/10.3389/fpsyg.2018.00468, PubMed: 29632507 Dennett, D. C. (1971). Intentional systems. Journal of Philosophy, 68(4), 87–106. https://doi.org/10.2307 /2025382 Dennett, D. C. (1989). The intentional stance. MIT Press. Dorffner, G., & Prem, E. (1993). Connectionism, symbol grounding, and autonomous agents. In Proceedings of the fifteenth annual meeting of the Cognitive Science Society (pp. 144–148). Erlbaum. Dreyfus, H. (1979). What computers can’t do (Rev. ed.). Harper and Row. (Original work published 1972). Duffy, B. (2003). Anthropomorphism and the social robot. Robotics and Autonomous Systems, 42(3–4), 177–190. https://doi.org/10.1016/S0921-8890(02)00374-3 Duffy, B., & Zawieska, K. (2012). Suspension of disbelief in social robotics. In 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication (pp. 484–489). IEEE. https://doi.org/10.1109/ROMAN.2012.6343798 Fiala, B. Arico, A., & Nichols, S. (2014). You, robot. In E. Machery & E. O’Neill (Eds.), Current controversies in experimental philosophy. Routledge. https://doi.org/10.4324/9780203122884-5 Fisher, J. A. (1996). The myth of anthropomorphism. In M. Bekoff & D. Jamieson (Eds.), Readings in animal cognition (pp. 3–16). MIT Press. Froese, T., & Ziemke, T. (2009). Enactive artificial intelligence: Investigating the systemic organization of life and mind. Artificial Intelligence, 173(3–4), 466–500. https://doi.org/10.1016/j.artint.2008.12.001 Fuchs, T. (2022). Understanding Sophia? On human interaction with artificial agents. Phenomenology and Cognitive Sciences. https://doi.org/10.1007/s11097-022-09848-0 Fussell, S. R., Kiesler, S., Setlock, L. D., & Yew, V. (2008). How people anthropomorphize robots. In 2008 ACM/IEEE international conference on human-robot interaction (pp. 145–152). IEEE. https://doi.org/10.1145 /1349822.1349842 Gray, H. M., Gray, K., & Wegner, D. M. (2007). Dimensions of mind perception. Science, 315(5812), 619. https://doi.org/10.1126/science.1134475, PubMed: 17272713 Guthrie, S. E. (1997). Anthropomorphism: A definition and a theory. In R. W. Mitchell, N. S. Thompson, & H. L. Miles (Eds.), Anthropomorphism, anecdotes, and animals (pp. 59–73). University of New York Press. Harnad, S. (1989). Minds, machines and Searle. Journal of Experimental and Theoretical Artificial Intelligence, 1(1), 5–25. https://doi.org/10.1080/09528138908953691 Harnad, S. (1990). The symbol grounding problem. Physica D, 42, 335–346. https://doi.org/10.1016/0167 -2789(90)90087-6 Heider, F., & Simmel, M. (1944). An experimental study of apparent behavior. American Journal of Psychology, 57(2), 243–259. https://doi.org/10.2307/1416950 Malle, B. F., & Knobe, J. (1997). The folk concept of intentionality. Journal of Experimental Social Psychology, 33(2), 101–121. https://doi.org/10.1006/jesp.1996.1314 364 Artificial Life Volume 29, Number 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots Malle, B. F., Moses, L. J., & Baldwin, D. A. (2004). Introduction: The significance of intentionality. In B. F. Malle, L. J. Moses, & D. A. Baldwin (Eds.), Intentions and intentionality: Foundations of social cognition (pp. 1–24). MIT Press. Marshall, A., & Davies, A. (2019). Uber’s self-driving car didn’t know pedestrians could jaywalk. Wired, May 11. http://www.wired.com/story/ubers-self-driving-car-didnt-know-pedestrians-could-jaywalk/ Melson, G. F., Kahn, P. H. Jr., Beck, A., & Friedman, B. (2006, July 17). Toward understanding children’s and adults’ encounters with social robots [Paper presentation]. AAAI Workshop on Human Implications of Human-Robot Interaction, Boston, MA, United States. Mitchell, M. (2021). Why AI is harder than we think. https://doi.org/10.48550/arXiv.2104.12871 Mitchell, S. D. (2005). Anthropomorphism and cross-species modeling. In L. Daston & G. Mitman (Eds.), Thinking with animals (pp. 100–118). Columbia University Press. Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers, and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai .2021.100041 Papagni, G., & Koeszegi, S. A. (2021). Pragmatic approach to the intentional stance—semantic, empirical and ethical considerations for the design of artificial agents. Minds and Machines, 31, 505–534. https://doi.org /10.1007/s11023-021-09567-6 Perez-Osorio, J., & Wykowska, A. (2020). Adopting the intentional stance toward natural and artificial agents. Philosophical Psychology, 33(3), 369–395. https://doi.org/10.1080/09515089.2019.1688778 Schellen, E., & Wykowska, A. (2019). Intentional mindset toward robots—Open questions and methodological challenges. Frontiers in Robotics and AI, 5, 139. https://doi.org/10.3389/frobt.2018.00139, PubMed: 33501017 Sciutti, A., Bisio, A., Nori, F., Metta, G., Fadiga, L., & Sandini, G. (2013). Robots can be perceived as goal-oriented agents. Interaction Studies, 14(3), 329–350. https://doi.org/10.1075/is.14.3.02sci Searle, J. R. (1980). Minds, brains, and programs. Behavioral and Brain Sciences, 3(3), 417–457. https://doi.org /10.1017/S0140525X00005756 Searle, J. R. (1990). Is the brain’s mind a computer program? Scientific American, January, 26–31. https://doi .org/10.1038/scientificamerican0190-26, PubMed: 2294583 Searle, J. R. (1999). Mind, language and society: Philosophy in the real world. Basic Books. Sharkey, A., & Sharkey, N. (2011). Children, the elderly, and interactive robots. IEEE Robotics and Automation Magazine, 18(1), 32–38. https://doi.org/10.1109/MRA.2010.940151 Sharkey, A., & Sharkey, N. (2021). We need to talk about deception in social robotics! Ethics and Information Technology, 23, 309–316. https://doi.org/10.1007/s10676-020-09573-9 Sharkey, N., & Ziemke, T. (2001). Mechanistic versus phenomenal embodiment: Can robot embodiment lead to strong AI? Cognitive Systems Research, 2(4), 251–262. https://doi.org/10.1016/S1389-0417(01)00036-5 Steels, L. (1993). The artificial life roots of artificial intelligence. Artificial Life, 1(1_2), 75–110. https://doi.org /10.1162/artl.1993.1.1_2.75 Steels, L. (2008). The symbol grounding problem has been solved, so what’s next? In M. de Vega, A. Glenberg, & A. Graesser (Eds.), Symbols and embodiment: Debates on meaning and cognition (pp. 223–244). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199217274.003.0012 Steels, L., & Brooks, R. A. (1995). The Artificial Life route to artificial intelligence: Building embodied, situated agents. Routledge. Suchman, L. A. (1987). Plans and situated actions: The problem of human–machine communication. Cambridge University Press. Sytsma, J., & Machery, E. (2010). Two conceptions of subjective experience. Philosophical Studies, 151(2), 299–327. https://doi.org/10.1007/s11098-009-9439-x Taube, M. (1961). Computers and common sense: The myth of thinking machines. Columbia University Press. https://doi.org/10.7312/taub90714 Thellman, S., de Graaf, M., & Ziemke, T. (2022). Mental state attribution to robots: A systematic review of conceptions, methods, and findings. ACM Transactions on Human–Robot Interaction, 11(4), 41. https:// doi.org/10.1145/3526112 Artificial Life Volume 29, Number 3 365 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3 T. Ziemke Understanding Social Robots Thellman, S., Holmgren, A., Pettersson, M., & Ziemke, T. (2023). Out of sight, out of mind? Investigating people’s assumptions about object permanence in self-driving cars. In HRI ’23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, March 2023 (pp. 602–606). Association for Computing Machinery Digital Library. https://doi.org/10.1145/3568294.3580156 Thellman, S., Pettersson, M., Holmgren, A., & Ziemke, T. (2023). In the eyes of the beheld: Do people think that self-driving cars see what human drivers see? In HRI ’23: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, March 2023 (pp. 612–616). Association for Computing Machinery Digital Library. https://doi.org/10.1145/3568294.3580158 Thellman, S., Silvervarg, A., & Ziemke, T. (2017). Folk-psychological interpretation of human vs. humanoid robot behavior: Exploring the intentional stance toward robots. Frontiers in Psychology, 8, 1962. https://doi .org/10.3389/fpsyg.2017.01962, PubMed: 29184519 Thellman, S., & Ziemke, T. (2019). The intentional stance toward robots: Conceptual and methodological considerations. In The 41st annual conference of the Cognitive Science Society, July 24–26, Montreal, Canada (pp. 1097–1103). Cognitive Science Society. Thellman, S., & Ziemke, T. (2020). Do you see what I see? Tracking the perceptual beliefs of robots. iScience, 23(10), 101625. https://doi.org/10.1016/j.isci.2020.101625, PubMed: 33089112 Thellman, S., & Ziemke, T. (2021). The perceptual belief problem: Why explainability is a tough challenge in social robotics. ACM Transactions on Human–Robot Interaction, 10(3), 29. https://doi.org/10.1145/3461781 Urquiza-Haas, E. G., & Kotrschal, K. (2015). The mind behind anthropomorphic thinking: Attribution of mental states to other species. Animal Behaviour, 109, 167–176. https://doi.org/10.1016/j.anbehav.2015 .08.011 Varela, F. J. (1992). Autopoiesis and a biology of intentionality. In B. McMullin & N. Murphy (Eds.), Proceedings of Autopoiesis and Perception: A Workshop with ESPRIT BRA 3352 (pp. 4–14). Dublin City University. Varela, F. J. (1997). Patterns of life: Intertwining identity and cognition. Brain and Cognition, 34(1), 72–87. https://doi.org/10.1006/brcg.1997.0907, PubMed: 9209756 Varela, F. J., & Bourgine, P. (Eds.) (1992). Toward a practice of autonomous systems. MIT Press. Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind: Cognitive science and human experience. MIT Press. https://doi.org/10.7551/mitpress/6730.001.0001 von Uexküll, J. (1982). The theory of meaning. Semiotica, 42(1), 25–79. https://doi.org/10.1515/semi.1982 .42.1.25 Ziemke, T. (1997). Rethinking grounding. In A. Riegler & M. Peschl (Eds.), Does representation need reality? Proceedings of the international conference “New Trends in Cognitive Science” (NTCS 97)—Perspectives from cognitive science, neuroscience, epistemology, and artificial life (pp. 87–94; Technical Report No. 9701). Austrian Society for Cognitive Science. Ziemke, T. (1999). Rethinking grounding. In A. Riegler, M. Peschl, & A. von Stein (Eds.), Understanding representation in the cognitive sciences (pp. 177–190). Plenum Press. https://doi.org/10.1007/978-0-585-29605 -0_20 Ziemke, T. (2020). Understanding robots. Science Robotics, 5(46), eabe2987. https://doi.org/10.1126 /scirobotics.abe2987, PubMed: 32999050 Ziemke, T. (2022). Embodiment in cognitive science and robotics. In A. Cangelosi & M. Asada (Eds.), Cognitive robotics (pp. 213–229). MIT Press. https://doi.org/10.7551/mitpress/13780.003.0016 Ziemke, T., & Sharkey, N. (2001). A stroll through the worlds of robots and animals: Applying Jakob von Uexküll’s theory of meaning to adaptive robots and artificial life. Semiotica, 134(1/4), 701–746. https://doi.org/10.1515/semi.2001.050 Ziemke, T., & Thellman, S. (2023). How puzzling is the social artifact puzzle? Behavioral and Brain Sciences, 46, e50. https://doi.org/10.1017/S0140525X22001571, PubMed: 37017073 Zlatev, J. (2001). The epigenesis of meaning in human beings, and possibly in robots. Minds and Machines, 11(2), 155–195. https://doi.org/10.1023/A:1011218919464 Zlatev, J. (2002). Meaning = Life (+ Culture): An outline of a unified biocultural theory of meaning. Evolution of Communication, 4(2), 253–296. https://doi.org/10.1075/eoc.4.2.07zla 366 Artificial Life Volume 29, Number 3 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 9 3 3 5 1 2 1 5 4 2 5 2 a r t l / _ a _ 0 0 4 0 4 p d . f b y g u e s t t o n 0 8 S e p e m b e r 2 0 2 3

下载pdf