Mohammadhossein
Moghimi*
School of Electronic, Electrical and
Systems Engineering
University of Birmingham

Robert Stone
School of Electronic, Electrical and
Systems Engineering
University of Birmingham

Pia Rotshtein
School of Psychology
University of Birmingham

Neil Cooke
School of Electronic, Electrical and
Systems Engineering
University of Birmingham

Presence, Bd. 25, NEIN. 2, Frühling 2016, 81–107

doi:10.1162/PRES_a_00249

Influencing Human Affective
Responses to Dynamic Virtual
Environments

Abstrakt

Detecting and measuring emotional responses while interacting with virtual reality
(VR), and assessing and interpreting their impacts on human engagement and
‘‘immersion,’’ are both academically and technologically challenging. While many
researchers have, in the past, focused on the affective evaluation of passive environ-
gen, such as listening to music or the observation of videos and imagery, virtual real-
ities and related interactive environments have been used in only a small number of
research studies as a mean of presenting emotional stimuli. This article reports the first
stage (focusing on participants’ subjective responses) of a range of experimental inves-
tigations supporting the evaluation of emotional responses within a virtual environ-
ment, according to a three-dimensional (Valence, Arousal, and Dominance) model of
affects, developed in the 1970s and 1980s. To populate this three-dimensional model
with participants’ emotional responses, an ‘‘affective VR,’’ capable of manipulating users’
emotions, has been designed and subjectively evaluated. The VR takes the form of a
dynamic ‘‘speedboat’’ simulation, Elemente (controllable VR parameters) of which were
assessed and selected based on a 35-respondent online survey, coupled with the
implementation of an affective power approximation algorithm. A further 68 partici-
pants took part in a series of trials, interacting with a number of VR variations, while
subjectively rating their emotional responses. The experimental results provide an early
level of confidence that this particular affective VR is capable of manipulating individu-
als’ emotional experiences, through the control of its internal parameters. Darüber hinaus,
the approximation technique proved to be fairly reliable in predicting users’ potential
emotional responses, in various affective VR settings, prior to actual experiences.
Endlich, the analysis suggested that the emotional response of the users, with different
gender and gaming experiences, could vary, when presented with the same affective
VR situation.

1

Einführung

Virtual reality (VR), and interactive 3D environments generally, have expe-

rienced a significant ‘‘comeback’’ of recent years, courtesy of developments in
the gaming industry and the relentless demand for high-fidelity escapist experi-
ences on the part of gamers and simulation users alike. Noch, despite many interna-
tional initiatives involving the design and development of highly innovative and
affordable human–computer interaction (HCI) technologies in the quest for the

ª 2016 vom Massachusetts Institute of Technology

*Correspondence to m.moghimi@pgr.bham.ac.uk.

Moghimi et al. 81

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

82 PRESENCE: VOLUME 25, NUMBER 2

ultimate ‘‘immersive’’ experience,1 some believe that true
‘‘immersion’’ may only ever be achieved through the use
of advanced brain–computer interfaces (BCI) (Cairns,
Cox, Berthouze, Dhoparee, & Jennett, 2006). Jedoch,
until that day arrives, it is important to understand how it
may be possible to measure and, In der Tat, influence human
engagement and emotional connectivity with virtual
worlds using psychophysiological techniques.

The term immersion has most often been used to
describe the multisensory experience of presence by indi-
viduals, while performing a task in VR. Jedoch, differ-
ent researchers have suggested different definitions for
this term (Braun & Cairns, 2004). As an illustration,
Cairns et al. suggested that immersion could be defined
as a feeling of being deeply engaged when people enter a
make-believe world and feel as if it is real (Cairns, Cox,
Berthouze, Dhoparee, & Jennett, 2006). In 2004,
Brown and Cairns suggested that immersion can be di-
vided into three levels: engagement (during which the
users invest time, effort, and most importantly attention),
engrossment (the time that the user’s emotions are
directly affected by the environment), and total immer-
sion (when the users are detached from reality and the
virtual world is, for them, all that matters). They claim
that engagement and engrossment could be achieved
much easier than a total level of immersion, believing
instead that it could be achieved by overcoming other
barriers. Such barriers include empathy as a ‘‘growth of
attachment’’ to the environment, and atmosphere as rep-
resenting the VR’s environmental realism. The authors
also mentioned that ‘‘total immersion can be difficult to
achieve: there are barriers to immersion from both the
human and the system perspectives’’ (Braun & Cairns).
Other researchers combine the immersive experience in
virtual realities and 3D environments with the term pres-
enz, which is defined as ‘‘the extent to which a person’s
cognitive and perceptual systems are tricked into believ-
ing they are somewhere other than their physical loca-
tion’’ (Patrick, Cosgrove, Slavkovie, Rode, Verratti, &
Chiselko, 2000). Based on the variety of definitions of

immersion evident in the literature, several discussions
have been presented on the topic of how to evaluate
immersive experiences. Many believe that true immersion
might even be impossible to achieve with the present
state of maturity in VR and gaming technologies. Others
believe it could be achieved simply by defining the term
more appropriately (Braun & Cairns).

To date, HCI systems designers have, in their

attempts to increase the sense of end user immersion,
introduced several multidimensional input/output devi-
ces, in order to provide user-friendly, intuitive techni-
ques and styles of interaction with real-time 3D worlds.
Jedoch, the area of HCI research that strives toward
establishing direct communication between a computer
system and the human brain has, until recently, been
treated as science fiction.2,3 In the HCI domain, BCI
systems attempt to improve human–computer interac-
tion and increase the sense of immersion by interfacing
directly with the human brain and, daher, removing the
artificial barriers to intuitive interaction afforded by con-
ventional input-display techniques. This new interface
channel has the potential to introduce a large number of
new communication techniques in advanced HCI sys-
Systeme, and may be able to improve the interaction process
considerably (z.B., translating imaginary movements to
virtual actions, improving levels of concentration, affect-
ing emotional states, usw.). So far the interaction process
has been based mostly on conventional methods, in that
computer users typically use physical interaction devices
to see, hören, Akt, sense haptic or olfactory stimuli, und in
some cases even talk to the system. The near-term goal
of BCI systems, as an extension to these conventional
Systeme (as opposed to a replacement, which is a longer-
term aspiration), would be to translate human thoughts
and emotions by direct connection to the human brain
and use this information as a new modality channel for
HCI systems (Nijholt, Plass-Oude, & Reuderink, 2009).
Turning briefly to the field of VR and the relevance of

issues of affect, to date, researchers have studied the
implementation of virtual realities in many different
domains. As well as entertainment, virtual realities and

1. Witness, Zum Beispiel, the wide range of visual displays, Daten
inputs, haptic, and other forms of devices available from ‘‘crowd-
funding’’ platforms, such as Kickstarter and Indiegogo.

2. https://en.wikipedia.org/wiki/The_Matrix
3. https://en.wikipedia.org/wiki/Pacific_Rim_(film)

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

Moghimi et al. 83

their so-called ‘‘serious games’’ counterparts have been
used for training purposes (Ahlberg et al., 2007; Zyda,
2005; Seymour et al., 2002), pain distraction (Mahrer &
Gold, 2009; Hoffman, Doctor, Patterson, Carrougher,
& Furness, 2000; Hoffman et al., 2004), rehabilitation
(Rizzo et al., 2002; Jack et al., 2001), and disorder ther-
apy (Parsons & Rizzo, 2008; Difede et al., 2007; Rizzo
et al., 2013; Kaganoff, Bordnick, & Fuhrmann, 2012). Der
focus of all of these studies has been to engage the
human users in an interactive virtual environment, Und
to increase the sense of presence and immersion within
ihnen, thereby effectively delivering new skills, knowl-
edge, or in some cases, acting as a form of clinical dis-
traction. In 2006, Joels et al. suggested that changes in
the excitement level (depending on pleasurable or dis-
pleasurable condition) affect the learning and memory
Verfahren. They proposed that memory performance
changes (either improvements or impairments) Sind
highly dependent on the time and context of the emo-
tional experience (Joels, Pu, Wiegert, Oitzl, & Krugers,
2006). daher, the recognition of the users’ emotions
when exposed to virtual realities, and controlling their
affective experiences within the virtual environments
(regardless of their purpose) can be as important as the
VR’s contextual outcome.

One of the subcategories of research into BCI systems
is described as affective computing. During the process of
affective computing, psychophysiological signals from
the users are recorded to enable the BCI system to
extract data of relevance to their emotional and cognitive
Staaten. This new input channel could provide several fea-
tures for an advanced HCI system attempting to support
the generation of believable immersive experiences. Als
an illustration, the system could use this information to
adapt itself to the user’s emotions and, by doing so,
increase his/her performance and immersion levels dur-
ing the interaction process. Kürzlich, new techniques in
HCI-mediated emotional recognition have been devel-
oped using noninteractive or passive environments, solch
as listening to music, or the observation of videos and
Bilder (Koelstra et al., 2012; Frantzidis, Bratsas, Papa-
delis, Konstantinidis, Pappas, & Bamidis, 2010; Yazdani,
Lee, & Ebrahimi, 2009; Rizon, Murugappan, Nagara-
jan, & Yaacob, 2008; Murugappan, Rizon, Nagarajan,

Yaacob, Hazry, & Zunaidi, 2008; Katsis, Katertsidis,
Ganiatsas, & Fotiadis, 2008; Takahashi & Tsukaguchi,
2003). Others are now beginning to focus on virtual
realities and more interactive environments (z.B., Par-
nandi, Son, & Gutierrez-Osuna, 2013; Wu et al., 2010;
Antje, Peter, Markert, Meer, & Voskamp, 2005).

To perform the affect recognition process in virtual real-

ities, first the affective features of VR environments need
to be carefully investigated. Zweite, a system has to be
designed, trained, and validated with respect to a psycho-
physiological affective database, recorded from a large
number of users exposed to a number of controlled and
known affective stimuli (considering supervised learning
Algorithmen; Mohri, Rostamizadeh, & Talwalkar, 2012).
To construct such a database, a number of controlled
emotional situations (evoking some specific affective states
on the part of the users4), would need to be presented to
participants in an experiment, while taking part in a physi-
ological measurement paradigm. These recordings,
tagged by the corresponding affective states, would then
be analyzed for the design, Ausbildung, and validation of the
affect recognition system. daher, two distinct steps in
the psychophysiological affective database construction
can be considered: (A) evoking controlled emotional expe-
riences and (B) the measurement of physiological parame-
ters. It would be important to ensure the implementation
of strict experimental designs in such a paradigm, in order
to avoid the development of an inappropriate psychophy-
siological affective database, which would invalidate the
recognition system’s training process. As an illustration, Wenn
the users’ emotional experiences were poorly controlled
(z.B., it was not possible to state with confidence that an-
ger had been experienced by the users during the corre-
sponding session), then the classification techniques
would be unable to train the affect recognition system
properly and the accuracy of the system would be affected
entsprechend. To prevent such incidents, the emotional
stimuli must be subjectively evaluated and categorized
prior to the undertaking of physiological measurements,
in order to validate their effectiveness in evoking the
required emotional experiences on the part of all users.

4. Such as images that evoke fear and disgust in users—image num-

ber 3000 Zu 3266 in IAPS (Lang, Bradley, & Cuthbert, 2008).

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

84 PRESENCE: VOLUME 25, NUMBER 2

To date, a number of evaluated affective stimuli data-

bases using images (the International Affective Picture
System—IAPS; Lang, Bradley, & Cuthbert, 2008),
Geräusche (the International Affective Digital Sounds—
IADS; Bradley & Lang, 1999), and video clips (Baveye,
Bettinelli, Dellandrea, Chen, & Chamaret, 2013) have
been presented in the literature. These established
databases provide investigators with a variety of pre-
evaluated affective stimuli, welche (from a subjective
outcome perspective) have been found to elicit specific
(and quite strong) emotions in recipients. Jedoch, Zu
the knowledge of the authors, no validated affective
VR-based stimuli database has been presented as yet.
The availability and reliability of such a database of stim-
uli in form of a virtual reality (VR) is crucial in the design
and validation of an affective computing system, welche
can be used in VR-based systems.

In the present article, an affective virtual reality and the

process by which it was conceptualized, designed, Und
subjected to an early validation study is discussed in detail.
The affective VR is capable of eliciting multiple emotions
within the users, and manipulating their affective experi-
ences within a 3-dimensional affective space (described
später), by controlling the VR’s internal parameters. A
number of ‘‘sub-games’’ (based on the selection of multi-
ple unique VR parameters) have been selected using an
affective power estimation process, and evaluated in a
subjective experiment employing 68 Teilnehmer. Der
work described in this article represents a number of early
steps in research that is working toward a more compre-
hensive psychophysiological understanding of the future
role of brain–computer interfaces in VR and so-called se-
rious games. The affective VR described herein is sup-
posed to be used in construction of an affective physiologi-
cal database, to be used in the conceptualization, Design,
and validation of an affect recognition system.

2 Model of Affects, Self-Assessment, Und

Affective Clusters

2.1 Model of Affects

One of the most important challenges in the study

of emotions is the definition one adopts. Bradley, In
2006, stated that ‘‘part of the complexity in studying

emotion is defining it: there are almost as many defini-
tions as there are investigators’’ (Bradley & Lang, 2006).
The common factor among all of these definitions is that
of physiological effects, broadly reflecting the fact that,
in emotional situations, the body reacts and performs
entsprechend. In high-tempo, high-pressure contexts, für
Beispiel, the heart rate changes, sweating occurs, Die
muscles tense, facial expressions such as smiling and
frowning appear, and many other less overt
physiological changes take place (Bradley & Lang,
2006). The term emotion has been presented by some
researchers in the form of either quantitative (dimen-
sional) or qualitative (categorical) Modelle, often referred
to as affective space.

In qualitative models, the affective space is presented
by using an emotion set (a number of emotional labels),
such that the user can be ‘‘categorized’’ as experiencing
either one or a mixture of these emotional labels. Als
illustration, Ekman and Friesen (2003) used a qualitative
presentation of emotions, categorizing them as surprise,
fear, disgust, anger, happiness, and sadness. Researchers
have introduced several emotion sets, although there are
some common strong emotions that are present in most
of them. These strong emotions include anger, fear, dis-
Böe, excitement, happiness, sadness, and boredom
(Bradley & Lang, 2006).

In contrast to the work by Bradley and Lang, beide
Russell and Mehrabian presented two similar quantita-
tive models in the 1980s and 1970s. Diese Modelle
define emotions based on two or three continuous inde-
pendent parameters (dimensions or axes) (Mehrabian,
1970; Russell, 1980). Mehrabian introduced three inde-
pendent quantities: Valence, defining pleasure and dis-
pleasure; Arousal, describing the excitation level; Und
Dominance, identifying the level of control within a
given situation. Russell, andererseits, ignored
Dominance, and created a 2-dimensional Circumplex of
Affect. Mehrabian and Russell believed that representa-
tion of verbal labels of emotions within either the 2D- oder
3D-model would differ between people with different
Kulturen, especially those with different languages (Meh-
rabian, 1970; Russell, 1980). In 1980, Russell repre-
sented some the most common English verbal labels,
within his Circumplex of Affect (shown in Figure 1).

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

Moghimi et al. 85

(z.B., high heart rate tempo means high arousal status),
and the emotional status of the users is evaluated accord-
ingly (Takahashi & Tsukaguchi, 2003). daher, als
self-assessment has been employed by the majority of
research studies, and also due to the lack of availability of
a psychologist or human emotion expert in the present
Studie, it was decided to employ self-assessment techniques
in the emotional evaluation process.

2.3 Self-Assessment

In der vorliegenden Studie, both qualitative and quantita-

tive affective spaces were employed within the experi-
ments when performing participants’ self-assessments.
The participants were asked to evaluate their emotional
experiences and self-report them in the 3-dimensional
affective space (as conceived by Mehrabian—scaled arbi-
trarily from (cid:2)3 to þ3), while each axis was defined and
presented to the participants as discussed next. To per-
form the emotional assessment in the 3D model of
affect, an interactive version of Bradley and Lang’s Self-
Assessment Manikin (SAM) questionnaire (Bradley &
Lang, 1994), was employed to enable participants to
self-report their Valence, Arousal, and Dominance levels.

1. Valence: How pleasurable this gaming experience
War. Higher positive values mean more pleasure
(z.B., you enjoyed it), and higher negative values
mean more displeasure (z.B., you did not enjoy it).

2. Arousal: How arousing this gaming experience
War. Higher positive values mean more aroused
(z.B., excited, alert, stressful, usw.), and higher neg-
ative values mean minimally aroused (z.B., relaxed,
tired, bored, usw.).

3. Dominance: How much control you had in this
gaming experience. Higher positive values mean
higher control in the game (z.B., proper controller
response, ability to perform required maneuvers,
usw.), and higher negative values mean lower con-
trol during game-play (z.B., inability in performing
required maneuvers, usw.).

The dimensional assessment was followed by a qualita-

tive eight-label assessment (labels: Relaxed, Content,
Happy, Excited, Angry, Afraid, Sad, and Bored). Diese

Figur 1. Simplified Russell Circumplex of Affect for English verbal
labels of emotions (Russell, 1980).

2.2 Emotional Experience Assessment

In the affective psychophysiological database con-
struction process, each session, in which participants are
exposed to an affective stimulus, has to be tagged by an
emotional experience state, within an affective space
(qualitatively or quantitatively or both). This assessment
has to be able to reliably categorize the participants’
emotional experience.

So far, researchers have, in the main, employed either

self or expert assessments. In expert assessments, a psy-
chologist or human emotion expert would be instructed
to evaluate the participant’s affective state, and catego-
rize it within an affective space (Katsis, Katertsidis,
Ganiatsas, & Fotiadis, 2008). Jedoch, the majority of
studies appear to employ self-assessment techniques to
evaluate participants’ emotional states within an affective
Raum (Murugappan, Rizon, Nagarajan, Yaacob, Hazry,
& Zunaidi, 2008; Rizon, Murugappan, Nagarajan, &
Yaacob, 2008; Frantzidis, Bratsas, Papadelis, Konstanti-
nidis, Pappas, & Bamidis, 2010). In this process the user
is instructed to evaluate his/her affective state according
to a particular model (either qualitatively or quantita-
aktiv). Andererseits, in manchen Fällen, a pre-
emotional hypothesis was presented prior to the experi-
ment. As an illustration, a certain physiological behavior
is considered as a result of a specific emotional status

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

86 PRESENCE: VOLUME 25, NUMBER 2

Figur 2. Presentation of 8 English verbal labels of emotions within the 3D affective model. Dots present the location of
each label within the 3D space. The dashed boxes present the affective clusters—the clusters’ names (PVLAPD, PVHPAPD,
NVPAND, and NVNAND) have been presented within the dashed boxes. The dashed vectors are the affective clusters’
centroid vectors.

eight labels were selected as they were assumed to be rel-
evant to most VR experiences, and equally distributed
along the multidimensional space (siehe Abbildung 1).

2.4 Affective Clusters—Subjective
Experiment

2.4.1 Participants and Method. To subjectively

evaluate the position of the selected eight labels within
the Circumplex of Affect (siehe Abbildung 1), a questionnaire
was designed and presented to all participants (103 In
total, with a mean age of 23.23 Jahre, and a distribution
von 52% male and 46% gamers), who partook in all experi-
gen (see Experiments 1 Und 2 in Sections 4 Und 5).
The purpose behind this experiment was to assess the
placement of the eight presented labels within the Rus-
sell’s Circumplex of Affect, within the gaming and VR
experience context.

The questionnaire contained eight questions, jeder von
which required the participants to locate one of the emo-
tional labels (Relaxed, Content, Happy, Excited, Angry,

Afraid, Sad, and Bored) within the 3-dimensional space.
The example given next presents one of the questions,
assessing the Relaxed label. The participants were asked
to choose one of the integer scalars, (arbitrarily) zwischen
(cid:2)3 to þ3, for each parameter.

‘‘What value of these parameters (Valence, Arousal,
and Dominance) would describe the experience of ‘Being
Relaxed’ in virtual realities?’’

2.4.2 Ergebnisse. Figur 2 presents the subjective

arrangement of these labels within the 3D (Valence,
Arousal, and Dominance) affective space. The mean rat-
ings across participants, within each axis, have been used
as the subjective position of the labels within the Cir-
cumplex. Wie in der Abbildung zu sehen ist 2, the labels follow
the Russell’s Circumplex order, while the position of
Relaxed and Content are associated with higher arousal
states than expected (compare Figure 1 and the ‘‘Valence
vs. Arousal’’ plot in Figure 2). It can be considered that
this reflects the fact that the ratings were undertaken in

l

D
Ö
w
N
Ö
A
D
e
D

F
R
Ö
M
H

T
T

P

:
/
/

D
ich
R
e
C
T
.

M

ich
T
.

/

e
D
u
P
v
A
R
/
A
R
T
ich
C
e
–
P
D

l

F
/

/

/

/

/

2
5
2
8
1
1
8
3
6
5
7
9
P
R
e
S
_
A
_
0
0
2
4
9
P
D

.

F

B
j
G
u
e
S
T

T

Ö
N
0
7
S
e
P
e
M
B
e
R
2
0
2
3

Moghimi et al. 87

the context of the gaming and VR experience. It can be
expected that relaxing while playing a game can be more
arousing than simply relaxing on a sofa, Zum Beispiel.

Andererseits, there is a high correlation between

the Valence and Dominance ratings (R (960) ¼ 0.69,
P < 0.001—this correlation can be observed in all affec- tive ratings in Experiments 1 and 2, described in Sections 4 and 5). This correlation makes the ‘‘Valence vs. Arousal’’ and ‘‘Dominance vs. Arousal’’ graphs almost identical (see Figure 2). Furthermore, it makes the ‘‘Neg- ative Valence, Positive Dominance, and Positive/Negative Arousal’’ and ‘‘Positive Valence, Negative Dominance, and Positive/Negative Arousal’’ octants5 (four octants out of eight) completely empty (containing no emotion label), while the other four contain all eight affective labels (see ‘‘Valence vs. Dominance’’ plot in Figure 2). To the contrary, the cluster containing the Relaxed and Content emotional labels is occupying the entire ‘‘Positive Valence, Negative Arousal, and Positive Domi- nance’’ and part of the ‘‘Positive Valence, Positive Arousal, and Positive Dominance’’ octants. Thus, it can be concluded that the verbal labels are not separated with respect to the octants; however, the separation is based on four affective clusters, defined as follows: 1. PVLAPD Cluster: Positive Valence, Low Positive Arousal, Positive Dominance (PVPPAPD), and Positive Valence, Negative Arousal, Positive Domi- nance (PVNAPD). Therefore this cluster can be named: Positive Valence, Low Arousal, Positive Dominance (PVLAPD)—Containing Relaxed and Content labels: 1) 0 (cid:3) Valence (cid:3) 3 2) (cid:2)3 (cid:3) Arousal (cid:3) 1.16 3) 0 (cid:3) Dominance (cid:3) 3 2. PVHPAPD Cluster: Positive Valence, High Posi- tive Arousal, Positive Dominance (PVHPAPD)— Containing Happy and Excited labels: 1) 0 (cid:3) Valence (cid:3) 3 2) 1.16 (cid:3) Arousal (cid:3) 3 3) 0 (cid:3) Dominance (cid:3) 3 5. An octant is one of the eight divisions of a Euclidean three- dimensional coordinate system, defined based on the signs of the coordinates. 3. NVPAND Cluster: Negative Valence, Positive Arousal, Negative Dominance (NVPAND)— Containing Afraid and Angry labels: 1) (cid:2)3 (cid:3) Valence (cid:3) 0 2) 0 (cid:3) Arousal (cid:3) 3 3) (cid:2)3 (cid:3) Dominance (cid:3) 0 4. NVNAND Cluster: Negative Valence, Negative Arousal, Negative Dominance (NVNAND)— Containing Sad and Bored labels: 1) (cid:2)3 (cid:3) Valence (cid:3) 0 2) (cid:2)3 (cid:3) Arousal (cid:3) 0 3) (cid:2)3 (cid:3) Dominance (cid:3) 0 2.4.3 Discussion. To be able to design the affect recognition system, an affective VR, capable of manipu- lating the users’ emotions within the entire emotional space, needs to be designed. Considering the previous discussions, it can be concluded that an affective virtual reality needs to be capable of manipulating users’ emo- tions such that they gravitate toward all four affective clusters. 3 Affective Virtual Reality Emotional stimuli play a vital role in the design and performance evaluation of any affective computing system. To date, the majority of the researchers have used images (Frantzidis, Bratsas, Papadelis, Konstantini- dis, Pappas, & Bamidis, 2010), video clips (Rizon, Mur- ugappan, Nagarajan, & Yaacob, 2008; Murugappan, Rizon, Nagarajan, Yaacob, Hazry, & Zunaidi, 2008; Yazdani, Lee, & Ebrahimi, 2009), music (Takahashi & Tsukaguchi, 2003; Koelstra, et al., 2012) and, occasion- ally, real-life scenarios (Katsis, Katertsidis, Ganiatsas, & Fotiadis, 2008) to evoke emotional experiences on the subjects. However, virtual realities, as a potentially powerful affective medium, have been used only in a small number of research studies as a means of present- ing emotional stimuli (Parnandi, Son, & Gutierrez- Osuna, 2013). The focus of this study was to design and subjectively evaluate an affective VR, capable of evoking multiple emotional experiences within the user population. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 88 PRESENCE: VOLUME 25, NUMBER 2 3.1 Affective VR Design Two different approaches are possible for design- ing an affective virtual reality, capable of evoking certain emotions: 1. Multiple VRs: A number of entirely different VRs can be designed to evoke different emotions. The first advantage of this approach is that every VR can be designed in a way that would have the maxi- mum impact on the users, in order to evoke a par- ticular emotion. The disadvantage of this method is the variability between the environments. Differ- ent environments may well result in different VR experiences, which may in turn lead to too much variability between the recorded data. This would leave no ground truth for any comparison between the independent situations. Also, each VR would take the form of a new environment for the partici- pants and could create an element of surprise in ev- ery attempt. This issue may decrease or even change the expected emotional experience on the part of the participants. 2. Single VR: A single but well-constructed virtual reality can be designed that is capable of evoking different emotions by changing the simulated envi- ronment’s internal properties. The first advantage of this approach is the minimum variability between the emotional experiences, as the back- ground environment or scenario (or so-called Neu- tral Scenario of VR—the overall theme, environ- ment, and rules of the VR) for all experiences would be the same, and changes in the parameters of the VR and incidents could elicit different emo- tions. Another advantage would be the minimum element of surprise on the participants (compared to the multiple-VRs approach). The overall VR environment, interaction algorithm, and other aspects related to the background scenario would stay the same and allow participants to concentrate on the affective parameters rather than the changes. The disadvantage of this approach is that the effec- tiveness of emotional experiences may be less influ- ential than the first method. The reason for this is that, in a multiple VR approach, one scenario can be designed to evoke boredom and another to elicit excitement, each in a very powerful way; while in this approach there is only one VR scenario, which should be capable of evoking all emotions in an effective manner. In human-centered experimentation, minimum vari- ability between VR experiences is an extremely impor- tant matter, as any acceptable analysis dealing with either affects or physiological databases has to be based on changes in emotional experiences, due to different envi- ronments (between games), rather than different personal experiences (between participants). Multiple VRs may create different experiences among participants, rather than a single VR with an overriding context, due to vari- ability between environments. Therefore, in the present project, the Single VR approach has been adopted as the design approach for the virtual affective medium. A Speedboat Simulation6 game (see Figure 3) was designed for use as the background scenario of the affec- tive VR. As the experimental cohort was anticipated to comprise both gamers and nongamers, it was decided to use a driving-based simulation with a simple directional interface style (a speedboat scenario in this case), to reduce the amount of prior gaming experience required for participation (i.e., when compared to the skills and experiences typically manifested by players of first-person combat and strategic games). Also, the creation of an environment with a very basic contextual setting in terms of graphical elements drove the choice of a speedboat simulation (as opposed to automobile driving, which typically consists of complex urban representations). Moreover, the dynamics of the environment would, it was felt, provide a wide range of possible parameters and variables that could be implemented and controlled in the environment (described in more detail in Section 3.2). In the neutral speedboat simulation environment, the user is able to navigate a small boat, freely, within a coastal virtual environment originally created for VR healthcare research (Stone & Hannigan, 2014). By 6. This simulation can be viewed at: https://www.youtube.com /watch?v=pqn-X1Z5AoM. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 89 Figure 3. Speedboat simulation environment. manipulating the VR parameters (described in more detail in Section 3.2), a number of different variations of the neutral environment were created. In this study, these variations have been called sub-games. In the majority of sub-games, there are a number of floating ‘‘ring buoys’’ on the water that the users can col- lect to gain higher scores. In all sub-games, the users can either finish the game by passing the finish line at a dis- tance (from where the game is started), or continue exploring for as long as they require (only in the sub- games, which do not have any time limitations). Regard- less of the time settings for each VR variation, no sub- game is allowed to continue beyond 5 minutes. If the participant spends longer than 5 minutes in a particular sub-game, it terminates automatically. Depending on the VR settings (Section 3.2), participants can interact with the virtual environment using either a mouse or a force-feedback joystick. The joystick is capable of dis- playing vibration effects according to simulated ‘‘water turbulence’’ and, in addition, left/right forces on the grip, simulating simple ‘‘water resistance’’ effects, cre- ated when the boat is turning. A Samsung 32-inch flat LCD screen was used to present the VR scenes, together with a Sennheiser RS-170 wireless headphone to play the environmental sound effects. 3.2 Affective Parameters (‘‘Incidents’’) In order to evoke multiple emotions in the partici- pants, a number of controllable affective parameters need to be identified and implemented within the VR. Manip- ulation of these parameters would (it was hypothesized) evoke different emotions within the participants. The general nature of these parameters or ‘‘incidents’’ needs to be studied prior to any identification or implementa- tion within the environment. 3.2.1 Categorization of Incidents. I. VR Aspects: For the purposes of this study, the pa- rameters or incidents presented in the speedboat simula- tion were categorized into four major aspects: 1. Visualization: Any aspect of the game related to visual stimulation, including lighting, textures, fi- delity, scale of the objects, realism of any action (such as avatar animation), and physical behaviors. 2. Auditory: All features of the game that are related to the auditory sense of the users, including the background music, sounds of objects, voices of avatars, and so on. 3. Interaction: Keyboard, mouse, joysticks, voice rec- ognition systems, gestural translators, and so on, all fall within the interaction category. 4. Narrative: Any aspect of the game (visual, audi- tory, and interaction) that is presented to the users in a meaningful or contextually relevant way through a narrative or background scenario. This aspect can influence the user’s perception and change his or her experience quite dramatically. As an illustration, even a game created using extraor- l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 90 PRESENCE: VOLUME 25, NUMBER 2 Table 1. Twenty-One Incidents Categorization According to VR and Timing Aspect l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . dinary visualizations, auditory, and interaction fac- tors can have different influences on users simply by the way the game’s narrative has been pre- sented. If the background presents a science fiction scenario, for example, the user may expect to expe- rience extreme levels of action, a high tempo, even fear. Yet, the same game presented with a real-life scenario as its background narrative, perhaps one that depicts a desert island or peaceful countryside setting, can create a completely different set of expectations and perceptions on the part of the user. short aggressive attack, a short screen vibration, etc.), or throughout the whole duration of a game (Game- Persistence, such as a time limitation, a change in the input device control law, etc.). 3.2.2 Incidents Identification and Assessment. According to the speedboat simulation’s environmental capabilities, 21 possible incidents were identified for implementation within the affective VR. These incidents were categorized based on their presen- tation timing, together with the VR aspects. Table 1 shows these incidents, clustered with respect to the VR aspect and timing classifications. II. Timing Aspects: Each incident in the game can be presented to the users either as a single event in the game (In-Game Discrete Event, such as a sudden sound, a Different combinations of these incidents could create different sub-games. Combination of elements within the columns can create 1444 different sub-game combi- f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 91 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . Figure 4. Incidents presentation examples within the Affective VR. A) Start line flag for time limitation scenarios. B) Ramps in the virtual environment. C) Mine avoidance. D) Jumping over ships. E) Driving freely outside the ring buoys lane. F) Finding hidden ring buoys inside the bushes, by using the radar. G) Torpedo avoidance. H) Splashing water to the flying ball. I) Inverse black and white screen in the torpedo avoidance sub-scenario. J) Black and white screen in maze sub-scenario. K) Finish line flag to terminate the game on demand. L) Score calculation at the game termination. f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Þ (cid:4) 2 C2ð Þ (cid:4) 2 C6ð Þ (cid:4) 2 C7ð Þ (cid:4) 3 C5ð nations (Ci means ithcolumn – 5 C1ð (cid:4) 2 C4ð As some of these combinations are not possible (e.g., no time limitation while the timer is faulty, etc.), the total number of possible combinations is, as a consequence, reduced to 792 different sub-games. Þ (cid:4) 2 C3ð Þ Þ ¼ 1444). Þ (cid:4) 2 C8ð 3.3 Affective Virtual Reality The speedboat VR is capable of generating all required combinations of incidents (described in Section 3.2.2). Figure 4 presents some examples of possible sub- game combinations. Each sub-game was allocated an 92 PRESENCE: VOLUME 25, NUMBER 2 8-digit code. Each code represented the index number within each column of Table 1. As an illustration, code ‘‘21223111’’ would set up a sub-game environment with the following settings: fore, a high number of participants need to be recruited, to enable each game to be played by ‘‘n’’ participants (approach II). 2. Subjective Approximation: In this method, the Mine Avoidance þ Time Limitation þ Faulty Timer þ Invisible Barrier þ Joystick with Force Feedback þ Normal Controller þ Shaking and Blurring the Camera þ Color Screen The experimenter generates a list of these codes for the VR, in order to create an automated sequential (random- ized) experiment. In addition, an interactive SAM (Bradley & Lang, 1994) questionnaire (scaled between (cid:2)3 to þ3 in all axes), followed by the eight-emotions list described earlier, was automatically presented to the user, at the end of each sub-game (Section 2.3). The rating results, fol- lowed by the sub-game information, were saved in a text file during the run-time of the experiment, and could be simply extracted after the experiment. 4 Pre-Experiment Survey (Experiment 1) 4.1 Sub-Games Affective Power Approximation As explained in Section 3.2.2, 792 different sub- games can be constructed using the 21 incidents. Two different approaches were available to test the emotional effect of each sub-game: 1. Subjective Assessment: In this approach, all sub- games need to be played at least once by one of the participants. It is impossible to allow each partici- pant to play all 792 sub-games, as no one individ- ual would be able to play all of them without expe- riencing extreme fatigue (even in multiple sessions, over different days). Therefore, all 792 sub-games can be divided into ‘‘m’’ subsets, each of which contains a number of sub-games. Then each subset can be played (I) either by ‘‘n’’ participants, or (II) by only one participant. To be able to perform a meaningful affective analysis, the affective power of sub-games cannot be assessed by subjective assess- ment of a single participant (approach I). There- emotional effect of each incident, rather than each sub-game, would be evaluated, using an approxi- mation technique. This means that each participant would estimate the possible emotional effect of each incident (all 21 incidents considered), described verbally (Section 4.2). Then, by employ- ing the approximated emotional effects of all incidents, and an estimation technique (Section 4.3), the overall affective power of each single sub-game, containing a number of incidents, can be approximated. Due to the high number of possible sub-game combi- nations, to reduce this number to include those affective combinations which would be most likely to manipulate the participants’ emotional status toward all four affec- tive clusters (described in Section 2), the Subjective Approximation approach was employed in this study. 4.2 Participants and Method Subsequently, a subjective survey was designed and presented online to 35 respondents (with a mean age of 24.72 years, and a distribution of 57% males and 66% nongamers). The study was reviewed and approved by the University of Birmingham’s Ethical Review Commit- tee (Ethical Reference Number: ERN_13-1157). To dis- tinguish gamers from nongamers, the following descrip- tion was presented to the respondents as part of the online survey to enable them to self-assess appropriately: ‘‘If you follow games in the market regularly and have a lot of experience playing games on PC and consoles, you are a gamer.’’ Within the survey, a brief overarching explanation of the VR, followed by a short video of the environment (as referenced earlier under footnote 6), was presented to the respondents. Then, each incident was described in text form, such as: ‘‘imagine that you need to drive the boat through mines scattered on the water,’’ or l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 93 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . Figure 5. Presentation of incidents within the 3D affective space. ‘‘imagine that the controller used to control the boat is faulty and is not responding to your actions.’’ The respondents were then required to estimate their Va- lence, Arousal, and Dominance levels, and to choose one of the eight emotion labels (as presented in Section 2), for each incident, by considering themselves within the described affective situation. 4.3 Results Using the mean ratings (across participants) of the respondents for each incident, the affective power of all VR parameters have been approximated within the 3- dimensional affective space and are shown in Figure 5. 1. Interacting and Additive Effect: Each individual incident can have an effect on another incident if they are both presented within the same sub-game. This means that incidents can have additive effects from each other. It also means that, if several inci- dents are presented in a sub-game, the overall emo- tional effect of that combination can be considered as the summation of Circumplex values of all indi- vidual incidents. 2. Background Game as the Neutral: The back- ground scenario can be considered as neutral, with (0, 0, 0) as its 3D emotional effect. This means that all possible combinations would be evaluated with respect to the background VR scenario. To analyze and estimate the total emotional power of Accepting these hypotheses, then, the affective power each sub-game, an estimation algorithm was designed based on the following two hypotheses. of all 792 sub-games can be estimated by adding the approximated 3D affective values of all incidents within f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 94 PRESENCE: VOLUME 25, NUMBER 2 Figure 6. Presentation of sub-games within the 3D affective space. Dots represent the sub-games. Circled dots represent the 22 selected sub-games. each combination. Figure 6 presents the positioning of all 792 sub-games within the 3-dimensional affective space. Furthermore, to estimate the Occurrence Probabil- ity (OP) of each categorical label for each game in the future subjective experiment, Equation 1 was employed in the analysis. By using this equation, the probability in which a particular label can be selected in a specific sub- game is approximated. Equation 1. Categorical Label Occurrence Probability Estimation Formula OP ¼ P ðOccurence Frequency of all incidents in a subgameÞ Number of Incidences within the subgame ð (cid:4) Number of Participants in the experiment Þ ð Þ 5 Preliminary Subjective Experiment (Experiment 2) 5.1 Sub-Games Selection for Preliminary Experiment After performing the affective power estimation process on the sub-games, the postexposure subjective affective response of the participants to a number of them can be assessed. This subjective affective evaluation can, first, assess the accuracy of the approximation process and second, identify the subjective emotional power of a num- ber of sub-games (rather than their estimated values), for future affective experiments. Therefore, a number of sub- games need to be selected to be presented to a number of participants, for subjective affective evaluation. For this experiment, it was decided to adopt an overall Experiment Duration (ED) of less than 2 hours (in order to minimize participant fatigue). Considering the maxi- mum duration of each sub-game as 5 minutes (no sub- game was permitted to take longer than 5 minutes, as described in Section 3.1, although they usually lasted less than this), and the training session (Section 0) duration of between 5 and 15 minutes (average 10 minutes), the maximum number of sub-games for the experiment (not to exceed 2 hours experiment duration), was calculated as 22 (Equation 2). Therefore the 22 most affective sub- games (those which are most likely to evoke emotional experience within the four affective clusters) have to be identified (using their approximated values) to be pre- sented in the experiment. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 95 Table 2. Seven-Dimensional Presentation of Games’ and Clusters’ Ideal Vectors Valence Arousal Dominance PVLAPD Occurrence Percentage PVHPAPD Occurrence Percentage NVPAND Occurrence Percentage NVNAND Occurrence Percentage Valence Mean Arousal Mean Value Across All Participants Value Across All Participants Dominance Mean Value Across All Participants Fraction of Fraction of Fraction of Fraction of Participants Who Chose Either Relaxed or Content Participants Who Chose Either Happy or Excited Participants Who Chose Either Angry or Afraid Participants Who Chose Either Sad or Bored Table 3. Clusters’ Ideal Vectors Cluster Valence Arousal Dominance PVLAPD Occurrence Percentage PVHPAPD Occurrence Percentage NVPAND Occurrence Percentage NVNAND Occurrence Percentage 1.5 PVLAPD PVHPAPD 1.5 NVPAND (cid:2)1.5 NVNAND (cid:2)1.5 (cid:2)0.85 2.14 1.5 (cid:2)1.5 1.5 1.5 (cid:2)1.5 (cid:2)1.5 100% 0% 0% 0% 0% 100% 0% 0% 0% 0% 100% 0% 0% 0% 0% 100% Equation 2. Calculation of Maximum Required Number of Sub-Games ED ¼ n (cid:4) 5 minutes Þ (cid:3) 2 hours ð Þ Þ þ 10 minutes ð ð n ¼ number of subgames n (cid:3) 22 5.2 Most Affective Sub-Games Selection In order to select the most affective combinations, each sub-game was presented as a vector, in a 7-dimen- sional space, presented in Table 2. The Valence, Arousal, and Dominance values were calculated through the sub- game affective power estimation algorithm (Section 4.3). In addition, the approximated Occurrence Probabil- ity (OP) for each sub-game was used to create the sub- games’ affective vectors (Section 4.3). In an ideal situation, the most affective game within each cluster would feature that cluster’s central values for Valence, Arousal, and Dominance (ideally, in the dimensional model—Section 2—the clusters’ centroids); while all participants have chosen one of the two verbal labels within that cluster (i.e., the probability of selecting either of the cluster’s labels is 100%—ideally in the cate- gorical model). Therefore the clusters’ ideal vectors could be presented as shown in Table 3. To select the most affective sub-games, the Cosine Similarity algorithm (Pang-Ning Tan, 2005) was employed to find the 4 most similar sub-game affective vectors to the clusters’ ideal vectors in each affective clus- ter. Therefore, in each cluster the 4 most powerful com- binations were selected to consider the 16 most affective sub-games, which can cover the entire 3D affects space effectively. Furthermore, 5 additional test combinations (added manually—those which have the maximum standard deviation and minimum level of agreement among participants), followed by the neutral scenario (the sub-game with background scenario settings— ‘‘12111121’’ combination) were added to create the 22- game experiment. Figure 6 presents the 22 selected sub- games among 792 combinations, highlighted with circles. Table 4 presents the arrangement of the incidents within the 22 selected sub-games. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 96 PRESENCE: VOLUME 25, NUMBER 2 Table 4. The 22 Selected Sub-Games’ Settings # 1 2 3 4 5 6 7 8 Narrative Main Scenario Free Environment Exploring Free Environment Exploring Free Environment Exploring Free Environment Exploring Free Environment Exploring Free Environment Exploring Free Environment Exploring Free Environment Exploring Interactive Visualization Time Limitation Timer Invisible Barrier Controller Type Faulty Controller Camera Screen Color No Time Limitation Normal Timer No Invisible Barrier Mouse Normal Controller No Shake or Blurring Color Screen No Time Limitation Normal Timer No Invisible Barrier Mouse Normal Controller No Shake or Blurring Black & white No Time Limitation Normal Timer No Invisible Barrier Mouse Faulty Controller No Shake or Blurring Black & white No Time Limitation Normal Timer No Invisible Barrier Mouse Faulty Controller No Shake or Blurring Inverse Black & white No Time Limitation Normal Timer No Invisible Barrier No Time Limitation Normal Timer No Invisible Barrier No Time Limitation Normal Timer Invisible Barrier Joystick Without Force Feedback Joystick With Force Feedback Mouse Normal Controller No Shake or Blurring Black & white Normal Controller No Shake or Blurring Black & white Normal Controller No Shake or Blurring Black & white No Time Limitation Normal Timer Invisible Barrier Mouse Faulty Controller No Shake or Blurring Black & white 9 Mine Avoidance Time Limitation Normal Timer No Invisible Barrier 10 Mine Avoidance Time Limitation Faulty Timer No Invisible Barrier Joystick With Force Feedback Joystick With Force Feedback Normal Controller Normal Controller Shaking and Blurring the Camera Shaking and Blurring the Camera Color Screen Color Screen l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 97 Screen Color Inverse Black & white Inverse Black & white Color Screen Color Screen Inverse Black & white Inverse Black & white l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . Table 4. (Continued) Narrative Interactive Visualization Main Scenario # Time Limitation Timer Invisible Barrier Controller Type Faulty Controller Camera 11 Mine Avoidance Time Limitation Faulty Timer Invisible Barrier Mouse Faulty Controller No Shake or Blurring 12 Mine Avoidance Time Limitation Faulty Timer Invisible Barrier Mouse Faulty Controller 13 Torpedo Avoidance Time Limitation Normal Timer No Invisible Barrier 14 Torpedo Avoidance Time Limitation Faulty Timer No Invisible Barrier 15 Torpedo Avoidance Time Limitation Faulty Timer Invisible Barrier Joystick With Force Feedback Joystick With Force Feedback Mouse Normal Controller Normal Controller Faulty Controller No Shake or Blurring Shaking and Blurring the Camera Shaking and Blurring the Camera Shaking and Blurring the Camera 16 Torpedo Avoidance Time Limitation Faulty Timer Invisible Barrier Mouse Faulty Controller Shaking and Blurring the Camera 17 Shooting a Flying Ball Time Limitation Normal Timer No Invisible Barrier 18 Shooting a Flying Ball Time Limitation Normal Timer No Invisible Barrier 19 Shooting a Flying Ball Time Limitation Normal Timer No Invisible Barrier 20 Shooting a Flying Ball Time Limitation Normal Timer No Invisible Barrier Joystick Without Force Feedback Joystick Without Force Feedback Joystick With Force Feedback Joystick With Force Feedback Normal Controller No Shake or Blurring Color Screen Faulty Controller No Shake or Blurring Color Screen Normal Controller No Shake or Blurring Color Screen Faulty Controller No Shake or Blurring Color Screen f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 98 PRESENCE: VOLUME 25, NUMBER 2 Table 4. (Continued) Narrative Interactive Visualization Main Scenario # Time Limitation Timer Invisible Barrier Controller Type Faulty Controller Camera Screen Color 21 Maze No Time Limitation Normal Timer No Invisible Barrier 22 Maze No Time Limitation Normal Timer No Invisible Barrier Joystick Without Force Feedback Joystick Without Force Feedback Normal Controller No Shake or Blurring Black & white Normal Controller Shaking and Blurring the Camera Black & white 5.3 Participants and Method 5.4 Results An experiment was conducted in which the 22 selected sub-games were presented to 68 participants (with a mean age of 24.12 years). The study was reviewed and approved by the University of Birmingham Ethical Review Committee (Ethical Reference Number: ERN_13-1157). The participants consisted of four differ- ent groups: male gamers, female gamers, male nongamers, and female nongamers (17 participants for each group— the gaming experience was subjectively assessed by the participants, according to the description presented in Section 4.2). Each experiment commenced with a training session (see Figure 7) to prepare the participants for every possible incident within the games. The training intro- duced the game environment to the participants and served to reduce any element of surprise in the games. The sessions were performed in a quiet room. All par- ticipants were provided with a 32-inch Samsung LCD display, a Microsoft Wireless Mouse 5000, a Logitech Wingman 3D force feedback joystick, and a Sennheiser RS-170 wireless headphone. On average, participants spent 58 minutes playing the games, and 1 hour, 46 minutes to complete the entire experiment. Therefore, on average, participants spent 48 minutes of the experi- ment to complete the questionnaire, or to rest between the sub-game sessions. 5.4.1 Raw Results. Table 6 presents the esti- mated (through Experiment 1) and subjectively reported (through Experiment 2) Valence, Arousal, and Domi- nance levels for each sub-game. The estimated values are calculated by adding the incident’s (VR parameter) affec- tive values (according to ‘‘Interactive and Additive Effect’’ presented in Section 4.3); therefore, the scaling is different from the sub-games’ measured ratings, which are scaled between (cid:2)3 and þ3. Table 5 presents the esti- mated and reported OP of each categorical label in each sub-game. 5.4.2 Estimated Versus Reported Correlation. Figure 8 shows the scatter plot of the esti- mated Valence, Arousal, and Dominance levels in the Pre- Experiment (Experiment 1), versus the subjectively reported levels in the Preliminary Experiment (Experi- ment 2). A comparison of the estimated values shows a high correlation factor (Pearson technique) with the reported values across Valence, Arousal, and Dominance axes (see Table 7). From this, one can conclude that not only were the participants able to accurately estimate their emotions for each incident, before the real experience (using the dimensional affective space), but also that the estimation algorithm (presented in Section 4) was suffi- l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 99 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . Figure 7. Affective VR training session. A) Practicing the maneuvering procedures. B) Explaining the radar. C) Describing the dot colors in radar and their definitions. D) Presenting the ‘‘ring buoys’’ and the scoring procedure. Table 5. Estimated Versus Subjectively Reported Categorical Levels for the 22 Selected Sub-Games # 1 2 3 4 5 6 7 8 9 Relaxed Content Happy Excited Angry Afraid Sad Bored Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage 18.41% 23.53% 17.14% 27.94% 13.97% 7.35% 15.24% 10.29% 15.87% 23.53% 13.97% 11.76% 15.24% 10.29% 13.97% 0.00% 10.16% 1.47% 14.60% 36.76% 13.65% 20.59% 11.43% 13.24% 12.38% 7.35% 13.65% 23.53% 11.43% 16.18% 12.38% 17.65% 11.43% 7.35% 10.48% 13.24% 10.79% 17.65% 9.52% 27.94% 6.98% 10.29% 9.21% 2.94% 12.38% 19.12% 8.57% 27.94% 8.25% 13.24% 6.98% 8.82% 9.84% 25.00% 17.46% 14.71% 15.56% 1.47% 12.70% 5.88% 13.97% 7.35% 16.51% 5.88% 22.54% 20.59% 14.29% 8.82% 12.70% 2.94% 32.38% 45.59% 5.08% 0.00% 5.71% 2.94% 20.63% 27.94% 14.92% 26.47% 5.71% 0.00% 9.52% 2.94% 13.33% 19.12% 20.63% 39.71% 9.84% 4.41% 2.54% 1.47% 2.22% 0.00% 2.86% 1.47% 3.49% 1.47% 1.90% 0.00% 3.17% 1.47% 2.86% 2.94% 2.86% 1.47% 6.03% 2.94% 0.00% 0.00% 2.22% 0.00% 3.17% 4.41% 1.90% 8.82% 2.22% 0.00% 2.54% 1.47% 2.86% 4.41% 3.17% 5.88% 0.32% 1.47% 31.11% 5.88% 33.97% 16.18% 28.25% 27.94% 28.89% 35.29% 31.75% 25.00% 28.25% 16.18% 30.79% 20.59% 28.25% 32.35% 20.95% 1.47% / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 100 PRESENCE: VOLUME 25, NUMBER 2 Table 5. (Continued) # 10 11 12 13 14 15 16 17 18 19 20 21 22 Relaxed Content Happy Excited Angry Afraid Sad Bored Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage Estimated Percentage Reported Percentage 8.25% 5.88% 6.35% 5.88% 4.76% 1.47% 10.16% 0.00% 8.25% 0.00% 6.35% 0.00% 4.76% 1.47% 12.38% 2.94% 11.11% 0.00% 12.06% 5.88% 10.79% 0.00% 13.97% 4.41% 12.38% 4.41% 9.52% 11.76% 8.25% 7.35% 6.67% 4.41% 10.48% 4.41% 9.52% 14.71% 8.25% 7.35% 6.67% 2.94% 12.06% 25.00% 11.11% 10.29% 11.43% 13.24% 10.48% 11.76% 12.38% 8.82% 10.79% 7.35% 9.21% 14.71% 7.30% 8.82% 6.03% 4.41% 10.16% 14.71% 9.52% 10.29% 7.62% 2.94% 6.35% 1.47% 14.92% 35.29% 13.65% 10.29% 12.38% 32.35% 11.11% 7.35% 11.75% 11.76% 10.48% 2.94% 32.70% 29.41% 20.95% 7.35% 23.17% 4.41% 31.43% 69.12% 31.75% 44.12% 20.00% 8.82% 22.22% 14.71% 23.81% 19.12% 22.22% 16.18% 27.62% 33.82% 26.03% 22.06% 16.83% 4.41% 19.05% 2.94% 15.87% 22.06% 29.52% 41.18% 33.65% 60.29% 10.48% 1.47% 16.51% 19.12% 30.16% 64.71% 34.29% 52.94% 6.98% 5.88% 14.29% 42.65% 6.67% 8.82% 13.97% 44.12% 8.57% 20.59% 12.70% 27.94% 6.03% 2.94% 6.67% 5.88% 7.62% 5.88% 6.67% 1.47% 6.67% 1.47% 7.30% 2.94% 8.25% 5.88% 5.40% 0.00% 5.40% 1.47% 5.71% 0.00% 5.71% 1.47% 2.54% 2.94% 3.49% 1.47% 0.32% 2.94% 2.54% 7.35% 2.86% 5.88% 0.32% 4.41% 0.32% 2.94% 2.54% 2.94% 2.86% 8.82% 0.32% 2.94% 0.63% 5.88% 0.32% 1.47% 0.63% 5.88% 2.54% 11.76% 2.86% 4.41% 18.10% 4.41% 18.41% 14.71% 15.24% 11.76% 20.32% 0.00% 17.46% 4.41% 17.78% 8.82% 14.60% 8.82% 24.13% 7.35% 21.59% 10.29% 23.81% 4.41% 21.27% 5.88% 31.43% 32.35% 28.25% 45.59% Table 6. Estimated Versus Subjectively Reported Dimensional Levels for the 22 Selected Sub-Games (SE Is the Standard Error) Valence Arousal Num Estimated 1 2 3 1.74 (SE ¼ 0.26) 0.2 (SE ¼ 0.26) (cid:2)2.37 (SE ¼ 0.26) Subjectively Rated 1.53 (SE ¼ 0.12) 1.23 (SE ¼ 0.15) (cid:2)0.57 (SE ¼ 0.18) Estimated (cid:2)1 (SE ¼ 0.26) (cid:2)2.43 (SE ¼ 0.26) (cid:2)0.91 (SE ¼ 0.26) Subjectively Rated 0.04 (SE ¼ 0.19) (cid:2)0.26 (SE ¼ 0.21) (cid:2)0.01 (SE ¼ 0.19) Dominance Estimated 8.54 (SE ¼ 0.22) 7.43 (SE ¼ 0.23) 2.8 (SE ¼ 0.24) Subjectively Rated 2.09 (SE ¼ 0.12) 2.64 (SE ¼ 0.09) (cid:2)0.39 (SE ¼ 0.19) l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Table 6. (Continued) Valence Arousal Num Estimated 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 (cid:2)1.49 (SE ¼ 0.26) 1.34 (SE ¼ 0.26) 2.11 (SE ¼ 0.25) (cid:2)0.94 (SE ¼ 0.26) (cid:2)2.37 (SE ¼ 0.26) 4.63 (SE ¼ 0.24) 4 (SE ¼ 0.25) (cid:2)2.29 (SE ¼ 0.26) (cid:2)2 (SE ¼ 0.27) 5 (SE ¼ 0.24) 4.37 (SE ¼ 0.25) (cid:2)1.91 (SE ¼ 0.26) (cid:2)1.63 (SE ¼ 0.27) 3.49 (SE ¼ 0.25) 2.06 (SE ¼ 0.25) 3.97 (SE ¼ 0.24) 2.54 (SE ¼ 0.24) 0.89 (SE ¼ 0.26) 1.17 (SE ¼ 0.26) Subjectively Rated (cid:2)0.6 (SE ¼ 0.16) 0.91 (SE ¼ 0.17) 1.1 (SE ¼ 0.16) 0.23 (SE ¼ 0.18) (cid:2)0.76 (SE ¼ 0.18) 1.43 (SE ¼ 0.16) 0.61 (SE ¼ 0.2) (cid:2)1.15 (SE ¼ 0.18) (cid:2)1.07 (SE ¼ 0.18) 1.85 (SE ¼ 0.15) 0.8 (SE ¼ 0.19) (cid:2)1.28 (SE ¼ 0.2) (cid:2)1.14 (SE ¼ 0.19) 1.31 (SE ¼ 0.15) (cid:2)0.45 (SE ¼ 0.21) 1.54 (SE ¼ 0.16) (cid:2)0.18 (SE ¼ 0.23) (cid:2)0.53 (SE ¼ 0.23) (cid:2)1.11 (SE ¼ 0.22) Estimated (cid:2)1.43 (SE ¼ 0.26) (cid:2)1.03 (SE ¼ 0.26) 0.57 (SE ¼ 0.25) (cid:2)1.4 (SE ¼ 0.26) (cid:2)0.91 (SE ¼ 0.26) 5.4 (SE ¼ 0.24) 6.74 (SE ¼ 0.25) 4.34 (SE ¼ 0.26) 5.37 (SE ¼ 0.21) 5.54 (SE ¼ 0.24) 6.89 (SE ¼ 0.25) 4.49 (SE ¼ 0.26) 5.51 (SE ¼ 0.27) 3.6 (SE ¼ 0.25) 4.09 (SE ¼ 0.25) 4.17 (SE ¼ 0.24) 4.66 (SE ¼ 0.24) 0.09 (SE ¼ 0.26) 1.11 (SE ¼ 0.26) Subjectively Rated (cid:2)0.15 (SE ¼ 0.19) (cid:2)0.24 (SE ¼ 0.22) 0.76 (SE ¼ 0.2) 0.23 (SE ¼ 0.19) 0.22 (SE ¼ 0.19) 1.77 (SE ¼ 0.11) 1.47 (SE ¼ 0.14) 0.6 (SE ¼ 0.2) 0.88 (SE ¼ 0.17) 2.31 (SE ¼ 0.1) 1.76 (SE ¼ 0.14) 1.03 (SE ¼ 0.16) 1.39 (SE ¼ 0.16) 1.24 (SE ¼ 0.14) 1.08 (SE ¼ 0.19) 1.69 (SE ¼ 0.13) 1.39 (SE ¼ 0.15) (cid:2)0.33 (SE ¼ 0.23) (cid:2)0.48 (SE ¼ 0.23) Moghimi et al. 101 Subjectively Rated (cid:2)0.54 (SE ¼ 0.2) 2.36 (SE ¼ 0.12) 1.61 (SE ¼ 0.16) 1.79 (SE ¼ 0.18) (cid:2)0.67 (SE ¼ 0.2) 1.42 (SE ¼ 0.16) 0.7 (SE ¼ 0.21) (cid:2)1.12 (SE ¼ 0.2) (cid:2)0.96 (SE ¼ 0.18) 1.35 (SE ¼ 0.17) 1.09 (SE ¼ 0.18) (cid:2)1.55 (SE ¼ 0.19) (cid:2)1.53 (SE ¼ 0.17) 1.81 (SE ¼ 0.16) (cid:2)1.14 (SE ¼ 0.19) 1.53 (SE ¼ 0.15) (cid:2)1.04 (SE ¼ 0.18) 1.7 (SE ¼ 0.17) 1.27 (SE ¼ 0.22) Dominance Estimated 4.23 (SE ¼ 0.23) 8.2 (SE ¼ 0.22) 6.94 (SE ¼ 0.23) 5.26 (SE ¼ 0.23) 2.8 (SE ¼ 0.24) 7.71 (SE ¼ 0.23) 6.23 (SE ¼ 0.24) 0.23 (SE ¼ 0.25) (cid:2)0.83 (SE ¼ 0.25) 7.63 (SE ¼ 0.23) 6.14 (SE ¼ 0.24) 0.14 (SE ¼ 0.25) (cid:2)0.91 (SE ¼ 0.26) 8.34 (SE ¼ 0.22) 5.89 (SE ¼ 0.23) 8.14 (SE ¼ 0.23) 5.69 (SE ¼ 0.23) 7.29 (SE ¼ 0.23) 6.23 (SE ¼ 0.23) l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 102 PRESENCE: VOLUME 25, NUMBER 2 Figure 8. Estimation vs. reported dimensional correlation. Table 7. Axis and Emotion Labels Correlation Report Label Correlation Axis Correlation Relaxed Content Happy Excited Angry Afraid Sad Bored r (22) ¼ 0.702, P ¼ 0.0003 r (22) ¼ 0.724, P ¼ 0.0001 r (22) ¼ 0.536, P ¼ 0.0100 r (22) ¼ 0.838, P < 0.0001 r (22) ¼ 0.878, P < 0.0001 r (22) ¼ 0.566, P ¼ 0.0060 r (22) ¼ 0.371, P ¼ 0.0892 r (22) ¼ 0.595, P ¼ 0.0034 Valence r (22) ¼ 0.774, P < 0.0001 Arousal r (22) ¼ 0.867, P < 0.0001 Dominance r (22) ¼ 0.837, P < 0.0001 ciently accurate to accumulate the overall affective power of each sub-game (using the dimensional affective space). Thus, it can be concluded that the presented dimensional affective power estimation algorithm appears to be capa- ble of predicting the sub-games’ affective power prior to the participants’ actual subjective experience. A comparison of the estimated versus reported cate- gorical emotional labels (occurrence probability) is pre- sented in Figure 9. As can be seen in the graphs, not only were the correlation coefficients smaller (see Table 7) compared to the dimensional model, but also they were slightly less significant (see Table 7). One can conclude that the dimensional affective power estimation process was more accurate than the categorical approximation. 5.4.3 Affective VR Effectiveness. A multivariate analysis of variance (MANOVA) showed that the differ- ent VR-combinations were an extremely important factor (PVR-Combinations < 0.001,7 Valence, Arousal, and Domi- 7. Calculated using Pillai’s Trace, Wilks’ Lambda, Hotelling’s Trace, and Roy’s Largest Root algorithms. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 103 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Figure 9. Estimation vs. reported categorical correlation. nance are considered as dependent variables and differ- ent sub-games as independent parameter) in creating dif- ferent emotional experiences (i.e., significantly different Valence, Arousal, and Dominance levels). From this one can conclude that the single controllable affective envi- ronment designed for this study has been able to manip- ulate the participants’ emotions significantly, by control- ling the VR parameters. 5.4.4 Groups Comparison. Further to the previ- ous discussion, the same (MANOVA) test supported the conclusion that both Gender (male vs. female) and Gam- ing Experience (gamer vs. nongamer) are extremely im- portant factors in the participants’ emotional experiences (PGender < 0.001, PGaming-Experience < 0.0017—Valence, Arousal, and Dominance are considered as dependent variables and gender and gaming-experience as inde- pendent parameters). Thus it can be concluded that the emotional experiences of each group (male-gamer, male- nongamer, female-gamer, and female-nongamer) are sig- nificantly different from each other. One of the most important challenges of designing any affective psychophysiological database is the minimi- zation of variability between participants (in each indi- vidual sub-game), while maximizing the variability between sub-games’ experiences. This is due to the fact that, in any human-centered experiment, minimum vari- ability between participants’ experiences in a single VR session is an extremely important issue. Any acceptable analysis, dealing with either affects or physiological data- bases, should, intuitively, be based on changes in emo- tional experiences, due to different environments, rather than different personal experiences. Therefore, to reveal the similarity level between all 4 participant groups (male-gamer, male-nongamer, female-gamer, and female-nongamer), the Cosine Simi- larity Algorithm (Pang-Ning Tan, 2005) was once again employed. Table 8 presents the mean similarity compari- son levels, across games, for the four groups, in order of the similarity level. As can be seen, the male gamers, male nongamers, and female gamers are the most similar groups, compared to the female nongamers (according 104 PRESENCE: VOLUME 25, NUMBER 2 Table 8. Groups Similarity Comparison Table Group Comparison Mean Similarity Level Across Games Male Gamer Vs. Male Non-Gamer Male Gamer Vs. Female Gamer Male Non-Gamer Vs. Female Gamer Female Gamer Vs. Female Non-Gamer Male Non-Gamer Vs. Female Non-Gamer Male Gamer Vs. Female Non-Gamer 94.23% (61.27%) 93% (61.54%) 90.98% (62.07%) 78.48% (68.88%) 77.22% (69.29%) 74.47% (610.82%) to higher average and lower standard deviation in simi- larity levels, across the games). This means that in an affective VR situation, the emotional experience of male gamers, male nongamers, and female gamers are very similar, particularly when compared to the female nongamers. 6 Discussion The analysis presented in this study shows a num- ber of early evaluation results based on an affective vir- tual reality scenario. The affective VR, based on a speed- boat simulation, is capable of evoking multiple controllable emotional experiences within the users, through changes of its internal parameters. The analysis has highlighted the ability of this affective VR to manip- ulate the users’ emotional experiences effectively, within the affective space (examined on both dimensional and categorical models), by combining a variety of affective incidents (VR parameters) within a number of sub- games. Moreover, the results suggest that the affective power approximation algorithm (also presented in this study) has been able to evaluate the emotional effect of each sub-game, fairly accurately, prior to the real experi- ence of the users. The analysis concluded that the approximation algorithm has been able to estimate the emotional experience of the users more accurately in the dimensional affective space when compared to the cate- gorical model. In addition, the analysis highlighted the fact that gender and gaming experience are significant factors in experiencing different emotional experience on the part of the users. This means that the male-gamers, male-nongamers, female-gamers, and female-nongamers would have significantly different emotional experiences (compared to each other), if exposed to a similar affec- tive stimulus. However, despite the different emotional experiences, the similarity level comparison revealed that male gamers, male nongamers, and female gamers have a higher similarity level in their affective responses, with least variability, when compared to the female non- gamers. 7 Conclusions The human–computer interface has become one of the most important research topics in computer science since the introduction of the first ‘‘computers’’ (calcula- tors) in the 17th century. Today, highly complex real- time computer-based systems and their interfaces with human operators are undergoing an evolution on a hith- erto unheard-of scale, in what has become a quest to ensure that they become synergistic, even symbiotic with their human users—transparent, usable, intuitive, sensi- tive, and reactive. As a key part of this evolution, psycho- physiological interfaces generally, and Brain–Computer Interaction (BCI) techniques specifically have intro- duced new dimensions to the human interaction process, by the introduction of direct human-to-computer con- nections. Enhancing this symbiosis is, today, both tech- nically and ethically possible. From a VR perspective, affective computing, as one the subclasses of BCI research endeavors, could be exploited in an attempt to measure users’ emotions and affective experiences, and, by incorporating them within advanced VR systems, sup- l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 105 port endeavors to enhance participants’ sense of ‘‘immersion’’ and engagement. This article has focused on the conceptualization, design, and early validation of an affective virtual reality, based on a dynamic VR environment, capable of evoking multiple emotional experiences on the part of end users. The literature reviewed demonstrated that much atten- tion has, in the recent past, focused on affective comput- ing motivated by passive human attention to music, static image or video-based stimuli, with less apparent in- terest being shown in the pursuit of similar interests in interactive, dynamic virtual environments. Moreover, it was also discovered that the construction of any affective computing and recognition system requires an affective psychophysiological database, recorded using reliable emotional stimuli when interacting with particular types of affective media. By focusing on affect recognition in virtual realities and artificial environments, a highly con- trollable affective VR-based simulation has been designed and evaluated through two early experiments, to be further used in the construction of such a database. Indeed, the aspiration of this ongoing project is to con- struct a reliable affective psychophysiological database to generate techniques that will ultimately support the design of adaptive virtual environments, not only by measuring human emotions stimulated by virtual envi- ronments through physiological parameters, but also by adapting the game’s contents and events accordingly, with the ultimate aim of increasing the ‘‘immersion fac- tor,’’ possibly even tailored to the needs and responses of each individual user. Elements of the research described in this article are currently being considered for application in other domains, such as the role of emotions in the exploitation of VR or serious games within healthcare contexts—for example, during cognitive restoration therapies (e.g., Stone, Small, Knight, Qian, & Shingari, 2014), or in rehabilitation and distraction therapies (e.g., Small, Stone, Pilsbury, Bowden, & Bion, 2015). In particular, these healthcare projects are generating early findings, based on issues with poor sustained participation on the part of certain patients, suggesting that the methods described herein may well be significant in future techni- ques of early ‘‘affective screening’’ during recruitment. References Ahlberg, G., Enochsson, L., Gallagher, A. G., Hedman, L., Hog- man, C., McClusky, D. A., et al. (2007). Proficiency-based vir- tual reality training significantly reduces the error rate for resi- dents during their first 10 laparoscopic cholecystectomies. The American Journal of Surgery, 193(6), 797–804. Antje, H., Peter, C., Markert, L., Meer, E. V., & Voskamp, J. (2005). Emotion studies in HCI—A new approach. HCI International Conference, 1. Las Vegas, Nevada. Baveye, Y., Bettinelli, J.-N., Dellandrea, E., Chen, L., & Cha- maret, C. (2013). A large video data base for computational models of induced emotion. 2013 Humaine Association Conference on Affective Computing and Intelligent Interac- tion, ACII 2013, 13–18. Bradley, M. M., & Lang, P. J. (2006). Emotion and motivation. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (2nd ed.), pp. 581–607. New York: Cambridge University Press. Bradley, M. M., & Lang, P. J. (1994). Measuring emotion, the self-assessment manikin and the semantic differential. Jour- nal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. Bradley, M., & Lang, P. (1999). International affective digi- tized sounds (IADS): Stimuli, instruction manual and affec- tive ratings. Gainesville: The Center for Research in Psycho- physiology, University of Florida. Brown, E., & Cairns, P. (2004). A grounded investigation of game immersion. CHI ’04 Extended Abstracts on Human Factors in Computing Systems, 1297–1300. Cairns, P., Cox, A., Berthouze, N., Dhoparee, S., & Jennett, C. (2006). Quantifying the experience of immersion in games. Proceedings of the CogSci 2006 Workshop: Cognitive Science of Games and Gameplay. Difede, J., Cukor, J., Jayasinghe, N., Patt, I., Jedel, S., Spiel- man, L., et al. (2007). Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following Sep- tember 11, 2001. The Journal of Clinical Psychiatry, 68(11), 1639–1647. Ekman, P., & Friesen, W. V. (2003). Unmasking the face. Los Altos: ISHK. Frantzidis, C. A., Bratsas, C., Papadelis, C. L., Konstantinidis, E., Pappas, C., & Bamidis, P. D. (2010). Toward emotion aware computing: An integrated approach using multichan- nel neurophysiological recordings and affective visual stimuli. IEEE Transactions on Information Technology in Biomedi- cine, 14(3), 589–597. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 106 PRESENCE: VOLUME 25, NUMBER 2 Hoffman, H. G., Doctor, J. N., Patterson, D. R., Carrougher, G. J., & Furness, T. A. (2000). Virtual reality as an adjunc- tive pain control during burn wound care in adolescent patients. Pain, 18(2), 305–309. Hoffman, H. G., Sharar, S. R., Coda, B., Everett, J. J., Ciol, M., Richards, T., et al. (2004). Manipulating presence influ- ences the magnitude of virtual reality analgesia. Pain, 11(1–2), 162–168. Jack, D., Boian, R., Merians, A. S., Tremaine, M., Burdea, G. C., Adamovich, S. V., et al. (2001). Virtual reality-enhanced stroke rehabilitation. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 9(3), 308–318. Joels, M., Pu, Z., Wiegert, O., Oitzl, M. S., & Krugers, H. J. (2006). Learning under stress: How does it work? Trends in Cognitive Sciences, 10(4), 152–158. Kaganoff, E., Bordnick, P. S., & Carter, B. (2012). Feasibility of using virtual reality to assess nicotine cue reactivity during treatment. Research on Social Work Practice, 22(2), 159–165. Katsis, C. D., Katertsidis, N., Ganiatsas, G., & Fotiadis, D. I. (2008). Toward emotion recognition in car-racing drivers: A biosignal processing approach. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, 38(3), 502–512. Koelstra, S., Mu¨hl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., et al. (2012). DEAP: A database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing, 3(1), 18–31. Lang, P., Bradley, M., & Cuthbert, B. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Gainesville: University of Florida. Mahrer, N. E., & Gold, J. I. (2009). The use of virtual reality for pain control: A review. Current Pain and Headache Reports, 13(2), 100–109. Mehrabian, A. (1970). A semantic space for nonverbal behaviour. Consulting and Clinical Psychology, 35(2), 248– 257. Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foun- dations of machine learning. Cambridge, MA: MIT Press. Murugappan, M., Rizon, M., Nagarajan, R., Yaacob, S., Hazry, D., & Zunaidi, I. (2008). Time-frequency analysis of EEG sig- nals for human emotion detection. Proceedings of the Inter- national Federation of Medical & Biological Engineering (IFMBE), 21, 262–265. Nijholt, A., Plass-Oude, B. D., & Reuderink, B. (2009). Turning shortcomings into challenges: Brain–computer interfaces for games. Entertainment Computing 1, 1(2), 85–94. Pang-Ning Tan, M. S. (2005). Introduction to data mining. Reading, MA: Addison-Wesley. Parnandi, A., Son, Y., & Gutierrez-Osuna, R. (2013). A con- trol-theoretic approach to adaptive physiological games. 2013 Humaine Association Conference on Affective Comput- ing and Intelligent Interaction, 7–12. Parsons, T. D., & Rizzo, A. A. (2008). Affective outcomes of virtual reality exposure therapy for anxiety and specific pho- bias: A meta-analysis. Journal of Behavior Therapy and Exper- imental Psychiatry, 39(3), 250–261. Patrick, E., Cosgrove, D., Slavkovie, A., Rode, J. A., Verratti, T., & Chiselko, G. (2000). Using a large projection screen as an alternative to head-mounted displays for virtual envi- ronments. CHI Letters, 2(1), 478–485. Rizon, M., Murugappan, M., Nagarajan, R., & Yaacob, S. (2008). Asymmetric ratio and FCM based salient channel selection for human emotion detection using EEG. WSEAS Transactions on Signal Processing, 4(10), 596–603. Rizzo, A. A., Bowerly, T., Buckwalter, J. G., Schultheis, M., Matheis, R., Shahabi, C., et al. (2002). Virtual environments for the assessment of attention and memory processes: The virtual classroom and office. Virtual Reality, 3–12. Rizzo, A., Buckwalter, J., Forbell, E., Reist, C., Difede, J., Rothbaum, B. O., et al. (2013). Virtual reality applications to address the wounds of war. Psychiatric Annals, 43, 123–138. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. Seymour, N. E., Gallagher, A. G., Roman, S. A., O’Brien, M. K., Bansal, V. K., Andersen, D. K., et al. (2002). Virtual real- ity training improves operating room performance. Annals of Surgery, 236(4), 458–464. Small, C., Stone, R. J., Pilsbury, J., Bowden, M., & Bion, J. (2015). Virtual restorative environment therapy as an adjunct to pain control during burns dressing changes. Tri- als, 16(1), 1–7. Stone, R. J., & Hannigan, F. P. (2014). Applications of virtual environments: An overview. In K. S. Hale & K. M. Stanney (Eds.), Handbook of virtual environments: Design, implemen- tation and applications (883–957). New York: CRC Press (Taylor & Francis). Stone, R. J., Small, C. L., Knight, J. F., Qian, C., & Shingari, V. (2014). Virtual environments for healthcare restoration and rehabilitation. Virtual and Augmented Reality in Healthcare 1. Intelligent Systems Reference Library, 68, 497– 521. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Moghimi et al. 107 Takahashi, K., & Tsukaguchi, A. (2003). Remarks on emotion rec- ognition from multi-modal bio-potential signals. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, 2, 1654–1659. Wu, D., Courtney, C. G., Lance, B. J., Narayanan, S. S., Dawson, M. E., Oie, K. S., et al. (2010). Optimal arousal identification and classification for affective computing using physiological signals: Virtual reality Stroop task. IEEE Transactions on Affective Computing, 1(2), 109–118. Yazdani, A., Lee, J.-S., & Ebrahimi, T. (2009). Implicit emo- tional tagging of multimedia using EEG signals and brain computer interface. Proceedings of the First SIGMM Workshop on Social Media, 81–88. Zyda, M. (2005). From visual simulation to virtual reality to games. Computer, 38(9), 25–32. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . / e d u p v a r / a r t i c e - p d l f / / / / / 2 5 2 8 1 1 8 3 6 5 7 9 p r e s _ a _ 0 0 2 4 9 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3

PDF Herunterladen