Enacting Sonic-Cyborg Performance
through the Hybrid Body in
Teka-Mori and Why Should
Our Bodies End at the Skin?
A u R i E H S u a n d S T E v E n k E m P E R
T
C
UN
R.
T
S
B
UN
In “A Cyborg Manifesto,” Donna Haraway explores implications of
the increasing hybridization of humans and machines. While society
has long been concerned with the encroachment of technology
onto human activity, Haraway challenges this concern, suggérant
instead a kinship between organism and machine, a hybrid body.
A sonic-cyborg performance realizes this understanding of the
human-machine hybrid through movement and sound, incorporating
a “kinesonic” approach to composition and an exploration of
“mechatronic” expression. In this article, the authors describe their
approach to enacting sonic-cyborg performance by outlining the
creative framework and associated technologies involved in two
collaborative pieces that explore questions of fluidity between
organism and machine: Teka-Mori and Why Should Our Bodies
End at the Skin?
In her essay “A Cyborg Manifesto” [1], Donna Haraway ex-
plores implications of the increasing hybridization of hu-
mans and machines. Haraway summarizes: “Late twentieth-
century machines have made thoroughly ambiguous the
difference between natural and artificial, mind and body,
self-developing and externally designed, and many other
distinctions that used to apply to organisms and machines”
[2]. In dispelling binaries and advocating for holistic perspec-
tives, Haraway’s writings have far-reaching philosophical and
sociopolitical implications, providing critique that redefines
notions of gender, identity and feminist approaches. While
society has long been concerned with the ever-growing en-
croachment of technology onto human activity, most re-
cently the development of artificial intelligence and robotics
[3], Haraway proposes instead a kinship between organism
and machine. In a broad sense, this kinship suggests hybrid
bodies that transcend mechanical and organic boundaries,
Aurie Hsu (artist, educator, researcher), Oberlin Conservatory of Music, Technologie
in Music and Related Arts (TIMARA) Department, 77 West College Street,
Oberlin, OH 44074, U.S.A. E-mail: ahsu@oberlin.edu. Web: www.auriehsu.com.
ORCID: 0000-0003-3852-446X.
Steven Kemper (artist, educator, researcher), Mason Gross School of the Arts, Music
Technology Department, Rutgers, State University of New Jersey, 81 George Street,
New Brunswick, New Jersey 08901, U.S.A. E-mail: skemper@mgsa.rutgers.edu.
Web: www.stevenkemper.com. ORCID: 0000-0003-4146-3792.
See www.mitpressjournals.org/toc/lmj/29 for supplemental files associated with
ce problème.
raising questions about embodiment in our contemporary
technoculture where the lines between organism and ma-
chine are fluid. In a book about the “Chthulucene,” Haraway
invokes the term sympoiesis, meaning “making-with” [4],
originally coined by M. Beth Dempster as “collectively-
producing systems that do not have self-defined spatial or
temporal boundaries. Information and control are distrib-
uted among components. The systems are evolutionary and
have the potential for surprising change” [5]. In contrast
to homeostatic, autonomous and autopoietic systems, “the
more ubiquitous symbiogenesis seems to be in living beings’
dynamic organizing processes, the more looped, braided,
outreaching, involuted, and sympoietic is terran worlding”
[6]. Ainsi, the hybridization of the body and machine reflects
a symbiotic merging of systems.
A sonic-cyborg performance realizes this understand-
ing of the human-machine hybrid through movement and
sound. In this article, we describe our creative framework
and the associated technologies involved in enacting sonic-
cyborg performance through two recent collaborative pieces
that explore these questions of fluidity between organism and
machine: Teka-Mori (2013), for dancer, RAKS system and
computer-generated sound; and Why Should Our Bodies
End at the Skin? (2018), for dancer, RAKS system, robotic
percussion, sound exciters and live sound processing. These
pieces feature the Remote electroAcoustic Kinesthetic Sens-
ing (RAKS) système, a wearable wireless sensor interface that
translates a dancer’s movement into sonic output, engaging
an embodied creative practice that foregrounds tactile, ki-
netic and kinesthetic sensory experience.
BACkgRound
Alongside research in the areas of gesture, human-computer/
robotic interaction and new interfaces for musical expres-
sion, artists have developed interactive performance sys-
tems that link movement and sound production. Sensor
technologies and robotics enable humans to interact with
computer-controlled mechanical systems. In electronic mu-
sic performance, sensors and robotics output movement data
©2019 ISAST
est ce que je:10.1162/LM J_a_01069
LEONARDO MUSIC JOURNAL, Vol. 29, pp. 83–87, 2019 83
Téléchargé depuis http://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 Septembre 2023
that can be mapped to musical parameters, dans-
abling a direct link between human action and
sound production. While these types of inter-
active systems have been created for a variety of
purposes, several projects actively embrace the
blurring of human and machine, including To-
mie Hahn and Curtis Bahn’s Pikapika [7], Lae-
titia Sonami’s lady’s glove [8], Onyx Ashanti’s
sonocyb1 exomesh bodyware system [9], Scott
Barton’s Cyther [10], LP Demers and Bill Vorn’s
Inferno [11], Åsa Unander-Scharin and Carl
Unander-Scharin’s Robocygne [12] and Marco
Donnarumma’s Xth Sense system [13]. Notre
pieces are positioned among these works with
a focus on exploring the hybrid body through
kinesonic composition and the interaction be-
tween mechatronic movement and sound.
figue. 1. Remote electroAcoustic Kinesthetic Sensing (RAKS) système, 2010. (© Steven Kemper)
HyBRid Body
The Remote electroAcoustic Kinesthetic Sensing (RAKS) sys-
tem is an Arduino-based wearable wireless sensor interface
that we designed (figue. 1). The RAKS system enables a direct
link between a dancer’s movement and computer-generated
sound production and processing. By linking these elements,
we explore both the human and mechanical aspects of the
hybrid body in performance. The RAKS system consists of
a corset and a lightweight belt with sensors, a LilyPad Ar-
duino and an XBee radio sewn into the belt. Sensors capture
the dancer’s movement and transmit the data wirelessly to a
computer running Max software over a point-to-point net-
travail. The system is modular, and different sensors can be
incorporated for different performances. The sensors used in
Teka-Mori and Why Should Our Bodies End at the Skin? dans-
clude a flex sensor in the corset and an accelerometer on the
belt. The system also includes two programmable LED rings
sewn into the corset that take data from the accelerometer
in order to visualize the dancer’s movements.
The development of the RAKS system is informed by
gesture studies [14] and specifically our concepts of “cho-
reographed sound” and “kinesonic composition” [15]. UN
TABlE 1. Movement vocabulary and RaKS system design.
kinesonic approach foregrounds embodied activity by in-
tegrating movement, kinetic and kinesthetic experience
and sonic elements in the compositional process. In pieces
using sensor-based interfaces, the designer directly defines
and maps the relationship between movement and sound,
employing physical gesture as a composable parameter. Par
focusing on the mechanics of movement and its relationship
to sound, choreographing sound engages an embodied per-
spective that involves tactile, kinetic and kinesthetic sensory
experience.
Movement vocabulary from contemporary belly dance
influences the design and functionality of the RAKS system,
including the hardware, types of sensors and costume archi-
tecture. The types of movements employed in belly dance
can be metaphorically linked to the fundamental concepts
of electronic music. Par exemple, continuous motion, tel
as torso body waves and chest circles, relates to analog oscil-
lators, specifically the sine wave. Isolated movements, tel
as hip and shoulder locks, are binary, evoking the idea of
mechanical switches. To capture these types of movements,
we placed the flex sensor near the chest and torso and the
accelerometer on the hips. Tableau 1 summarizes the elements
of the movement and corresponding design features on the
movement vocabulary
Body movement
Qualitative description
RAkS System design Feature
wave
rotation
torso
continu, down-to-up/up-to-down
flex sensor
chest, hip
circular motion
figure-eight (vertical/horizontal)
torso, chest, hip
continu, isolated
flex sensor
flex sensor
accelerometer
isolations
chest, hip
quick, slow, sharp, lisse
accelerometer
layering
upper/lower body
smooth/sharp, isolated
flex sensor
accelerometer
(each sends separate data)
84 Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin?
Téléchargé depuis http://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 Septembre 2023
RAKS system. This approach to correlating movement and
sound engages an embodied, kinesonic perspective [16] comme
movement serves to trigger, generate and alter sonic material.
Integrating movement with sound production, the “body be-
comes the instrument” [17] and the dancer inhabits a “sonic
body” [18].
mECHATRoniC ExPRESSion
Our performance strategies are influenced by the concept
of “mechatronic expression,” defined by Steven Kemper
and Scott Barton, in relation to music performed by ro-
botic instruments [19]. This idea challenges the assumption
that musical expressivity results from the capacity for sonic
nuance and argues that robotic instruments are capable of
their own unique vocabularies of expressive gestures that are
specific to mechanical instruments, as distinct from human
performers. These include the capability for hypervirtuosic
speed, complex rhythms, humanly impossible articulations
and algorithmic control, as well as their own idiosyncrasies
and limitations that vary by instrument, such as microvari-
ations in timing caused by physical forces like friction and
gravity [20]. Mimicking these types of gestures through
movement and sound, as well as performing with robotic
instruments, represents a central element of sonic-cyborg
performance.
TEka-MOri
Teka-Mori, for dancer, RAKS system and computer-gener-
ated sound, features an interactive, bidirectional relationship
between movement and sound. Teka refers to the vocaliza-
tion of two different drum strokes (“tek” and “ka”) on a
doumbek. Mori, adapted from the Latin phrase memento
mori, evokes the idea of lifelessness and decay. Teka-Mori
conveys a dystopian, “broken-machine” aesthetic through
noisy, distorted sonic materials.
Teka-Mori enacts a hybrid body through the direct con-
nection between movement and timbral control, where the
performer inhabits and controls this sonic machine. Ce
relationship is intensified through the use of distorted, elec-
tronic sounds, including a bowed bar model [21], pulse-
width modulation, wave-shaping, filtering and an oscillator
bank. En plus, sonically degraded acoustic recordings
of the different drum strokes played back on a grid produce
a mechanical reinterpretation of a deconstructed drum pat-
tern. The piece begins with the dancer controlling a physical
model of a bowed bar. The dancer’s torso movements, inclure-
ing waves and a side-to-side figure-eight motion, increase
and decrease the bow pressure. Hip circles and torso waves
control the amplitude of the first 12 partials of a Chebyshev
wave-shaping module. Lateral hip circles modulate midrange
frequencies, and reverse torso waves modulate high frequen-
cies. To shift between pitch and noise, lateral figure-eight
movements in the hips, punctuated by pauses and slow side
bends in the torso, change the center frequency of a low pass
filter. The combination of these movements simultaneously
affects pitch, rhythm and timbre. In the second part of the
piece, the choreography consists of a combination of slow
and rapid turn sequences punctuated by pauses in the move-
ment. When the dancer is turning, an oscillator bank pro-
duces a cascade of sine waves up to a high partial determined
by the level of movement. When the dancer pauses, ces
upper partials disappear, leaving only the lowest frequency
audible. The output signal from this module is delayed, pro-
ducing a sonic echo of the dancer’s movement.
Why ShOuld Our BOdiES End aT ThE Skin?
Why Should Our Bodies End at the Skin?, for dancer, RAKS
système, the Configurable Automatic Drumming Instrument
(CADI), sound exciters and live sound processing, explores
hybridization of organism and machine through the idea
of embodied mechatronic expression. The title references
Haraway’s suggestion of a kinship between organism and
machine [22]. This piece realizes the capability of the hybrid
body in performance, sonically connecting mechanized hu-
man movement and humanized robotic action. Robots are
often designed to complete human tasks; their motions re-
flect a mechanical abstraction of human movement [23]. Le
RAKS system combined with CADI connects the dancer’s
actions with machine tasks [24].
CADI is a solenoid-driven, robotic percussion battery
(figue. 2). We designed the 3D-printed striking arms to hold
a variety of beaters, and they are mounted on microphone
stands, making it easy to position the beaters to hit a variety
of different instruments. CADI is controlled via MIDI, et
velocity messages control the solenoids’ on-times, affecting
striking force and allowing for dynamic control. Steven Kem-
par, Troy Rogers and Scott Barton of Expressive Machines
figue. 2. Configurable Automatic Drumming Instrument (CADI).
(© Steven Kemper)
Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin? 85
Téléchargé depuis http://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 Septembre 2023
phrases of hip isolations and locks.
After a few phrases, CADI “awak-
ens” with rhythmic patterns that
result from the preceding acceler-
ometer movement data. The rhythm
mimics the movement but also adds
extra reverberations to which the
dancer reacts through movement.
The next section weaves in and out
of several interactive modes be-
tween the movement and the sonic
texture. D'abord, the dancer moves to
automated timbral changes in the
texture; then the dancer’s move-
ment itself controls timbral shifts.
The flex sensor data controls the
amplitude of the 12 upper partials
of Chebyshev wave-shaping played
through the sound exciters. The re-
sult is an ebb and flow in the density
and amplitude of the texture, refer-
encing the continuous movements
found in the torso, hips and chest.
The middle section of the piece features two interactive
drum solos, one where the dancer leads the robotic percus-
sion and one where the robotic percussion leads the dancer.
Within the context of a 13-beat rhythm, the dancer slowly
rotates and faces one of the toms, making “eye contact” with
the instrument. À ce point, control over the rate of the arm
hitting the drum is transferred to the dancer, who can control
the speed of rolls through a stomach flutter movement. After
this solo, the performer returns to make eye contact with the
floor tom and responds to an algorithmically generated solo
on that instrument.
In the final section, the dancer’s actions intertwine with the
machine, which begins to “malfunction.” The dancer plays
the drums along with CADI, which performs a series of in-
creasingly irregular and accelerating loops. The piece ends
with the dancer resting drumsticks on the floor toms. Le
sound exciters mounted to these drums resonate, causing
the drumsticks to vibrate on the drumheads. This creates a
physical and sonic connection between the dancer’s body
and the electromechanically actuated acoustic instruments.
ConCluSion
The development of sensor-mediated performance has en-
abled us to investigate Haraway’s conception of the hybrid
body and develop a practice of sonic-cyborg performance.
By exploring the performative possibilities of this type of
extended system, we have attempted to recontextualize nega-
tive implications of a hybrid body. In this sense, we actualize
Haraway’s notion of a kinship between organism and ma-
chine, exploring a world where the cyborg represents not
an inevitable march toward a technological singularity but
rather an evolving, creative body.
figue. 3. Why Should Our Bodies End at the Skin? in performance, 17 Février 2018.
(Photo courtesy of the Ammerman Center for Arts and Technology.)
Musical Instruments (EMMI) originally designed CADI,
with the iteration used in Why Should Our Bodies End at the
Skin? designed by Kemper at Rutgers University.
The performance of Why Should Our Bodies End at the
Skin? features a dancer equipped with the RAKS system,
CADI, sound exciters actuating both drums and shaken per-
cussion, and electroacoustic textures created from processed
mechanical sounds (figue. 3). To blur the boundaries between
organism and machine, we have the percussion instruments
encircle the dancer, serving as a visual and sonic extension
of the body. The choreography reflects the mechanical nature
of robotic movement, using isolations and body locks, alors que
CADI produces a visual and sonic echo of this movement
through rhythmic and sustained textures. The RAKS system
translates the dancer’s movement into sonic control in a vari-
ety of ways, including triggering CADI’s attacks and varying
the dynamics, panning sound events within the semicircular
field of the setup and processing the electroacoustic texture. UN
feedback loop emerges between the dancer’s movements and
CADI’s mechanical actions, calling into question whether the
human is controlling the machine or the machine is control-
ling the human. These interactions are sympoietic, creating
a symbiotic merging of the different systems.
The piece consists of three overarching motivic ideas. Le
beginning develops through an accumulation of textural
layers. Contrasting rhythmic sections reference the “drum
solo,” a Middle Eastern musical form for solo drummer or
drummer with dancer consisting of a fast-tempo virtuosic
improvisation where the lines between leader and follower
are indistinguishable. In the final section, the dancer’s actions
combine with CADI, which itself begins to break down.
The piece begins with a solo dance encompassing short
86 Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin?
Téléchargé depuis http://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 Septembre 2023
References and Notes
1 Donna Haraway, “A Cyborg Manifesto: Science, Technologie, et
Socialist-Feminism in the Late Twentieth Century,” in Simians, Cy-
borgs, and Women: The Reinvention of Nature (New York: Routledge,
1991).
2 Haraway [1] pp. 293–294.
3 Aaron Smith and Janna Anderson, “AI, Robotics, and the Future
of Jobs” (6 Août 2014): www.pewinternet.org/2014/08/06/future-of
-jobs (accessed 6 Novembre 2018).
16 Aurie Hsu and Steven Kemper, “Kinesonic Approaches to Mapping
Movement and Music with the Remote electroAcoustic Kinesthetic
Sensing (RAKS) System,” Proceedings of the 2nd International Work-
shop on Movement and Computing (MOCO ’15) (New York: ACM,
2015) pp. 45–47. EST CE QUE JE: www.dx.doi.org/10.1145/2790994.2791020.
17 Godfried-Willem Raes, “ ‘Namuda Studies’: Doppler Radar-Based
Gesture Recognition for the Control of a Music Robot Orchestra”
(2012): www.logosfoundation.org/ii/Namuda_JIM_paper.doc (ac-
cessed 5 Novembre 2018).
4 Donna Haraway, Staying with the Trouble: Making Kin in the Chthu-
lucene (Durham, Caroline du Nord: Duke Univ. Presse, 2016) p. 58.
18 Hahn and Bahn [7] pp. 229–238.
5 M.. Beth Dempster quoted in Haraway [4] p. 61.
6 Haraway [4] p. 61.
7 Tomie Hahn and Curtis Bahn, “Pikapika—The Collaborative Com-
position of an Interactive Sonic Character,” Organised Sound 7, Non.
3, 229–238 (2002).
19 Steven Kemper and Scott Barton, “Mechatronic Expression: Recon-
sidering Expressivity in Music for Robotic Instruments,” Proceedings
of the 18th International Conference on New Interfaces for Musical
Expression (NIME) (Blacksburg, VA: Virginia Tech University, 2018)
pp. 84–87.
20 Ajay Kapur et al., “Collaborative Composition for Musical Robots,»
Journal of Science and Technology of the Arts 1, Non. 1 (2009) p. 49.
8 Laetitia Sonami, lady’s glove: www.sonami.net/ladys-glove (accessed
12 Mars 2019).
21 Dan Trueman and R. Luke DuBois, PeRColate: www.music.columbia
.edu/percolate (accessed 1 Janvier 2019).
9 Onyx Ashanti, Wreck now—sonocybin: www.youtube.com/watch
?v=gaFi_jUHS4s (accessed 12 Mars 2019).
22 Haraway [1] p. 295.
10 Scott Barton, Ethan Prihar and Paulo Carvalho, “Cyther: A Human-
Playable, Self-Tuning Robotic Zither,” Proceedings of the 2017 Nouveau
Interfaces for Musical Expression Conference (Copenhagen, 2017) pp.
319–324.
11 LP Demers and Bill Vorn, Inferno: www.billvorn.concordia.ca
/menuall.html (accessed 12 Mars 2019).
12 Åsa Unander-Scharin and Carl Unander-Scharin, “Robocygne:
Dancing Life into an Animal-Human-Machine,” Leonardo 49, Non.
3, 212–219 (2016).
13 Marco Donnarumma, “Music for Flesh II: Informing Interactive
Music Performance with the Viscerality of the Body System,” Pro-
ceedings of the 2012 Conference on New Interfaces for Musical Expres-
sion (Ann-Arbor, MI: Université du Michigan, 2012).
14 Claude Cadoz and Marcelo M. Wanderley, Gesture—Music (Paris:
IRCAM, 2000).
15 Aurie Hsu and Steven Kemper, “Kinesonic Composition as Cho-
reographed Sound: Composing Gesture in Sensor-Based Music,»
Actes du 2015 International Computer Music Conference
(Denton, TX: University of North Texas, 2015) pp. 412–415.
23 Cynthia Breazeal and Brian Scassellati, “Robots That Imitate Hu-
mans,” Trends in Cognitive Sciences 6, Non. 11, 481–487 (2002): www
.est ce que je.org/10.1016/S1364-6613(02)02016-8.
24 Oscar D. Lara and Miguel A. Labrador, “A Survey on Human Ac-
tivity Recognition Using Wearable Sensors,” IEEE Communications
Surveys Tutorials 15, Non. 3, 1192–1209 (2013): www.doi.org/10.1109
/SURV.2012.110112.00192.
Manuscript received 2 Janvier 2019.
Aurie Hsu is a composer and performer. She is currently
assistant professor of computer music and digital arts in the
Technology in Music and Related Arts (TIMARA) department
at the Oberlin Conservatory.
steven Kemper is a composer and music technologist.
He is currently associate professor of music technology and
composition at the Mason Gross School of the Arts at Rutgers
University.
Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin? 87
Téléchargé depuis http://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 Septembre 2023
