Enacting Sonic-Cyborg Performance
through the Hybrid Body in
Teka-Mori and Why Should
Our Bodies End at the Skin?
A u R i E H S u a n d S T E v E n k E m P E R
时间
C
A
右
时间
S
乙
A
In “A Cyborg Manifesto,” Donna Haraway explores implications of
the increasing hybridization of humans and machines. While society
has long been concerned with the encroachment of technology
onto human activity, Haraway challenges this concern, suggesting
instead a kinship between organism and machine, a hybrid body.
A sonic-cyborg performance realizes this understanding of the
human-machine hybrid through movement and sound, incorporating
a “kinesonic” approach to composition and an exploration of
“mechatronic” expression. 在本文中, the authors describe their
approach to enacting sonic-cyborg performance by outlining the
creative framework and associated technologies involved in two
collaborative pieces that explore questions of fluidity between
organism and machine: Teka-Mori and Why Should Our Bodies
End at the Skin?
In her essay “A Cyborg Manifesto” [1], Donna Haraway ex-
plores implications of the increasing hybridization of hu-
mans and machines. Haraway summarizes: “Late twentieth-
century machines have made thoroughly ambiguous the
difference between natural and artificial, mind and body,
self-developing and externally designed, and many other
distinctions that used to apply to organisms and machines”
[2]. In dispelling binaries and advocating for holistic perspec-
特维斯, Haraway’s writings have far-reaching philosophical and
sociopolitical implications, providing critique that redefines
notions of gender, identity and feminist approaches. 尽管
society has long been concerned with the ever-growing en-
croachment of technology onto human activity, most re-
cently the development of artificial intelligence and robotics
[3], Haraway proposes instead a kinship between organism
and machine. In a broad sense, this kinship suggests hybrid
bodies that transcend mechanical and organic boundaries,
Aurie Hsu (artist, 教育家, researcher), Oberlin Conservatory of Music, 技术
in Music and Related Arts (TIMARA) Department, 77 West College Street,
Oberlin, 哦 44074, 美国. 电子邮件: ahsu@oberlin.edu. Web: www.auriehsu.com.
ORCID: 0000-0003-3852-446X.
Steven Kemper (artist, 教育家, researcher), Mason Gross School of the Arts, 音乐
Technology Department, Rutgers, State University of New Jersey, 81 George Street,
New Brunswick, 新泽西州 08901, 美国. 电子邮件: skemper@mgsa.rutgers.edu.
Web: www.stevenkemper.com. ORCID: 0000-0003-4146-3792.
See www.mitpressjournals.org/toc/lmj/29 for supplemental files associated with
this issue.
raising questions about embodiment in our contemporary
technoculture where the lines between organism and ma-
chine are fluid. In a book about the “Chthulucene,” Haraway
invokes the term sympoiesis, meaning “making-with” [4],
originally coined by M. Beth Dempster as “collectively-
producing systems that do not have self-defined spatial or
temporal boundaries. Information and control are distrib-
uted among components. The systems are evolutionary and
have the potential for surprising change” [5]. 相比之下
to homeostatic, autonomous and autopoietic systems, “这
more ubiquitous symbiogenesis seems to be in living beings’
dynamic organizing processes, the more looped, braided,
outreaching, involuted, and sympoietic is terran worlding”
[6]. 因此, the hybridization of the body and machine reflects
a symbiotic merging of systems.
A sonic-cyborg performance realizes this understand-
ing of the human-machine hybrid through movement and
声音. 在本文中, we describe our creative framework
and the associated technologies involved in enacting sonic-
cyborg performance through two recent collaborative pieces
that explore these questions of fluidity between organism and
机器: Teka-Mori (2013), for dancer, RAKS system and
computer-generated sound; and Why Should Our Bodies
End at the Skin? (2018), for dancer, RAKS system, robotic
percussion, sound exciters and live sound processing. 这些
pieces feature the Remote electroAcoustic Kinesthetic Sens-
英 (RAKS) 系统, a wearable wireless sensor interface that
translates a dancer’s movement into sonic output, 吸引人的
an embodied creative practice that foregrounds tactile, ki-
netic and kinesthetic sensory experience.
BACkgRound
Alongside research in the areas of gesture, human-computer/
robotic interaction and new interfaces for musical expres-
锡安, artists have developed interactive performance sys-
tems that link movement and sound production. Sensor
technologies and robotics enable humans to interact with
computer-controlled mechanical systems. In electronic mu-
sic performance, sensors and robotics output movement data
©2019 ISAST
土井:10.1162/LM J_a_01069
LEONARDO MUSIC JOURNAL, 卷. 29, PP. 83–87, 2019 83
从http下载的://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 九月 2023
that can be mapped to musical parameters, 在-
abling a direct link between human action and
sound production. While these types of inter-
active systems have been created for a variety of
目的, several projects actively embrace the
blurring of human and machine, including To-
mie Hahn and Curtis Bahn’s Pikapika [7], Lae-
titia Sonami’s lady’s glove [8], Onyx Ashanti’s
sonocyb1 exomesh bodyware system [9], 斯科特
Barton’s Cyther [10], LP Demers and Bill Vorn’s
Inferno [11], Åsa Unander-Scharin and Carl
Unander-Scharin’s Robocygne [12] and Marco
Donnarumma’s Xth Sense system [13]. 我们的
pieces are positioned among these works with
a focus on exploring the hybrid body through
kinesonic composition and the interaction be-
tween mechatronic movement and sound.
如图. 1. Remote electroAcoustic Kinesthetic Sensing (RAKS) 系统, 2010. (© Steven Kemper)
HyBRid Body
The Remote electroAcoustic Kinesthetic Sensing (RAKS) 系统-
tem is an Arduino-based wearable wireless sensor interface
that we designed (如图. 1). The RAKS system enables a direct
link between a dancer’s movement and computer-generated
sound production and processing. By linking these elements,
we explore both the human and mechanical aspects of the
hybrid body in performance. The RAKS system consists of
a corset and a lightweight belt with sensors, a LilyPad Ar-
duino and an XBee radio sewn into the belt. Sensors capture
the dancer’s movement and transmit the data wirelessly to a
computer running Max software over a point-to-point net-
工作. The system is modular, and different sensors can be
incorporated for different performances. The sensors used in
Teka-Mori and Why Should Our Bodies End at the Skin? 在-
clude a flex sensor in the corset and an accelerometer on the
belt. The system also includes two programmable LED rings
sewn into the corset that take data from the accelerometer
in order to visualize the dancer’s movements.
The development of the RAKS system is informed by
gesture studies [14] and specifically our concepts of “cho-
reographed sound” and “kinesonic composition” [15]. A
TABlE 1. Movement vocabulary and RaKS system design.
kinesonic approach foregrounds embodied activity by in-
tegrating movement, kinetic and kinesthetic experience
and sonic elements in the compositional process. In pieces
using sensor-based interfaces, the designer directly defines
and maps the relationship between movement and sound,
employing physical gesture as a composable parameter. 经过
focusing on the mechanics of movement and its relationship
to sound, choreographing sound engages an embodied per-
spective that involves tactile, kinetic and kinesthetic sensory
经验.
Movement vocabulary from contemporary belly dance
influences the design and functionality of the RAKS system,
including the hardware, types of sensors and costume archi-
结构. The types of movements employed in belly dance
can be metaphorically linked to the fundamental concepts
of electronic music. 例如, continuous motion, 这样的
as torso body waves and chest circles, relates to analog oscil-
lators, specifically the sine wave. Isolated movements, 这样的
as hip and shoulder locks, are binary, evoking the idea of
mechanical switches. To capture these types of movements,
we placed the flex sensor near the chest and torso and the
accelerometer on the hips. 桌子 1 summarizes the elements
of the movement and corresponding design features on the
movement vocabulary
Body movement
Qualitative description
RAkS System design Feature
wave
rotation
torso
continuous, down-to-up/up-to-down
flex sensor
胸部, hip
circular motion
figure-eight (vertical/horizontal)
torso, 胸部, hip
continuous, 孤立
flex sensor
flex sensor
accelerometer
isolations
胸部, hip
quick, slow, sharp, smooth
accelerometer
layering
upper/lower body
smooth/sharp, 孤立
flex sensor
accelerometer
(each sends separate data)
84 Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin?
从http下载的://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 九月 2023
RAKS system. This approach to correlating movement and
sound engages an embodied, kinesonic perspective [16] 作为
movement serves to trigger, generate and alter sonic material.
Integrating movement with sound production, the “body be-
comes the instrument” [17] and the dancer inhabits a “sonic
body” [18].
mECHATRoniC ExPRESSion
Our performance strategies are influenced by the concept
of “mechatronic expression,” defined by Steven Kemper
and Scott Barton, in relation to music performed by ro-
botic instruments [19]. This idea challenges the assumption
that musical expressivity results from the capacity for sonic
nuance and argues that robotic instruments are capable of
their own unique vocabularies of expressive gestures that are
specific to mechanical instruments, as distinct from human
performers. These include the capability for hypervirtuosic
速度, complex rhythms, humanly impossible articulations
and algorithmic control, as well as their own idiosyncrasies
and limitations that vary by instrument, such as microvari-
ations in timing caused by physical forces like friction and
重力 [20]. Mimicking these types of gestures through
movement and sound, as well as performing with robotic
仪器, represents a central element of sonic-cyborg
表现.
TEka-MOri
Teka-Mori, for dancer, RAKS system and computer-gener-
ated sound, features an interactive, bidirectional relationship
between movement and sound. Teka refers to the vocaliza-
tion of two different drum strokes (“tek” and “ka”) on a
doumbek. 森, adapted from the Latin phrase memento
mori, evokes the idea of lifelessness and decay. Teka-Mori
conveys a dystopian, “broken-machine” aesthetic through
嘈杂, distorted sonic materials.
Teka-Mori enacts a hybrid body through the direct con-
nection between movement and timbral control, 哪里的
performer inhabits and controls this sonic machine. 这
relationship is intensified through the use of distorted, elec-
tronic sounds, including a bowed bar model [21], pulse-
width modulation, wave-shaping, filtering and an oscillator
bank. 此外, sonically degraded acoustic recordings
of the different drum strokes played back on a grid produce
a mechanical reinterpretation of a deconstructed drum pat-
tern. The piece begins with the dancer controlling a physical
model of a bowed bar. The dancer’s torso movements, 包括-
ing waves and a side-to-side figure-eight motion, 增加
and decrease the bow pressure. Hip circles and torso waves
control the amplitude of the first 12 partials of a Chebyshev
wave-shaping module. Lateral hip circles modulate midrange
频率, and reverse torso waves modulate high frequen-
化学系. To shift between pitch and noise, lateral figure-eight
movements in the hips, punctuated by pauses and slow side
bends in the torso, change the center frequency of a low pass
filter. The combination of these movements simultaneously
affects pitch, rhythm and timbre. In the second part of the
片, the choreography consists of a combination of slow
and rapid turn sequences punctuated by pauses in the move-
蒙特. When the dancer is turning, an oscillator bank pro-
duces a cascade of sine waves up to a high partial determined
by the level of movement. When the dancer pauses, 这些
upper partials disappear, leaving only the lowest frequency
audible. The output signal from this module is delayed, 亲-
ducing a sonic echo of the dancer’s movement.
Why ShOuld Our BOdiES End aT ThE Skin?
Why Should Our Bodies End at the Skin?, for dancer, RAKS
系统, the Configurable Automatic Drumming Instrument
(CADI), sound exciters and live sound processing, explores
hybridization of organism and machine through the idea
of embodied mechatronic expression. The title references
Haraway’s suggestion of a kinship between organism and
机器 [22]. This piece realizes the capability of the hybrid
body in performance, sonically connecting mechanized hu-
man movement and humanized robotic action. Robots are
often designed to complete human tasks; their motions re-
flect a mechanical abstraction of human movement [23]. 这
RAKS system combined with CADI connects the dancer’s
actions with machine tasks [24].
CADI is a solenoid-driven, robotic percussion battery
(如图. 2). We designed the 3D-printed striking arms to hold
a variety of beaters, and they are mounted on microphone
stands, making it easy to position the beaters to hit a variety
of different instruments. CADI is controlled via MIDI, 和
velocity messages control the solenoids’ on-times, affecting
striking force and allowing for dynamic control. Steven Kem-
每, Troy Rogers and Scott Barton of Expressive Machines
如图. 2. Configurable Automatic Drumming Instrument (CADI).
(© Steven Kemper)
Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin? 85
从http下载的://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 九月 2023
phrases of hip isolations and locks.
After a few phrases, CADI “awak-
ens” with rhythmic patterns that
result from the preceding acceler-
ometer movement data. The rhythm
mimics the movement but also adds
extra reverberations to which the
dancer reacts through movement.
The next section weaves in and out
of several interactive modes be-
tween the movement and the sonic
texture. 第一的, the dancer moves to
automated timbral changes in the
texture; then the dancer’s move-
ment itself controls timbral shifts.
The flex sensor data controls the
amplitude of the 12 upper partials
of Chebyshev wave-shaping played
through the sound exciters. 那里-
sult is an ebb and flow in the density
and amplitude of the texture, 参考-
encing the continuous movements
found in the torso, hips and chest.
The middle section of the piece features two interactive
drum solos, one where the dancer leads the robotic percus-
sion and one where the robotic percussion leads the dancer.
Within the context of a 13-beat rhythm, the dancer slowly
rotates and faces one of the toms, making “eye contact” with
the instrument. At this point, control over the rate of the arm
hitting the drum is transferred to the dancer, who can control
the speed of rolls through a stomach flutter movement. 后
this solo, the performer returns to make eye contact with the
floor tom and responds to an algorithmically generated solo
on that instrument.
In the final section, the dancer’s actions intertwine with the
机器, which begins to “malfunction.” The dancer plays
the drums along with CADI, which performs a series of in-
creasingly irregular and accelerating loops. The piece ends
with the dancer resting drumsticks on the floor toms. 这
sound exciters mounted to these drums resonate, causing
the drumsticks to vibrate on the drumheads. 这创建了一个
physical and sonic connection between the dancer’s body
and the electromechanically actuated acoustic instruments.
ConCluSion
The development of sensor-mediated performance has en-
abled us to investigate Haraway’s conception of the hybrid
body and develop a practice of sonic-cyborg performance.
By exploring the performative possibilities of this type of
extended system, we have attempted to recontextualize nega-
tive implications of a hybrid body. 在这个意义上, we actualize
Haraway’s notion of a kinship between organism and ma-
中国, exploring a world where the cyborg represents not
an inevitable march toward a technological singularity but
rather an evolving, creative body.
如图. 3. Why Should Our Bodies End at the Skin? in performance, 17 二月 2018.
(Photo courtesy of the Ammerman Center for Arts and Technology.)
Musical Instruments (EMMI) originally designed CADI,
with the iteration used in Why Should Our Bodies End at the
Skin? designed by Kemper at Rutgers University.
The performance of Why Should Our Bodies End at the
Skin? features a dancer equipped with the RAKS system,
CADI, sound exciters actuating both drums and shaken per-
坐垫, and electroacoustic textures created from processed
mechanical sounds (如图. 3). To blur the boundaries between
organism and machine, we have the percussion instruments
encircle the dancer, serving as a visual and sonic extension
身体的. The choreography reflects the mechanical nature
of robotic movement, using isolations and body locks, 尽管
CADI produces a visual and sonic echo of this movement
through rhythmic and sustained textures. The RAKS system
translates the dancer’s movement into sonic control in a vari-
ety of ways, including triggering CADI’s attacks and varying
the dynamics, panning sound events within the semicircular
field of the setup and processing the electroacoustic texture. A
feedback loop emerges between the dancer’s movements and
CADI’s mechanical actions, calling into question whether the
human is controlling the machine or the machine is control-
ling the human. These interactions are sympoietic, 创造
a symbiotic merging of the different systems.
The piece consists of three overarching motivic ideas. 这
beginning develops through an accumulation of textural
layers. Contrasting rhythmic sections reference the “drum
solo,” a Middle Eastern musical form for solo drummer or
drummer with dancer consisting of a fast-tempo virtuosic
improvisation where the lines between leader and follower
are indistinguishable. In the final section, the dancer’s actions
combine with CADI, which itself begins to break down.
The piece begins with a solo dance encompassing short
86 Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin?
从http下载的://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 九月 2023
参考文献和注释
1 Donna Haraway, “A Cyborg Manifesto: 科学, 技术, 和
Socialist-Feminism in the Late Twentieth Century,” in Simians, Cy-
borgs, and Women: The Reinvention of Nature (纽约: 劳特利奇,
1991).
2 Haraway [1] PP. 293–294.
3 Aaron Smith and Janna Anderson, “AI, Robotics, and the Future
of Jobs” (6 八月 2014): www.pewinternet.org/2014/08/06/future-of
-工作 (访问过 6 十一月 2018).
16 Aurie Hsu and Steven Kemper, “Kinesonic Approaches to Mapping
Movement and Music with the Remote electroAcoustic Kinesthetic
Sensing (RAKS) 系统,” Proceedings of the 2nd International Work-
shop on Movement and Computing (MOCO ’15) (纽约: ACM,
2015) PP. 45–47. DOI: www.dx.doi.org/10.1145/2790994.2791020.
17 Godfried-Willem Raes, “ ‘Namuda Studies’: Doppler Radar-Based
Gesture Recognition for the Control of a Music Robot Orchestra”
(2012): www.logosfoundation.org/ii/Namuda_JIM_paper.doc (交流电-
cessed 5 十一月 2018).
4 Donna Haraway, Staying with the Trouble: Making Kin in the Chthu-
lucene (达勒姆, NC: Duke Univ. 按, 2016) p. 58.
18 Hahn and Bahn [7] PP. 229–238.
5 中号. Beth Dempster quoted in Haraway [4] p. 61.
6 Haraway [4] p. 61.
7 Tomie Hahn and Curtis Bahn, “Pikapika—The Collaborative Com-
position of an Interactive Sonic Character,” Organised Sound 7, 不.
3, 229–238 (2002).
19 Steven Kemper and Scott Barton, “Mechatronic Expression: Recon-
sidering Expressivity in Music for Robotic Instruments,” Proceedings
of the 18th International Conference on New Interfaces for Musical
Expression (NIME) (Blacksburg, VA: Virginia Tech University, 2018)
PP. 84–87.
20 Ajay Kapur et al., “Collaborative Composition for Musical Robots,”
Journal of Science and Technology of the Arts 1, 不. 1 (2009) p. 49.
8 Laetitia Sonami, lady’s glove: www.sonami.net/ladys-glove (访问过
12 行进 2019).
21 Dan Trueman and R. Luke DuBois, PeRColate: www.music.columbia
.edu/percolate (访问过 1 一月 2019).
9 Onyx Ashanti, Wreck now—sonocybin: www.youtube.com/watch
?v=gaFi_jUHS4s (访问过 12 行进 2019).
22 Haraway [1] p. 295.
10 Scott Barton, Ethan Prihar and Paulo Carvalho, “Cyther: A Human-
Playable, Self-Tuning Robotic Zither,” Proceedings of the 2017 新的
Interfaces for Musical Expression Conference (哥本哈根, 2017) PP.
319–324.
11 LP Demers and Bill Vorn, Inferno: www.billvorn.concordia.ca
/menuall.html (访问过 12 行进 2019).
12 Åsa Unander-Scharin and Carl Unander-Scharin, “Robocygne:
Dancing Life into an Animal-Human-Machine,” Leonardo 49, 不.
3, 212–219 (2016).
13 Marco Donnarumma, “Music for Flesh II: Informing Interactive
Music Performance with the Viscerality of the Body System,” Pro-
ceedings of the 2012 Conference on New Interfaces for Musical Expres-
锡安 (安娜堡, MI: 密歇根大学, 2012).
14 Claude Cadoz and Marcelo M. Wanderley, Gesture—Music (巴黎:
IRCAM, 2000).
15 Aurie Hsu and Steven Kemper, “Kinesonic Composition as Cho-
reographed Sound: Composing Gesture in Sensor-Based Music,”
诉讼程序 2015 International Computer Music Conference
(丹顿, TX: University of North Texas, 2015) PP. 412–415.
23 Cynthia Breazeal and Brian Scassellati, “Robots That Imitate Hu-
芒斯,” Trends in Cognitive Sciences 6, 不. 11, 481–487 (2002): 万维网
.doi.org/10.1016/S1364-6613(02)02016-8.
24 Oscar D. Lara and Miguel A. Labrador, “A Survey on Human Ac-
tivity Recognition Using Wearable Sensors,” IEEE Communications
Surveys Tutorials 15, 不. 3, 1192–1209 (2013): www.doi.org/10.1109
/SURV.2012.110112.00192.
稿件收到 2 一月 2019.
Aurie Hsu is a composer and performer. She is currently
assistant professor of computer music and digital arts in the
Technology in Music and Related Arts (TIMARA) 部门
at the Oberlin Conservatory.
steven Kemper is a composer and music technologist.
He is currently associate professor of music technology and
composition at the Mason Gross School of the Arts at Rutgers
大学.
Hsu and Kemper, Enacting Sonic-Cyborg Performance through the Hybrid Body in Teka-Mori and Why Should Our Bodies End at the Skin? 87
从http下载的://direct.mit.edu/lmj/article-pdf/doi/10.1162/lmj_a_01069/1716721/lmj_a_01069.pdf by guest on 07 九月 2023
