Andrew McPherson - 麻省理工学院人工智能研究专业

Andrew McPherson
Centre for Digital Music
Queen Mary University of London
Mile End Road, London E1 4NS, 英国
a.mcpherson@qmul.ac.uk

Buttons, Handles, 和
Keys: 进展
Continuous-Control
Keyboard Instruments

抽象的: The keyboard is one of the most popular and enduring musical interfaces ever created. 今天, the keyboard
is most closely associated with the acoustic piano and the electronic keyboards inspired by it, which share the
essential feature of being discrete: Notes are defined temporally by their onset and release only, with little control
over each note beyond velocity and timing. Many keyboard instruments have been invented, 然而, that let the
player continuously shape each note. This article provides a review of keyboards whose keys allow continuous control,
from early mechanical origins to the latest digital controllers and augmented instruments. Two of the author’s own
contributions will be described in detail: a portable optical scanner that can measure continuous key angle on any
acoustic piano, and the TouchKeys capacitive multi-touch sensors, which measure the position of fingers on the key
surfaces. These two instrument technologies share the trait that they transform the keys of existing keyboards into
fully continuous controllers. In addition to their ability to shape the sound of a sustaining note, both technologies also
give the keyboardist new dimensions of articulation beyond key velocity. Even in an era of new and imaginative musical
interfaces, the keyboard is likely to remain with us for the foreseeable future, and the incorporation of continuous
control can bring new levels of richness and nuance to a performance.

In his “Interaction Design Sketchbook,” Bill Ver-
plank divides user interface controls into buttons for
discrete actions and handles for continuous actions:

When I press a button (例如, 在) the machine
takes over. . . . Handles can be “analogic.” With
buttons, I am more often faced with a sequence
of presses. With a handle a sequence becomes a
gesture. I use buttons for precision, handles for
expression (Verplank 2009, p. 7).

The musical keyboard presents an interesting
cross between buttons and handles. 就其一而言
手, familiar keyboard instruments like the piano,
harpsichord, and organ are essentially discrete, A
row of buttons. Key presses are temporally and
dynamically defined by their onset and release; 钥匙
motion after onset is irrelevant to the sound. 在
另一方面, the physical motions of the human
performer are necessarily continuous. The precise
qualities of hand and finger movement, 通常
known as “touch,” hold great importance for expert
performers and teachers. 而且, the listener
typically hears a keyboard performance not as
sequences of isolated events, but as continuous lines

电脑音乐杂志, 39:2, PP. 28–46, 夏天 2015
土井:10.1162/COMJ一 00297
C(西德:2) 2015 麻省理工学院.

and phrases, just as a violinist or vocalist might
perform.

尽管如此, for all the expressive capabilities
of the keyboard, instrument designers have long
wanted more, pursuing what Dolan (2008, p. 4)
describes as “an age-old musical problem: 这
creation of an instrument that combined the dy-
namic nuance and sustaining power available to
bowed-string instruments with the convenience
of a keyboard instrument.” In other words, 可以
the keyboard instead consist of a series of handles,
each one capable of continuously shaping its note,
while retaining the advantages of polyphony and
familiarity? Proposed solutions to this problem date
back as far as Leonardo da Vinci’s sketches for the
“viola organista” from 1488–1489, but more than
five hundred years later, there is still no single
agreed-upon approach.

This article explores advances in the design of
keyboards featuring continuous control of one or
more musical dimensions, from early mechanical
origins to digital instruments of the past three
几十年. The focus is placed on the keyboard inter-
face itself rather than the associated means of sound
生产. When discussing digital instruments,
the scope of the article is also limited to interfaces
that resemble the familiar keyboard, 而不是
the large and active research area of novel contin-
uous controllers. A general review is followed by

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

discussions in greater detail of two of the author’s
recent interfaces: a continuous optical scanner for
the acoustic piano keyboard (McPherson 2013b),
and the TouchKeys capacitive touch-sensor system,
which measures finger position on the key surfaces
(McPherson and Kim 2011a). A recurring theme
throughout this article is “playability,” exploring
how design decisions can affect the performer’s
经验.

Gesture and Touch at the Keyboard

Playing any keyboard instrument places stringent
cognitive and biomechanical demands on the
player, and accordingly, studying the movements
of pianists has long held interest for psychologists
and musicologists. In the 1920s, Otto Ortmann
studied the mechanics of piano performance from
the perspectives of the instrument (Ortmann 1925)
and the player’s body (Ortmann 1929). Recent
studies have examined the details of finger motion
as it relates to tempo and timing accuracy (Goebl
and Palmer 2008; Dalla Bella and Palmer 2011;
Goebl and Palmer 2013). Force between finger and
key has been studied in relation to dynamic level
(Kinoshita et al. 2007) and movement efficiency
(Parlitz, Peschel, and Altenm ¨uller 1998). 其他
research has explored the kinematics of the hand
(Furuya, Flanders, and Soechting 2011), the upper
limbs (Furuya and Kinoshita 2008), and the torso and
头 (Castellano et al. 2008; Thompson and Luck
2012). Studies often compare expert pianists with
amateurs, with experts demonstrating better use
of the innate connectivity between fingers (Winges
and Furuya 2014), more economical use of finger
力量 (Parlitz, Peschel, and Altenm ¨uller 1998) 和
more efficient coordination of upper limb motion
(Furuya and Kinoshita 2008). Further review of the
biomechanics of musical performance has been
documented by Metcalf et al. (2014).

The essential discreteness of the acoustic piano

has been a subject of a longstanding debate: Is
it possible to alter the sound of a piano note
independently of its loudness? 哈特, Fuller, 和
Lusby (1934) showed that the velocity of hammer–
string collisions was exclusively responsible for

the sound of a note: Every change in tone was
accompanied by a change in loudness and vice versa.
尽管如此, pianists often describe their touch in
rich, multidimensional terms, which go far beyond
this simple velocity-based model.

部分, the musical importance of touch can be
explained by its effect on the timing and velocities of
longer musical phrases. Goebl, Bresin, and Fujinaga
(2014) showed that auxiliary impact noises between
finger and key or between key and key bed influence
the perception of a single tone, and that these noises
allow a listener to discriminate between pressed
(non-percussive) and struck (percussive) touches. 在
my own previous work (McPherson and Kim 2011b),
I found that, regardless of the acoustic effects, A
pianist can control the continuous motion of the
key in several simultaneous dimensions besides
velocity. These dimensions include percussiveness,
重量, and depth. Bernays and Traube (2014) 还
found meaningful variations in continuous key-
motion profiles across four pianists, dependent both
on individual style and their intended timbre in
表现.

These studies offer instrument designers cause
for both optimism and caution. Keyboard technique,
even on essentially discrete instruments, consists
of meaningful, reproducible patterns of continuous
motion that could be adapted for new musical
结束. 同时, the complexity of existing
keyboard practice means that any new continuous
controls should be introduced carefully to avoid
overburdening the performer.

Aspects of Discrete and Continuous Technique

Before considering specific instruments, it is worth
clarifying the ways in which a keyboard might be
classified as continuous or discrete. I will leave aside
the complex issue of phrasing across successive
notes to consider three aspects of a single note:
onset, sustain, and release.

Onset concerns the brief period when the note
begins. Though velocity (dynamic control) 是个
most familiar feature of piano note onset, 这是
only one possible feature of the broader quality of
articulation. The many types of note onset possible

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

on bowed string instruments show that there is
considerable space left to explore in this area (为了
an overview of string articulations, see Schoonder-
waldt and Demoucron 2009). One subtle example
can be found on mechanical tracker organs, 在哪里
the speed of key onset allows control over the initial
transients when the pipe speaks. Another example is
the classic Hammond B3 organ, where each key had
nine electrical contacts, one for each drawbar. 这
system was responsible for the “key click” sound
musicians came to prize, and in fact the click can
be subtly varied in length and loudness by altering
the speed of the key, causing different drawbars to
engage at slightly different times. These effects were
deemed sufficiently important that the 2003 digital
recreation of the Hammond B3 replicated the origi-
nal multi-contact keyboard (Robjohns 2003). 更远
possibilities for note articulation will be explored in
the TouchKeys section later in this article.

Sustain covers the middle of the note between
onset and release. In the instrument design litera-
真实, “continuous control” is often shorthand for
the ability to modulate the sustain of the note,
changing its pitch, 体积, or timbre. 尤其,
the ability to change dynamics or add vibrato from
the keyboard have long been sources of inspiration
for designers. This article will focus on ways of
shaping a note’s sustain from the keyboard itself,
leaving aside supplementary controls (例如, the organ
swell pedal or the synthesizer pitch wheel).

Release can be as simple as stopping a note
abruptly, but variations are possible. Perhaps sur-
prisingly, the acoustic piano keyboard offers a degree
of continuous control at release, since the damper
can be brought in contact with the string gradually or
abruptly. Jazz brass instrument technique suggests
other musical possibilities on note release, 包括
falls (dropping pitch on release) and doits (upward
glissando on release). The TouchKeys section in this
article will explore how these techniques can be
adapted to the keyboard.

from 1488–1489, depicting an instrument with a
rosined wheel to bow a series of strings (Dolan 2008).
A keyboard pushes each string into contact with the
wheel, causing the note to speak. The instrument
was apparently never built by da Vinci himself, 但
it has been recreated in modern times (Zubrzycki
2013). The earliest similar instrument to be con-
structed was Hans Haiden’s Geigenwerk in 1575.
This instrument and most subsequent attempts at
sustaining keyboard instruments were based on the
same principle of strings bowed by rosined wheels
(Dolan 2008).

New instrument technologies flourished in the
18th century; Dolan (2008) explores the motiva-
tions and creations of this period. Perhaps the most
famous invention combining polyphony with con-
tinuous note shaping was Benjamin Franklin’s glass
harmonica (1761), consisting of a rotating spindle of
glasses, which are induced to vibration by moistened
fingers placed on the rims. Subsequent inventors
sought to improve the instrument by replacing direct
finger–glass contact with a keyboard, showing even
in that era the allure of the keyboard as a convenient
and versatile interface. 很遗憾, keyboard
variants tended to reduce the subtlety of control,
and modern recreations of the glass harmonica
typically follow Franklin’s original approach.

Among keyboard instruments, the most cele-
brated example of continuous sustain shaping can
be found on the clavichord. Pressing the keys causes
a metal tangent to strike the string; the tangent
remains in contact with the string, forming one of
its termination points. Pressure on the key alters
the string tension, giving rise to the characteristic
Bebung vibrato effect (柯克帕特里克 1981). The del-
icate sound of the clavichord was too soft for use
in chamber music, and in the 18th century it was
supplanted by the harpischord and later the piano. 它
would take until the electronic instruments of the
20th century for this vibrato capability to return to
keyboard instruments.

Historical Continuous-Control Keyboards

Early Electronic Instruments

The earliest sketches for a continuous-control key-
board instrument are found in da Vinci’s notebooks

Continuous pitch control was a prominent feature
of early electronic instruments. Reviews of early

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

electronic instruments have been presented by
Curtis Roads (1996) and Joseph Paradiso (1997). 这
theremin (1920) controlled pitch and volume using
electric field sensing of the player’s hands in the air.
On the Trautonium (1929), the performer controlled
pitch and volume by changing the position and
pressure of the finger on a metal wire (Galpin
1937). The Ondes Martenot (1928) is well known
for controlling pitch either from a sliding ring worn
on the finger or from a keyboard, which slid from
side to side to allow vibrato. 实际上, the first version
of the instrument had only the ring controller; 这
keyboard was a later addition (Paradiso 1997).

The Ondes Martenot used a separate pressure
控制, played by the left hand, to regulate the
体积 (Quartier and Meurisse 2014). The Ondio-
线 (1941) combined side-to-side vibrato from the
keyboard with the ability to control volume through
key pressure. The Electronic Sackbut (1948) added
a third dimension of timbre, controlled with the
left hand (Young 1989). These instruments were
all monophonic, but in fact the first polyphonic
electronic keyboard predates them all: Thaddeus
Cahill’s polyphonic Telharmonium was first in-
vented in 1901 and redesigned several times. 这
third version, introduced 1910, featured a pressure-
sensitive keyboard to control the volume of each
笔记 (Weidenaar 1995). Weighing in at 200 tons and
requiring over 600 kilowatts of power, 然而,
the Telharmonium was soon supplanted by cheaper
electric organs.

在 1973, keyboard vibrato reappeared on the
polyphonic Yamaha GX-1, an analog synthesizer
whose keyboard could be slid from side to side
to alter the pitch (Paradiso 1997). Some early
polyphonic synthesizers, for example the Yamaha
CS-80 (1976), featured polyphonic aftertouch, 哪个
allows independent modulation of the volume or
timbre of each sustaining note through key pressure.
Polyphonic aftertouch is part of the standard MIDI
specification, and was featured in early MIDI
controllers such as the Kurzweil Midiboard (Moog
1987), although it is less often found on current
keyboards. Aftertouch is limited in two ways: (1) 它
becomes active only during the sustain of the note,
when the key is down, and hence cannot affect the
onset or release; 和 (2) it has a single direction

of control (increasing pressure, with key release
necessarily passing through minimum pressure).
Prolonged heavy pressure on the keys also risks
fatiguing the player.

Two novel prototypes developed over the 1970s
and 1980s added entirely new dimensions to the
standard keyboard. The Key Concepts Notebender
(1978–84) had keys that could mechanically slide
forwards and backwards to bend the pitch up or
向下 (Moog 1987). The Moog Multiply Touch-
Sensitive Keyboard (1972–90, 比照. Moog and Rhea
1990) sensed the finger position in two dimensions
on the surface of each key, letting the player control
several parameters simultaneously. The prototype
instrument was used by composer John Eaton
(Eaton and Moog 2005) but was never commercially
produced.

Recent Developments

自20世纪90年代以来, many new extensions to the
keyboard have been developed. 其中一些
involve sensor technology embedded into the
keyboard itself, whereas others add extra sensors to
capture hand and finger motion while the performer
戏剧. I will leave aside systems that add new
dimensions to the keyboard by using separate modes
of interaction (例如, foot pedals) or that involve
automated processes not under the control of the
performer.

Measurement Systems for Performance Analysis

In addition to general-purpose optical motion-
tracking systems often used in performance analysis,
several sensor technologies have been developed
specifically for the keyboard. The systems in this
section appear to be designed primarily to generate
data for later offline analysis rather than for new
modes of real-time musical control, 虽然
systems could presumably be adapted to the latter
purpose.

The B ¨osendorfer 290SE reproducing grand piano

features a measurement system for continuous
key angle as well as MIDI (Moog and Rhea 1990);

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

this feature is also present on the successor CEUS
系统. Bernays and Traube (2012) have developed a
software toolbox for extracting subtle gestural detail
from CEUS key angle data. Aristotelis Hadjakos
developed several sensor systems for motion capture
at the piano, including ones that use accelerometers
(Hadjakos, Aitenbichler, and M ¨uhlhauser 2009)
and the Kinect depth camera (Hadjakos 2012).
MacRitchie and Bailey (2013) developed a piano-
specific system for tracking the fingers and wrists
using a high-speed camera and painted dots on the
knuckles. Grosshauser and Tr ¨oster (2013) explored
attaching custom force-sensing resistors onto the
keyboard surface to measure location and pressure.
For a general review of musical instrument sensor
技巧, see Medeiros and Wanderley (2014).

Keyboards Integrating Continuous Control

Following the development of the Moog Multiply
Touch-Sensitive keyboard in the 1970s and 1980s
(Moog and Rhea 1990), several other instruments
have been created that incorporate continuous
measurement. Freed and Avizienis (2000) created a
keyboard controller that measured the continuous
angle of each key and that could stream data
over a digital audio or Ethernet link. My own
TouchKeys system (McPherson and Kim 2011a;
麦克弗森 2012) added capacitive touch sensing to
the surface of the keys, providing two-dimensional
measurement of each finger position. This system,
which is further described later in this article, 是
designed to attach to any existing keyboard. 数据
from each dimension can be flexibly mapped to
MIDI or Open Sound Control (OSC) 消息.
Jeff Snyder’s JD-1 synthesizer controller (斯奈德
and McPherson 2012) also uses capacitive sensing,
measuring the continuous contact area of the fingers
on fixed aluminum keys.

Recent commercial examples include the Endeav-
our EVO (Kirn 2012), which used capacitive sensing
on one axis to measure finger position along a portion
of the key, and the Keith McMillen QuNexus (万维网
.keithmcmillen.com/qunexus), a miniature pressure
pad controller in a keyboard layout that features

polyphonic aftertouch. Roger Linn’s Linnstrument
(www.rogerlinndesign.com/linnstrument.html), 这
Keith McMillen QuNeo (www.keithmcmillen.com/
QuNeo), and the Eigen Labs Eigenharp (万维网
.eigenlabs.com) include pressure pads that sense tilt
in two axes, though the pads are organized in a grid
rather than a keyboard layout.

Some instruments have taken continuity a
step further by replacing discrete keys with a
single continuous control surface. Lippold Haken’s
Continuum Fingerboard (Haken, Tellman, and Wolfe
1998) presents the performer with a continuous
pressure-sensitive control surface that measures
the three-dimensional position of each finger. 这
horizontal axis is commonly used to control pitch,
and printed patterns on the surface indicate where
the black and white notes might be found.

The ROLI Seaboard (Lamb and Robertson 2011)
also uses a continuous pressure-sensitive control
surface shaped into raised “keywaves” resembling
the black and white keys. Both the Continuum
and the Seaboard allow the performer to glide
between notes by dragging a finger horizontally. 到
address the challenge of starting a note in tune on a
continuous surface, both instruments have options
to guide the starting pitch toward the nearest
semitone. The desire for finer control over pitch
has also prompted the development of microtonal
keyboards (Keislar 1987) with more than 12 keys to
the octave, though each key tends to be a discrete
控制 (onset and release only).

Augmented Instruments with
Continuous-Control Keyboards

在 2009, I developed the magnetic resonator piano
(麦克弗森 2010), which manipulates the vibra-
tions of the acoustic piano using electromagnets
(see also the Electromagnetically-Prepared Piano,
Bloland 2007). The electromagnets induce vibrations
in the strings independently of the hammers, 在-
abling infinite sustain and crescendos from silence.
Initially the instrument was controlled by two
MIDI keyboards, but to achieve the full potential of
continuous note shaping, control from the keyboard

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

with more detail was needed. 在 2010 I modified a
Moog PianoBar (described subsequently) 测量
continuous key angle at a rate of 600 Hz per key
(McPherson and Kim 2010). Multidimensional map-
pings between key motion and magnet signals allow
continuous shaping of the volume, 沥青, 音色,
and even individual harmonics of each note. 这
work laid the foundation for the later redesign of
the keyboard scanner presented in the following
部分.

与此相类似, Shear and Wright (2012) aug-
mented the Fender Rhodes electric piano to ma-
nipulate the vibrations of the metal tines using
electromagnets, with continuous key angle used
to alter their volume. Other augmented keyboards
have taken a different approach, adding external
sensors that are aimed at the hands while they play
the keys. Yang and Essl (2012) use a Kinect above
a MIDI keyboard to create a three-dimensional
control space above and around the keys; Gillian
and Nicolls (2012) and Van Zandt-Escobar, Carami-
aux, and Tanaka (2014) use Kinect tracking with
gesture recognition to control audio processing of
the acoustic piano sound. William Brent’s Gestu-
rally Extended Piano (Brent 2012) uses IR camera
blob tracking to track the hands and forearms,
controlling real-time audio processing. Hadjakos
and Waloschek (2014) demonstrated the use of
wrist-worn accelerometers to add vibrato to a MIDI
keyboard by shaking a hand back and forth.

Active Haptic Control

A final class of continuous-control keyboards
focuses not on the sound production, but on the
feel of the key action. Tactile feedback is important
for expert performance (Goebl and Palmer 2008),
but a limitation of MIDI keyboards is that their
action feels identical regardless of sound setting.
Cadoz, Lisowski, and Florens (1990) created a high-
bandwidth active-force feedback system using a
specially designed motor whose goal was to recreate
the feel of familiar keyboard actions including
the piano and harpsichord. A similar concept was
patented by Richard Baker (1990). This work was
further extended in Brent Gillespie’s Touchback

Keyboard (Gillespie 1996; Gillespie et al. 2011)
and Oboe and De Poli’s MIKEY (multi-instrument
virtual keyboard, 比照. Oboe and De Poli 2002; Oboe
2006). Lozada, Hafez, and Boutillon (2007) 使用
magneto-rheological fluid to achieve similar force
feedback. Bill Verplank’s “Plank” allows force-
feedback keys to be built from voice-coil motors
taken from surplus hard drives (Verplank, Gurevich,
and Mathews 2002).

讨论

Many new keyboard instruments have been invented
over the centuries, but few have caught on. 在里面
past decade, new controllers have flourished in
both the academical and commercial worlds, 所以
perhaps we will see a greater adoption of new
keyboard instruments in the coming years. 仍然,
two impediments to wider use stand out. 第一的,
most new instruments have been expensive and
many were heavy or unwieldy, limiting their
appeal to the most dedicated performers. 第二,
keyboard technique is already challenging, 和
continuous polyphonic control is harder still. 在
an interview (New Jersey Star-Ledger, 13 行进
2006), John Eaton reflected on the Moog Multiply
Touch-Sensitive Keyboard: “It’s very difficult to
玩. But an instrument should be difficult to play.
That’s the only way to master musical materials, 经过
overcoming these difficulties.”

With widespread, low-cost sensor technology
and few limitations on real-time audio synthesis,
current challenges for continuous-control keyboards
are not technical but human. The capabilities of
the player—including the lengths of the fingers,
the constraints of traditional technique, 和
sensorimotor processes developed through years of
practice—must become explicit considerations in
the design of new instruments. 在很多情况下, 一个新的
sensor dimension is only as useful as the mapping
that is applied to it (for a discussion of playing
new interfaces, see Paradiso and O’Modhrain 2003).
When working with an established interface like
the keyboard, it is particularly important that new
mappings do not interfere with familiar technique
(麦克弗森, Gierakowski, and Stark 2013).

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 1. Optical sensing
architecture for the Moog
PianoBar, with a side view
of white key and a front
view of black key (A) 和
new scanner, with a side

看法 (乙). The PianoBar
uses beam-interruption
sensing on the black keys,
which does not allow
measurement of
continuous key angle.

The new design uses
identical reﬂectance
sensors on each key, each
sensor containing an LED
and phototransistor in a
single package.

of these instruments mean that few composers
will make use of this capability, 然而. 的确,
any electroacoustic music involving acoustic piano
can face barriers to performance. Few venues have
MIDI-enabled acoustic pianos, so when performance
data from the keyboard is needed, electronic key-
boards are often used even if an excellent acoustic
instrument is available.

This section presents a new hardware solution for

portable, detailed sensing of continuous key angle
on any acoustic piano, featuring communication
and mapping options specifically aimed at electro-
acoustic and augmented instrument performance.

Moog PianoBar

The PianoBar, designed by Donald Buchla in 2001
and sold by Moog Music from 2003 到 2007, 是
a popular accessory for combining piano with
electronics. An optical sensor strip rests at the back
of the keyboard, generating MIDI data in response
to key motion. Discontinued for several years, 这
PianoBar has now become increasingly scarce but
remains in demand as one of the few convenient,
practical options for adding MIDI capability to any
acoustic piano.

Internally, the PianoBar uses optical reflectance

sensing to measure the white keys and beam-
interruption sensing on the black keys (见图
1A). A separate magnetic proximity sensor measures
the position of the left (una corda) and right (damper)
pedals. LEDs within the keyboard sensor bar indicate
active notes, with orange LEDs used for the white
keys and green LEDs for the black keys. Unlike most
系统, which are tied to a specific instrument or
require lengthy setup and calibration, the PianoBar
can be deployed in less than five minutes.

动机: Beyond the PianoBar

The scanner system described here aims to address
the void left by the discontinuation of the PianoBar,
while adding new capabilities for performance and
research that go beyond any existing key-angle
measurement system. Goals include:

Measuring and Mapping Continuous Key Angle

Though note onset on the acoustic piano is essen-
tially a discrete process defined by the instant when
the hammer, flying freely, strikes the string, 这
hand and finger motions used in playing the piano
are continuous and multidimensional. Whether
or not these extra dimensions have any direct
acoustic effect, they can be reliably controlled by
the performer (Goebl, Bresin, and Galembo 2005;
McPherson and Kim 2011b) and thus they can be
used for augmented piano performance (麦克弗森
and Kim 2010).

The B ¨osendorfer 290SE and CEUS acoustic pianos
incorporate sensors measuring the continuous angle
of each key (500 赫兹, 8-bit sampling on the CEUS;
see Bernays and Traube 2014). The cost and scarcity

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 2. Optical scanner
hardware. Optical sensors
over each key (A). 白色的
stickers are attached to the
black keys to increase

their reﬂectance. View of
reﬂectance sensors, each
containing an infrared
LED and phototransistor
in a compact package (乙).

1. Continuous key angle at high temporal and
spatial resolution, from which MIDI data can
be derived as needed.

2. Real-time extraction of key-touch features
associated with aspects of keyboard tech-
nique that go beyond velocity. 这些包括
percussiveness (pressed versus struck keys,
see Goebl, Bresin, and Galembo 2005) 和
aftertouch (key pressure).

3. Flexible mapping options from key motion
to sound, building on standard protocols,
such as OSC (Wright and Freed 1997),
and augmented instruments, 例如
magnetic resonator piano (McPherson and
Kim 2010).

4. Rich visual feedback from RGB LEDs above
each key, providing contextual information
beyond MIDI note on and note off.

5. Physical portability, including the ability to

pack down in pieces.

Direct hammer measurement (as found in the

B ¨osendorfer instruments) and extended sensing
能力 (capacitive, 视频, ETC。) were impractical
within the setup and portability constraints. Hard-
ware audio synthesis (as found in the PianoBar) 曾是
not a priority. The height of the PianoBar scanner
can be accidentally changed when bumped, so a
more secure adjustment mechanism was desired,

even if this required a somewhat longer setup
时间. 最后, because most current practice uses
computer processing rather than hardware MIDI
synthesis, a USB connection was preferred to the
MIDI ports found on many systems.

Hardware Design

数字 2 shows the scanner design, which features
four circuit boards attached to an acrylic mounting
bracket. Each board covers roughly two octaves of
sensors (25 sensors for the top board, which includes
the high C, 并且只有 15 for the bottom board).

Optical Sensors and Communication

Near-field optical reflectance sensing is used to
measure the position of each key. Fairchild QRE1113
sensors, which include an LED and a phototransistor
in a compact package, are mounted across the
bottom edge of each board (见图 2). A schematic
of the circuit and analysis of its operation can be
found in an earlier publication (McPherson 2013b).
Sensor data is reported at a 1-kHz sample rate for
each key, with 12-bit resolution. To reduce noise
and interference from ambient light, each sample
is the average of eight differential measurements.

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

Each differential measurement is the difference
between the reading with LED on and the reading
with LED off; 因此, each 1-msec sample contains 16
analog-to-digital conversions.

An important difference from the Moog PianoBar

concerns the treatment of the black keys. 在
the PianoBar, the emitter and detector are placed on
opposite sides of the key, such that the key interrupts
the beam when at rest (see Figure 1a). Although this
is sufficient for MIDI note-on and note-off data, 这
arrangement cannot sense continuous key position
over the entire key range. My previous work
modified the PianoBar to extract continuous sensor
价值观 (McPherson and Kim 2010), but many of the
novel mappings, which depended on full-range key
位置, were only possible on the white keys. 经过
对比, the new scanner uses identical reflectance
sensors on every key. Because the black keys do
not reflect enough light to be reliably measured,
removable white stickers are affixed to them before
the scanner is installed (see Figure 1b). The stickers
can be small, using residue-free adhesive, 和他们
do not shorten the playable length of the black keys
beyond the space already taken up by the scanner.
The process of affixing stickers adds two to three
minutes to the setup time, but the higher data
quality easily outweighs this drawback.

Measurements of key position are sent to a
computer via USB (Communication Device Class),
using a custom binary protocol. 在实践中, 数据
rates of approximately 2 Mbps are required to fully
sample an 88-note keyboard, comfortably within
the 12-Mbps capability of full-speed USB.

RGB LEDs

The PianoBar included orange or green LEDs over
each key. When I used the PianoBar on the magnetic
resonator piano (see the earlier Augmented Instru-
ments section; 还, McPherson and Kim 2010),
performers often asked what the colors meant,
suggesting that multicolored LED feedback could
provide useful additional information. The new
scanner includes RGB LEDs above each key that can
be set to arbitrary hue, saturation, and brightness.
The Mappings section, later in this article, discusses
possible relationships between key motion and

LED color; 最终, any relationship is possible,
including driving the LEDs independently of the
keys.

Real-Time Data Analysis

This scanner is intended to provide multidimen-
sional key-gesture sensing on any piano. 连续的
key angle data significantly exceeds the level of de-
tail provided by MIDI, and it can be used to derive
several features of each key press (McPherson and
Kim 2011b). Prior to use, the scanner is calibrated
by pressing each key to set the minimum and max-
imum values. From this point, in addition to raw
sensor data, each new note onset generates several
特征:

1. Velocity, similar to MIDI, but the point

of measurement (“escapement point”) 能
be changed programmatically, unlike other
scanners. 例如, a shallower escape-
ment point will respond more quickly to new
key presses. Resolution of the measurement
is not limited by 7-bit MIDI.

2. Percussiveness, which includes several

features related to the initial velocity spike
that struck keys exhibit. This includes
magnitude and location of the initial velocity
spike and the relative amount that the key
position changed before and after the spike.
An overall percussiveness score is also
calculated, which can be mapped to an
independent dimension of sound production.
Previous work showed that performers
can control percussiveness and velocity
independently (McPherson and Kim 2011b).
3. Aftertouch, or weight, which measures the
amount of force the player exerts on the key
床. This can be a single score, immediately
following note onset (the deepest point
of the key throw), or it can be measured
continuously throughout a key press.
4. Release velocity, supported by the MIDI

standard but rarely implemented, 措施
the speed of key release. This is calcu-
lated identically to onset velocity at a

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 3. Continuous key
position for two notes. A
note played percussively
with aftertouch (A). 笔记
the spike at beginning and
position variations at full

press. Vertical lines
indicate beginning and
end of onset and release
阶段. A vibrato gesture
played by shaking the key
between thumb and

foreﬁnger (乙), 哪个
produces a harmonic
glissando on the magnetic
resonator piano.

user-definable position threshold. Alterna-
主动地, with careful calibration, continuous
key position can identify the point at which
the damper first lightly touches the string
during a slow release, and also when the
damper fully rests on the string.

数字 3 shows two plots of key motion with
these features; these plots are generated in real time
from the controller software. Internally, the software
operates a state machine, which tracks the minima
and maxima of the key position, segmenting each
note into onset, sustain, and release phases. 这
software also captures partial key presses and taps
that would fall below the traditional note onset
临界点.

Mappings

The scanner is intended to be a flexible device
for capturing the expressive details of keyboard
技术. This section describes two approaches
to mapping, one that generates MIDI data, 和
another that applies to the magnetic resonator piano
(McPherson and Kim 2010). Users are also free to
develop their own mappings based on raw data or
features communicated by OSC.

MIDI Mappings

MIDI onset and release, with velocities, 可
derived from continuous position. Key pressure, 作为
detected by subtle variations in position when the
key is fully pressed, is transmitted as polyphonic
aftertouch. The software can act as a virtual MIDI
source for other programs.

A novel MIDI mapping is the use of the percus-
siveness feature to trigger a second instrument. 在
this mode, each key press generates MIDI notes on
two channels. The first channel retains the standard
行为, whereas on the second, the onset velocity
corresponds to the percussiveness of the note. 一个
example musical application uses a sustained voice
with slow attack (例如, strings) on the main channel
and a short, percussive sound (例如, marimba) 在
the percussiveness channel. 这样, the type of
key touch creates a readily apparent variation in the
output sound quality.

In MIDI mode, the RGB LED over a key lights up

green on the initial touch. When the key reaches
the key bed, further pressure (aftertouch) alters the
hue of the LED, moving toward red at maximum
pressure. Notes played percussively begin with a
blue flash to indicate the different touch.

Magnetic Resonator Piano

The magnetic resonator piano (MRP) is an elec-
tromagnetically augmented acoustic piano (看到
Augmented Instruments section of this article). 在
2011, the MRP underwent a complete redesign and
several rounds of polishing in response to collabo-
rations with composers and performers (麦克弗森
and Kim 2012). Continuous key motion is founda-
tional to MRP technique, and this is the first scanner
to enable the use of a full complement of extended
techniques on both black and white keys. 一些
mappings have been developed that depend on key
位置, velocity, and the state of the detection
系统.

Key position, before it reaches the key bed,
determines the intensity of the note. Intensity is an

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

intermediate parameter that can, 反过来, be mapped
to changes in amplitude and spectral content.
Lightly touching the keys without pressing them all
the way down can thus produce soft, subtle tones.
When the key is down, key pressure engages as a
second brightness dimension that is mapped to the
spectral centroid of the electromagnet waveform,
pushing the energy higher up the harmonic series
for a brighter sound. On release, intensity again
scales with position, enabling gradual releases.
Because piano keys bounce slightly after release, A
post-release state suppresses any sound from these
unwanted motions.

Key vibrato is an extended technique made possi-
ble by the scanner (see Figure 3b). When the low-pass
filtered key velocity exhibits periodic positive and
negative peaks spaced less than 300 msec apart,
the vibrato mapping is engaged. A vibrato motion
causes a progressive increase in the pitch of the note,
which moves stepwise up the harmonic series of the
string. 这样, tapping repeatedly on the key, 或者
oscillating it between thumb and forefinger, causes
a shimmering effect as the string rings at each of its
harmonics.

Other mappings break down the traditional
independence of the keys. On non-keyboard instru-
评论, the sound of a note is strongly affected by
what preceded it and what else sounds simultane-
乌斯. On the MRP, when one key is held down and
a second key one or two semitones away is touched
lightly, the partially pressed key bends the pitch
of the original note. The bend is proportional to
key position, with a full press bending the note to
the pitch of the second key. This enables detailed
control of portamento effects.

In MRP mode, the RGB LEDs scale in brightness

with key position for partial presses. Pitch-bend
gestures, which always involve two or more keys,
shift the hue toward the blue end of the spectrum,
with green indicating no bend and violet indicating a
bend of over one full semitone. Harmonics produced
by vibrating the key cycle rapidly through the hues
with a lower color saturation (IE。, a whitish tint).
These visual mappings highlight the activation of
the extended techniques and help the performer
regulate their execution.

Observations on Performance

Two interesting observations have emerged from
using the new scanner with the MRP as opposed
to the earlier modified PianoBar. 第一的, the MRP
sounds slightly different, depending on which
scanner is used, even though both support the
same techniques (on the white keys) 和
electromagnetic hardware is identical. The response
to key motion on the new scanner tends to sound
smoother and more predictable, probably because its
ground-up design for continuous angle eliminates
some subtle timing and resolution problems on the
modified PianoBar. This shows that the keyboard,
even when it is mechanically independent of
the sound production, can play a crucial role in
establishing an instrument’s character.

第二, for some musicians, the limitations of
the PianoBar and the subtle artifacts it introduces
into the strings became a crucial part of the MRP’s
musical character. It may therefore be necessary
to emulate some of the apparent failures of the
original technology (especially certain forms of
sensor noise) in order to capture the original sound.
The experience here mirrors the “key click” effect
on the Hammond organ, which was a byproduct
of routing audio signals through mechanical con-
tacts on each key. Originally considered a design
flaw, it became a cherished part of the Hammond
sound that all modern synthetic recreations must
emulate.

TouchKeys

Beyond continuous key angle, a promising source of
continuous control from the keyboard is the location
and motion of the fingers on the key surfaces. Sliding
or rocking the fingers on the keys are gestures
that are easy to understand, but historically they
have been hard to detect. Robert Moog’s Multiply
Touch-Sensitive keyboard (Moog and Rhea 1990) 是
perhaps the only previous instrument that allowed
polyphonic, two-dimensional measurements of
finger position on the key surface, but it never
moved beyond the prototype stage.

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 4. TouchKeys sensor
overlays installed on the
surface of a keyboard. 这
surface of each overlay
包含 26 capacitive
sensor pads.

This section presents the TouchKeys, capacitive
touch-sensor overlays that attach to the surface of
any keyboard, transforming each key into a contin-
uous multi-touch control surface. An introduction
to the TouchKeys idea was presented in McPherson
and Kim (2011A), and further details on the capaci-
tive sensing hardware have been documented in an
earlier publication (麦克弗森 2012). A mapping
approach to add vibrato and pitch bends is described
by McPherson, Gierakowski, and Stark (2013), 和
an experimental approach to controlling a physi-
cally modeled guitar is described by Heinrichs and
麦克弗森 (2012). After a brief overview of the
sensors and their integration into the keyboard, 这
article introduces three new mappings that add new
effects during note onset and release.

Hardware

数字 4 shows the TouchKeys sensor hardware. 每个
sensor consists of a circuit board with 26 capacitive
sensor pads on the top and a microcontroller on
the bottom. The sensors attach to any full-sized
keyboard using strong but removable adhesive. 这
board and adhesive together are 1.6 mm thick and
weigh 5 G (白色的) 或者 2 G (黑色的). The added height
is the same for every key, and the weight does not
significantly alter the keyboard action. Controller
boards placed inside the instrument gather data from
the sensors via flexible ribbon cables and stream it

to a computer via USB. The modular design allows
for any keyboard size from one to eight octaves.

The sensors measure finger position along two
axes at a sampling rate of 200 赫兹, with 8-bit resolu-
tion in the narrow (水平的) axis and more than
10-bit resolution in the long (垂直的) 轴. Up to
three fingers per key can be sensed, and the contact
area of each touch is also measured, differentiating
fingertip from the pad of the finger. These capabil-
ities have evolved over several design iterations in
response to feedback from performers. 例如,
the initial design had only a single sensor dimension
on the black keys (麦克弗森 2012), but in response
to a study on vibrato (麦克弗森, Gierakowski, 和
Stark 2013), the sensors were redesigned to support
two-dimensional control like the white keys.

Integration and Mapping

The TouchKeys are intended to be used with a MIDI
keyboard, as the capacitive sensors do not measure
key motion. Cross-platform, open-source software
(code.soundsoftware.ac.uk/projects/touchkeys) inte-
grates MIDI and touch data, generating MIDI or OSC
output messages that can control any synthesizer.
The key to this process is a set of modular map-
pings (见图 5) that support a variety of flexible
relationships between finger motion and sound.

Sustain Shaping: Simple Mappings

Any touch dimension (x or y position, contact
区域, distance between multiple touches) 可
assigned to any MIDI continuous-control parameter,
including pitch bend or aftertouch. Touch data can
be used either raw or relative to their value at note
onset, and the ranges are adjustable. These mappings
allow the player to shape the sustain of the note,
and they are well suited for timbre effects.

Vibrato and Pitch Bend

Controlling pitch presents a number of subtleties
if the instrument is to avoid interfering with
traditional technique. One mapping allows the
player to add vibrato by rocking a finger horizontally

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

数字 5. TouchKeys
software interface,
including touch display
(左边), general settings with
list of mappings (中心),
and detailed settings for
the release angle mapping
(正确的).

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

on the key surface. Horizontal motion on the keys
is common when relocating the hand, 然而, so a
back-and-forth motion is required before the vibrato
engages.

Another mapping creates a pitch bend when the
finger slides back and forth on the long axis of the
钥匙. To maintain playability, the initial contact
location always produces the expected pitch, 和
only by moving the finger beyond a specified
临界点 (例如, 10% of the key length) does the pitch
bend engage. The pitches at the endpoints of the keys
can be “variable” (pitch bend depends directly on
the distance moved) or “fixed” (reaching the end of
the key in either direction always produces a known
bend, 例如, two semitones up or down). 在之前的
工作, we also experimented with automatically
snapping the note into the nearest semitone when
the bend stops (麦克弗森, Gierakowski, and Stark
2013).

Onset Shaping

The mappings presented thus far shape the sustain of
the note between its onset and release. As discussed
earlier in this article, 然而, onset and release also
have possibilities for continuous control beyond the
simple MIDI-velocity model.

In jazz saxophone technique, a scoop is a note
that begins below its intended pitch and rises into
它. The TouchKeys software includes an “onset
angle” mapping, which can add a scoop at the
beginning of a note. Figure 6a shows its operation.
To trigger a scoop, the key is pressed with the finger
already in motion along the y-axis (IE。, sliding
away from the performer). When the MIDI note-on
message from the key is received, the mapping looks
back through the preceding frames of touch data
to calculate the speed of the finger. If the speed
exceeds a threshold (chosen to avoid interference
with traditional playing), the scoop is triggered. 这

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

电脑音乐杂志

数字 6. Onset angle
mapping adds a scoop
effect when ﬁnger is in
motion at note-on (A).
Release angle mapping

adds falls, doits, or other
conﬁgurable effects for
ﬁnger motion at
note-off (乙).

In the current system, the mapping is designed for
use with wind and brass synthesis, where it triggers
fall and doit effects. These are jazz techniques where
the pitch of the note drops or rises, 分别, 作为
the note is released. The mapping uses key switches
provided by the synthesizer: When a particular
MIDI note is received outside the sounding range of
the instrument, the fall or doit is executed by the
synthesizer.

These onset and release mappings introduce an
important principle in keyboard gesture mapping:
The timescale of the sound effect need not match the
timescale of the physical gesture. Digital musical
instruments often take a frame-by-frame approach
to mapping, where the current frame of sensor data
controls the current sound-synthesis parameters.
相比之下, the finger motion here precedes the
audible effect, and the length of time the finger is
in motion may differ from the length of the scoop
or fall. In my experience, these mappings do not
feel any less immediate to play than the frame-by-
frame mappings during a note’s sustain. 毕竟,
any keyboard playing already involves preparatory
gestures before the sound.

Rapid Retrigger

On the keyboard, unlike many other instruments,
fast repetitions of the same note are difficult. 这
multiple finger sensing of the TouchKeys can be
used to bypass this limitation: When a second finger
is added to a held note, a new onset message can
be generated. Optionally, onset messages can also
be generated when the second finger is removed,
making rapid tremolo effects easy to play.

note begins the pitch bend one or two semitones
below its normal value. The pitch then linearly
ramps up to its normal value over a short time (50
到 200 毫秒). Faster finger speed produces a larger
and longer scoop, although in principle these two
dimensions could be controlled independently.

或者, the same measurement of finger

speed could control other effects at onset.

Special Release Effects

Finger speed can also be measured on note release. 在
the “release angle” mapping, when a MIDI note-off
message is received, the mapping looks back through
preceding touch data to calculate finger speed along
the y-axis (see Figure 6b). In standard technique, 这
speed is typically low, with the finger lifting straight
off the key rather than sliding along it. 所以, A
deliberate motion in either direction can be mapped
to new effects.

Distribution via Kickstarter

As discussed earlier, many new keyboards have
been invented, but few achieve widespread use.
Impediments include cost, inconvenience, 和
general availability. To get the TouchKeys into the
hands of musicians, I launched a Kickstarter crowd-
funding campaign (www.kickstarter.com/projects/
instrumentslab/touchkeys-multi-touch-musical
-keyboard) 在 2013. The campaign successfully

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

supported the distribution of instruments and self-
install sensor kits to musicians around the world.
Further discussion on this campaign and the value of
crowd funding for the instrument design community
can be found in McPherson (2013A).

结论: Beyond “Beyond the Keyboard”?

Inexpensive, ubiquitous computing power has en-
abled a huge variety of new instruments offering
“control and interaction beyond the keyboard
paradigm” (Miranda and Wanderley 2006, p. xx).
The real and exciting promise of radically new
forms of musical interaction can create a temp-
tation to see the keyboard as a throwback, 一个
engineering necessity of previous eras or an easy
default for designers who could find more creative
solutions. History suggests otherwise. 不仅
have keyboards been used for over 500 年, 在-
viving many generations of new technology, 他们
have also been incorporated into instruments that
were already fully functional without them, 在-
cluding glass harmonica derivatives and the Ondes
Martenot. Fundamentally, musical instruments
depend at least as much on human factors as on
技术. Whether because of something inher-
ent in its design, or simply because of the inertia
of previous training, generations of players have
found the keyboard to be conducive to their musical
结束.

尽管如此, not all keyboards are equivalent,
and there is good reason to believe that the discrete
control offered by typical MIDI keyboards imposes
unnecessary limitations on current digital instru-
评论. For most of the instruments mentioned in
this article, their characteristic sounds would not
be possible with only discrete onsets and releases.
而且, the harpsichord, clavichord, pipe organ,
Ondes Martenot, and Hammond B3 are all keyboard
仪器, but no two feel or behave alike. 然而
all will be at least passingly familiar to a trained
pianist. New continuous-control keyboards have the
potential to connect with the expertise of millions
of performers, while offering new ways to shape the
声音.

The Value of Imperfection

In the early days of electronic music, its proponents
cited the tantalizing promise of being able to create
any sound imaginable, freed from the constraints
of mechanical acoustic instruments. But an essay
by J. A. Fuller-Maitland, written over 90 几年前,
seems prescient, pointing out that precisely because
of the interest in creating “perfect” instruments,
that “it seems worth while to point out what
value there may be in the inherent defects of the
various instruments, and in how large a measure
their character is due to these very shortcomings”
(Fuller-Maitland 1920, p. 91).

Perhaps we are in a similar situation today with
the control of musical sounds. Dolan (2013, p. 11)
suggests that the keyboard has, 历史地, been
associated with a vision of complete technological
control of music, 以及更根本的, “that the
basic idea of what we think of as music is bound
up with the interface of the keyboard.” Dolan’s
suggestion refers not only to keyboard music, 但
to the Western musical canon at large. 这个想法
that the organization of music is inherently tied
to the tools for making it is echoed in a different
form by Miranda and Wanderley (2006, p. xx):
“those musicians interested in musical innovation
are increasingly choosing to design their own new
digital musical instruments as part of their quest
for new musical composition and performance
practices.”

换句话说, the creation of new forms of
control is perhaps a way to stretch the boundaries of
music itself. There can follow a temptation to see the
ideal instrument as one that allows as many degrees
of freedom as possible, to control the widest possible
artistic space. Tanaka (2000, p. 396), 然而,
observes the tendency to control ever-increasing
numbers of synthesis parameters, perhaps even
across multiple media, cautioning that “the danger
is in ending up not with a Gesamtkunstwerk,
but with a kind of theme park ‘one-man band’.”
Following Fuller-Maitland’s argument, the essential
value of a musical controller may also be in its
limitations as much as in its capabilities.

In closing, consider again the most famous
of continuous-control keyboard instruments, 这

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

clavichord. 柯克帕特里克 (1981, p. 295) 提供了一个
personal account of playing clavichord, talking not
only of learning to control vibrato, but in learning
when not to use it: “In later years, I was to use less
and less of the kind of vibrato that obtrudes itself
upon the attention of the hearer, reserving it only for
the most subtle colouration of sound, and in my later
recordings I eliminated it almost entirely.” Even as
new instruments change the way we interact with
声音, the keyboard is likely to remain with us.
There is much still to be developed and refined, 但
perhaps the greatest contributions will come not
from complex multidimensional controls, but from
new levels of nuance and subtlety.

致谢

Thanks to Jennifer MacRitchie, Bill Verplank, 和
Giulio Moro for feedback on earlier drafts of this
文章.

The section on Measuring and Mapping Contin-
uous Key Angle is adapted, with permission, 从
my paper presented at the 2013 International Con-
ference on New Interfaces for Musical Expression
(McPherson 2013b). Further implementation detail
can be found in the original.

参考

贝克, 右. 磷. 1990. Active Touch Keyboard. US Patent

4,899,631, filed 24 可能 1988 and issued 13 二月
1990.

Bernays, M。, 和C. 葡萄. 2012. “Piano Touch Analysis:
A MATLAB Toolbox for Extracting Performance De-
scriptors from High-Resolution Keyboard and Pedalling
Data.” In Actes des Journ ´ees d’Informatique Musicale
会议, PP. 55–64.

Bernays, M。, 和C. 葡萄. 2014. “Investigating Pianists’
Individuality in the Performance of Five Timbral
Nuances through Patterns of Articulation, Touch,
Dynamics, and Pedaling.” Frontiers in Psychology
5. Available online at www.ncbi.nlm.nih.gov/pmc
/articles/PMC3941302. Accessed November 2014.

Bloland, 磷. 2007. “The Electromagnetically-Prepared Piano
and Its Compositional Implications.” In Proceedings
of the International Computer Music Conference, PP.
125–128.

Brent, 瓦. 2012. “The Gesturally-Extended Piano.” In

Proceedings of the International Conference on New
Interfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2012/nime2012 102.pdf.
Accessed November 2014.

Cadoz, C。, L. Lisowski, and J.-L. Florens. 1990. “A Modular
Feedback Keyboard Design.” Computer Music Journal
14(2):47–51.

Castellano, G。, 等人. 2008. “Automated Analysis of
Body Movement in Emotionally Expressive Piano
Performances.” Music Perception 26(2):103–120.
Dalla Bella, S。, 和C. 帕尔默. 2011. “Rate Effects on

Timing, Key Velocity, and Finger Kinematics in Piano
Performance.” PloS One 6(6):e20518. Available online
at journals.plos.org/plosone/article?id=10.1371/journal
.pone.0020518. Accessed November 2014.

Dolan, 乙. 我. 2008. “E.T.A. Hoffmann and the Ethereal

Technologies of ‘Nature Music’.” Eighteenth Century
音乐 5(01):7–26.

Dolan, 乙. 我. 2013. “Toward a Musicology of Inter-

faces.” Keyboard Perspectives V. 在线提供：
westfield.org/publications/kp5 (subscription required).
Accessed November 2014.

Eaton, J。, 和R. Moog. 2005. “Multiple-Touch-Sensitive
Keyboard.” In Proceedings of the International Con-
ference on New Interfaces for Musical Expression,
PP. 258–259.

Freed, A。, 和R. Avizienis. 2000. “A New Music Keyboard
Featuring Continuous Key-Position Sensing and High-
Speed Communication Options.” In Proceedings of the
International Computer Music Conference, PP. 515–
516.

Fuller-Maitland, J. A. 1920. “Of Defects in Musical
Instruments and Their Value.” Musical Quarterly
6:91–97.

Furuya, S。, 中号. Flanders, 和 J. F. Soechting. 2011. “Hand
Kinematics of Piano Playing.” Journal of Neurophysiol-
奥吉 106(6):2849–2864.

Furuya, S。, 和H. Kinoshita. 2008. “Organization of

the Upper Limb Movement for Piano Key-Depression
Differs between Expert Pianists and Novice Players.”
Experimental Brain Research 185(4):581–593.

Galpin, F. 瓦. 1937. “The Music of Electricity. A Sketch
of Its Origin and Development.” Proceedings of the
Musical Association 64:71–83.

Gillespie, 右. 乙. 1996. “Haptic Display of Systems with
Changing Kinematic Constraints: The Virtual Piano
Action.” PhD dissertation, 斯坦福大学, Depart-
ment of Mechanical Engineering.

Gillespie, 右. B., 等人. 2011. “Characterizing the Feel of the
Piano Action.” Computer Music Journal 35(1):43–57.

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

Gillian, N。, 和S. Nicolls. 2012. “A Gesturally Con-

trolled Improvisation System for Piano.” In Proceedings
of the International Conference on Live Interfaces:
Performance, Art, 音乐 (LiPAM). Available online
at www.nickgillian.com/archive/publications
/GillianLIPAM2012.pdf. Accessed November 2014.
Goebl, W., 右. Bresin, 和我. Fujinaga. 2014. “Perception
of Touch Quality in Piano Tones.” Journal of the
Acoustical Society of America 136(5):2839–2850.

Goebl, W., 右. Bresin, 和一个. Galembo. 2005. “Touch and
Temporal Behavior of Grand Piano Actions.” Journal of
the Acoustical Society of America 118:1154.

Kinoshita, H。, 等人. 2007. “Loudness Control in Pianists
as Exemplified in Keystroke Force Measurements on
Different Touches.” Journal of the Acoustical Society
of America 121(5):2959–2969.

柯克帕特里克, 右. 1981. “On Playing the Clavichord.” Early

音乐 9(3):293–306.

Kirn, 磷. 2012. “Endeavour’s Evo, Touch-Sensitive
Keyboard, Reimagined.” Create Ditigal Music.
Available online at createdigitalmusic.com/2012
/11/endeavours-evo-touch-sensitive-keyboard
-reimagined-now-from-eur499-gallery-videos. Accessed
十一月 2014.

Goebl, W., 和C. 帕尔默. 2008. “Tactile Feedback and

Lamb, R。, 和一个. 罗伯逊. 2011. “Seaboard: A New

Timing Accuracy in Piano Performance.” Experimental
脑研究 186(3):471–479.

Goebl, W., 和C. 帕尔默. 2013. “Temporal Control
and Hand Movement Efficiency in Skilled Music
Performance.” PloS One 8(1):e50901. Available online
at journals.plos.org/plosone/article?id=10.1371/journal
.pone.0050901. Accessed November 2014.

Grosshauser, T。, and G. Tr ¨oster. 2013. “Finger Position and
Pressure Sensing Techniques for Strings and Keyboard
Instruments.” In Proceedings of the International
Conference on New Interfaces for Musical Expression,
PP. 27–30.

Hadjakos, A. 2012. “Pianist Motion Capture with the
Kinect Depth Camera.” In Proceedings of the Sound
and Music Computing Conference, PP. 303–310.
Hadjakos, A。, 乙. Aitenbichler, 和M. M ¨uhlhauser.

2009. “Probabilistic Model of Pianists’ Arm Touch
Movements.” In Proceedings of the International
Conference on New Interfaces for Musical Expression,
PP. 7–12.

Hadjakos, A。, 和S. Waloschek. 2014. “ViP: Controlling
the Sound of a Piano with Wrist-Worn Inertial Sen-
sors.” Paper presented at the International Conference
on New Interfaces for Musical Expression, Keyboard
Salon Workshop, 30 June–4 July, Goldsmiths, 大学-
versity of London. Available online at www.zemfi.de
/papers/vip2014.pdf. Accessed November 2014.

Haken, L。, 乙. Tellman, 和P. 沃尔夫. 1998. “An Indiscrete
Music Keyboard.” Computer Music Journal 22(1):30–48.

哈特, H. C。, 中号. 瓦. Fuller, 和W. S. Lusby. 1934. “A

Precision Study of Piano Touch and Tone.” Journal of
the Acoustical Society of America 6(2):80–94.

Heinrichs, C。, 和一个. 麦克弗森. 2012. “A Hybrid

Keyboard-Guitar Interface Using Capacitive Touch
Sensing and Physical Modeling.” In Proceedings of the
Sound and Music Computing Conference, PP. 277–283.

Keislar, D. 1987. “History and Principles of Microtonal
Keyboards.” Computer Music Journal 11(1):18–28.

Piano Keyboard-Related Interface Combining Discrete
and Continuous Control.” In Proceedings of the
International Conference on New Interfaces for Musical
Expression, PP. 503–506.

Lozada, J。, 中号. Hafez, and X. Boutillon. 2007. “A Novel
Haptic Interface for Musical Keyboards.” In Proceed-
ings of the IEEE/ASME International Conference on
Advanced Intelligent Mechatronics, PP. 1–6.

MacRitchie, J。, and N. J. 贝利. 2013. “Efficient Tracking
of Pianists’ Finger Movements.” Journal of New Music
研究 42(1):79–95.

麦克弗森, A. 2010. “The Magnetic Resonator Piano:

Electronic Augmentation of an Acoustic Grand Piano.”
新音乐研究杂志 39(3):189–202.

麦克弗森, A. 2012. “TouchKeys: Capacitive Multi-
Touch Sensing on a Physical Keyboard.” In Pro-
ceedings of the International Conference on New
Interfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2012/nime2012 195.pdf.
Accessed November 2014.

麦克弗森, A. 2013A. “Crowdfunding Digital Musical
Instruments: A Case Study.” In Proceedings of the
Innovation in Music Conference. 在线提供：
inmusic13.innovationinmusic.com/proceedings.php
(subscription required). Accessed November 2014.
麦克弗森, A. 2013乙. “Portable Measurement and

Mapping of Continuous Piano Gesture.” In Proceedings
of the International Conference on New Interfaces for
Musical Expression, PP. 152–157.

麦克弗森, A。, A. Gierakowski, 和一个. Stark. 2013. “这
Space Between the Notes: Adding Expressive Pitch
Control to the Piano Keyboard.” In Proceedings of the
ACM Conference on Human Factors in Computing
系统, PP. 2195–2204.

麦克弗森, A。, 和 Y. Kim. 2010. “Augmenting the
Acoustic Piano with Electromagnetic String Actu-
ation and Continuous Key Position Sensing.” In
Proceedings of the International Conference on New

电脑音乐杂志

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

Interfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2010/nime2010 217.pdf.
Accessed November 2014.

麦克弗森, A。, 和 Y. Kim. 2011A. “Design and

Applications of a Multi-Touch Musical Keyboard.” In
Proceedings of the Sound and Music Computing Confer-
恩斯. Available online at smcnetwork.org/system/files
/smc2011 submission 80.pdf. Accessed November 2014.

麦克弗森, A。, 和 Y. Kim. 2011乙. “Multidimensional
Gesture Sensing at the Piano Keyboard.” In Proceed-
ings of the ACM Conference on Human Factors in
Computing Systems, PP. 2789–2798.

麦克弗森, A。, 和 Y. Kim. 2012. “The Problem of the
Second Performer: Building a Community around an
Augmented Instrument.” Computer Music Journal
34(4):10–27.

Medeiros, C. B., 和M. 中号. Wanderley. 2014. “A Com-
prehensive Review of Sensors and Instrumentation
Methods in Devices for Musical Expression.” Sensors
14(8):13556–13591.

梅特卡夫, C. D ., 等人. 2014. “Complex Hand Dexterity:
A Review of Biomechanical Methods for Measur-
ing Musical Performance.” Frontiers in Psychology
5(414). Available online at www.ncbi.nlm.nih.gov
/pmc/articles/PMC4026728. Accessed November 2014.
Miranda, 乙. R。, 和M. 中号. Wanderley. 2006. New Digital
Musical Instruments: Control and Interaction beyond
the Keyboard. Middleton, 威斯康星州: A-R Editions.
Moog, 右. A. 1987. “Position and Force Sensors and Their
Application to Keyboards and Related Control Devices.”
In Proceedings of the AES International Conference:
Music and Digital Technology, PP. 173–181.

Moog, 右. A。, 和T. L. Rhea. 1990. “Evolution of the
Keyboard Interface: The B ¨osendorfer 290 SE Record-
ing Piano and the Moog Multiply-Touch-Sensitive
Keyboards.” Computer Music Journal 14(2):52–60.
Oboe, 右. 2006. “A Multi-Instrument, Force-Feedback
Keyboard.” Computer Music Journal 30(3):38–52.

Oboe, R。, and G. De Poli. 2002. “Multi-Instrument Virtual
Keyboard: The MIKEY Project.” In Proceedings of
the International Conference on New Interfaces for
Musical Expression, PP. 1–6.

Ortmann, 氧. 1925. The Physical Basis of Piano Touch and

语气. 伦敦: Kegan Paul Trench, Tr ¨ubner.

Ortmann, 氧. 1929. The Physiological Mechanics of Piano

Technique. 伦敦: Kegan Paul Trench, Tr ¨ubner.
Paradiso, J. A. 1997. “Electronic Music: New Ways to

Play.” IEEE Spectrum 34(12):18–30.

Paradiso, J. A。, 和S. O’Modhrain. 2003. “Current Trends
in Electronic Music Interfaces. Guest Editors’ Introduc-
tion.” Journal of New Music Research 32(4):345–349.

Parlitz, D ., 时间. Peschel, and E. Altenm ¨uller. 1998. “As-

sessment of Dynamic Finger Forces in Pianists: Effects
of Training and Expertise.” Journal of Biomechanics
31(11):1063–1067.

Quartier, L。, 和T. Meurisse. 2014. “La “Touche

d’Intensit ´e” des Ondes Martenot: un contr ˆoleur physi-
ologiquement adapt ´e `a la main.” In Actes des Congr `es
Franc¸ ais d’Acoustique. Available online at www
.conforg.fr/cfa2014/cdrom/data/articles/000190.pdf.
Accessed November 2014.

Roads, C. 1996. “Early Electronic Music Instruments:
Time Line 1899–1950.” Computer Music Journal
20(3):20–23.

Robjohns, H. 2003. “Review: Hammond B3 Modelled
Electromechanical Tonewheel Organ.” Sound on
Sound, 七月. Available online at www.soundonsound
.com/sos/jul03/articles/hammondb3.asp. Accessed
十一月 2014.

Schoonderwaldt, E., 和M. Demoucron. 2009. “Extrac-
tion of Bowing Parameters from Violin Performance
Combining Motion Capture and Sensors.” Journal
of the Acoustical Society of America 126(5):2695–
2708.

Shear, G。, 和M. 赖特. 2012. “Further Developments

in the Electromagnetically Sustained Rhodes Piano.” In
Proceedings of the International Conference on New
Interfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2012/nime2012 284.pdf.
Accessed November 2014.

斯奈德, J。, 和一个. 麦克弗森. 2012. “The JD-1: 一个
Implementation of a Hybrid Keyboard/Sequencer
Controller for Analog Synthesizers.” In Proceed-
ings of the International Conference on New In-
terfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2012/nime2012 187.pdf.
Accessed November 2014.

Tanaka, A. 2000. “Musical Performance Practice on

Sensor-Based Instruments.” Trends in Gestural Control
of Music 13:389–405.

汤普森, M。, and G. Luck. 2012. “Exploring Re-
lationships between Pianists’ Body Movements,
Their Expressive Intentions, and Structural Ele-
ments of the Music.” Musicae Scientiae 16(1):19–
40.

Van Zandt-Escobar, A。, 乙. Caramiaux, 和一个. Tanaka.
2014. “PiaF: A Tool for Augmented Piano Perfor-
mance Using Gesture Variation Following.” In Pro-
ceedings of the International Conference on New
Interfaces for Musical Expression. 在线提供：
www.nime.org/proceedings/2014/nime2014 511.pdf.
Accessed November 2014.

麦克弗森

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
/
C
哦
米

j
/

我

A
r
t
我
C
e
–
p
d

F
/

3
9
2
2
8
1
8
5
6
1
5
2
/
C
哦
米
_
A
_
0
0
2
9
7
p
d

乙
y
G
你
e
s
t

哦
n
0
8
S
e
p
e
米
乙
e
r
2
0
2
3

Verplank, 乙. 2009. “Interaction Design Sketch-

book.” Available online at www.billverplank.com
/IxDSketchBook.pdf. Accessed November 2014.

Verplank, B., 中号. Gurevich, 和M. Mathews. 2002. “这
Plank: Designing a Simple Haptic Controller.” In
Proceedings of the International Conference on New
Interfaces for Musical Expression, PP. 177–180.

Weidenaar, 右. 1995. Magic Music from the Telharmonium.

拉纳姆, Maryland: Scarecrow Press.

Synthesizers.” In Proceedings of the International
电脑音乐会议, PP. 101–104.

哪个, Q., and G. Essl. 2012. “Augmented Piano Per-
formance Using a Depth Camera.” In Proceedings
of the International Conference on New Inter-
faces for Musical Expression. 在线提供：
www.nime.org/proceedings/2012/nime2012 203.pdf.
Accessed November 2014.