Vuk Vujovic** - Recherche en IA spécialisée au MIT

Vuk Vujovic**
ETH Zurich

Andre Rosendo*,†
The University of Cambridge

Luzius Brodbeck**
ETH Zurich

Fumiya Iida†
The University of Cambridge

Mots clés
Evolutionary robotics, developmental
robotics, evo-devo, genetic algorithms,
ontogeny, phylogenetic

Evolutionary Developmental
Robotics: Improving Morphology
and Control of Physical Robots

Abstract Evolutionary algorithms have previously been applied to
the design of morphology and control of robots. The design space for
such tasks can be very complex, which can prevent evolution from
efficiently discovering fit solutions. In this article we introduce an
evolutionary-developmental (evo-devo) experiment with real-world
robots. It allows robots to grow their leg size to simulate ontogenetic
morphological changes, and this is the first time that such an
experiment has been performed in the physical world. To test diverse
robot morphologies, robot legs of variable shapes were generated
during the evolutionary process and autonomously built using
additive fabrication. We present two cases with evo-devo experiments
and one with evolution, and we hypothesize that the addition of a
developmental stage can be used within robotics to improve
performance. De plus, our results show that a nonlinear system-
environment interaction exists, which explains the nontrivial
locomotion patterns observed. In the future, robots will be present in
our daily lives, and this work introduces for the first time physical
robots that evolve and grow while interacting with the environment.

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

1 Introduction

The design automation of artificial machines—physical or virtual—remains an interesting research
challenge. In the early nineties, Sims presented his work on the simulated evolution of virtual life
forms [27]. Further work explored the application of evolutionary algorithms in many artificial sce-
narios [24, 4]. The goal was to exploit the ability of evolutionary processes to optimize such complex
design problems, c'est, to improve the robotʼs performance by implementing a task-specific fitness
fonction. This way, both the control [12, 8] and the morphology [14] of the robot can be subject to
the design automation, and often both aspects evolve in parallel [25, 6].

When evolving real-world robotics, the challenge for evolutionary algorithms is to cope with
noise and disturbance from the physical world. Given the available computational power, most sys-
tems try to circumvent this problem with simulated environments, some of which are eventually
transferred to the real world [15, 17, 9]. Whereas Hornby et al. [17] propose a computer-simulated

* Contact author.
** Institute of Robotics and Intelligent Systems, ETH Zurich, Zurich, Suisse.
† Department of Engineering, Trumpington Street, The University of Cambridge, Cambridge, CB21PZ, ROYAUME-UNI. E-mail: andre.rosendo@
eng.cam.ac.uk (A.R.)

V. Vujovic et al.

Evolutionary Developmental Robotics

antenna that outperforms human-designed antennas in real life, Mouret et al. [9] present a series of
motions that confirm the physiological representation of a robot and the compatibility with its in-
ternal virtual model. Cependant, the transfer of simulated results into a real system is affected by a
phenomenon called the reality gap [18]. A priori, a simulation can never exactly model the behavior of
a physical robot in its environment, due to stochastic disturbances and oversimplification of ground
friction. Ainsi, a solution that performs well in simulation is not necessarily good when implemented
on a physical system. So as to overcome this limitation, several approaches have been presented:
Koos et al. [21] co-optimized the transferability of evolved solutions; Floreano et al. evolved a hom-
ing navigator by lifting design constraints, which define the interaction with the environment [11].
De la même manière, several other works have demonstrated the evolution of robot controllers in the real world
[12, 26] and how robots can even learn to update their internal morphological representation
through interactions with the environment [5]. In our previous publication we have shown that
the evolution of robot morphologies in the real world is also a possibility through an automated
assembly system [6].

The search space that the automated evolution has to cover can be very large, sometimes even
considered infinite. Given the limited speed at which robots can be evaluated in the real world, ce
can prove prohibitive. Biology, cependant, shows that plants and animals adapt their body structures
to the environment, not only over generations (phylogenetic) but also during their lifetime (onto-
genetic) [23]. Par exemple, plants adjust their leaf biomass to the lighting conditions [28], and frogs
have been reported to be capable of repeatedly changing their skin texture [13]. The field of devel-
opmental robotics aims to build robots that, throughout their lifetime, adapt to their environment
[22]; and both morphology and control are potential targets for development [19]. Though devel-
opment is usually approached by biologists [7, 16], computer scientists can also devise simulations to
study ontogenetic changes in virtual organisms [4, 10]. As it has been shown that simulated mor-
phological changes can improve the evolution of virtual robots [3] and animats [20], we hypothesize
that such improvement can, in the form of a developmental stage, be combined with an evolutionary
stage and improve the fitness of real-world robots.

In this work we demonstrate a combination of evolutionary and developmental processes for the
automated design and generation of physical robots, as illustrated in Figure 1. On the evolutionary
time scale, a population of robots is evolved, as the best solutions from each generation are selected
and mutations and crossovers are used to perpetuate their genotypes. During its lifetime, each robot

Chiffre 1. (un) The evolutionary process and (b) the nested evolutionary and developmental process for robot morphol-
ogies. The two processes take place at different time scales, although the difference is not comparable to that in
biological processes. The implementation of the growth process is decisive for the success of our approach.

170

Artificial Life Volume 23, Nombre 2

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

assumes three developmental stages, and at each stage some of its parts (par exemple., the leg length) grow
according to the robotʼs real-world interaction. We present experimental results that imply that the
introduction of a developmental process potentially improves the evolutionary process when an
appropriately chosen growth function is selected. Evolutionary processes improve robots in a phylo-
genetic approach, and this article studies for the first time a comparison between different devel-
opmental parameters for physical robots capable of changing their morphology over time.

In this article we apply evo-devo in order to develop robots able to walk over flat horizontal
espace, the fittest robots being those capable of traveling the longest distance within ten seconds.
An external robotic arm is employed to implement our evo-devo process in a physical system, comme
it is capable of autonomously fabricating and testing evolved legged robots. Contrasting with our
previous work [6], we increase the variety of shapes that such legged robots can assume by additively
fabricating soft legs. Such legs follow gene design parameters; they are built by the robotic arm on
the fly with hot glue, and are finally combined with other components to form the legged robots.

2 Materials and Methods

2.1 Evolution of Physical Robots
Evolutionary algorithms are optimization methods inspired by the evolutionary mechanism found in
nature and defined by Charles Darwin. While using genomes to define individual robots, these evo-
lutionary algorithms test behaviors and store the best performances to further create new robots
from combinations of the best ones.

The implementation of the evolutionary algorithm here consists of an evolutionary parameter
recherche, with an initial population of legged robots that evolves from one generation to the next.
In each generation, an internal search iterates over the whole set of individuals of the current gen-
eration to physically build new robots based on their encoded genotypes and experimentally evaluate
their fitnesses. The robots with higher fitness values are selected as the elite and are used to generate
the next generation of robots through crossover and mutation. The idea behind crossovers and mu-
tations is to balance the exploration of the design space and the exploitation of strong features of the
already tested robots. The mutation can randomly change each parameter in a robotʼs genotype with
probability 0.7, while the crossover function combines the two parent genomes through a randomly
decided one-point crossover.

Unlike other authors, here we use the terms “robots” and “robotics,” in a strict sense, to refer to
real-world agents, as opposed to virtual agents. De plus, the phenotype (c'est à dire., the observable traits)
is the physical representation in the legged robot, while the encoding of this potential solution de-
fines the genotype. The chosen fitness function is the distance traveled during ten seconds of lo-
comotion. Together with the robotʼs morphology, the control is coevolved during our evolutionary
optimization. All parameters considered during the evolution of the legged robot are detailed in
Chiffre 2.

2.2 Evolution and Development of Robots
Aiming to improve the evolutionary process and to reduce the complexity of the fitness landscape, un
developmental part is added to our second experiments to achieve an evo-devo process. The evo-
lutionary part, which has been previously defined, is extended by a nested developmental search—a
unidirectional search (only growth is allowed) of specified parameters. Within robotics the novelty of
this search is that the robotsʼ morphology changes in multiple stages through a growth function, et le
aim is to understand how ontogenetic morphological changes can affect robotics.

Figure 1b shows the flow chart of such a system, where the developmental stage is executed for
each individual of a population. The growth function determines how leg length changes from one
developmental stage to the next. In this article, two different growth functions are explored and
applied to two shape parameters of the legged robots.

Artificial Life Volume 23, Nombre 2

171

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 2. Pictogram with five different parameters used to evolve and develop the legged robots. L1 and L2 represent left
and right legs, respectivement. T is the leg thickness in layers, a is the leg angle in the horizontal plane, and Amplitude is the
amplitude of the control parameter for the rotation of legs during locomotion.

(1)

(2)

The first parameter that can be altered in the developmental process is the leg length L. Le
1 within its genome. For evo-devo processes,
1 and L 2
1 is half of the maximum L value. From stage n to

initial leg lengths are defined by the genes L 1
1 and L 2
specifically, the maximum value for L 1
n + 1 both legs are increased by DL, depending on the robotʼs fitness f n, as shown in
Ln þ1 ¼ Ln þ DL;

DL ¼ A exp kfð

Þ þ b:

This function is designed so that a larger physical growth is given to robots that do not travel long
distances (low fitnesses grow more), and we aim to simulate a self-stabilizing mechanism, as good
behaviors remain almost unchanged and badly performing robots are drastically altered. From an en-
ergy perspective, cependant, lower fitnesses intuitively represent little or no movement, and the energy
saved with the lack of locomotion can be used to boost the ontogenetic morphological change. Le
parameter b is the minimum growth of the leg, A is the amplitude of growth, and k is the coefficient
of exponential decrease of growth with respect to fitness. The actual parameter values for the evo-
devo experiments are listed in Table 1.

The minimum leg thickness adopted is manufactured with two layers of additive material, et
each layer is approximately 1 mm thick. While evolution experiments encode two, three, and four
layers as design possibilities, the developmental approach starts the first developmental stage with
two layers and adopts fixed increments of 1 mm. The limitation of the number of layers during
construction restricts the encoding of the thickness gene, and this prevents the system from
developing length and thickness in similar ways:

T 1 ¼ 2 mm;

T nþ1 ¼ T n þ DT ;

DT ¼ 1 mm:

(3)

(4)

(5)

The second parameter is the leg thickness T. A minimum thickness of 2 mm was chosen, comme
preliminary experiments hinted that thinner limbs could not support the weight of the robot. Upon

172

Artificial Life Volume 23, Nombre 2

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Tableau 1. Evo-devo algorithm parameters within the developmental stage.

Parameter name

Generations

Development stages

Length growth

Thickness growth

Value

Eq. 2

Eq. 5

Description

Number of generations evaluated

Number of developmental stages each individual experiences

Function defining length increments

Function defining thickness increments

12 mm

Amplitude of growth according to development function

−0.05

3 mm

Exponential descent rate of development function

Minimum leg growth during development

Length increments

3 mm

Minimum 45 mm, maximum 72 mm

Amplitude increments

Angle increments

5°

10°

Minimum 30°, Maximum 50°

Minimum 20°, Maximum 60°

reaching its last developmental stage, the fully grown legged robot is evaluated and the best genes
carried forward. The rationale behind this decision is to mimic a reproductive age; the best children
are not necessarily the best adults, and another adult would seek the best partner based on current
information, and not on its past performance.

2.3 Materials
The legged robots are automatically built by a robotic arm, which additively fabricates the robot legs
and assembles these legs with a motor module. The system is capable of running for 5 h without
human interference, at which point a user is required to add new raw materials to allow the con-
struction process to continue. The developmental growth, mentioned in the previous section, est
reproduced by incrementally replacing the previously glued legs with new, longer ones.

2.3.1 Robotic Assembly
The robotic arm (Universal Robots UR5) has six degrees of freedom and is equipped with two main
tools for this project, as shown in Figure 3. It contains a hot melt adhesive extruder, controlled via
the robot controller and capable of releasing a continuous stream of hot melt adhesive to unite cubes
or fabricate parts. The nozzle diameter of 0.2 mm allows for a thin enough glue thread for the
fabrication of precise parts. The material is released at 165°C and hardens within seconds.

A second tool is a pneumatic parallel gripper, and it is mounted next to the nozzle. It is powered by a
compressor and also controlled through a digital output of the robot controller. The gripper is used to
pick different-size objects, and it is therefore designed with stepped fingers such that the lower part of
the gripper can pick objects of roughly 6-cm width and the upper part can pick objects of 4-cm width.

2.3.1.1 Automated Fabrication of Leg Shapes An important improvement in this study
over our previous work [6] is the capability to generate glued parts in real time. The robot legs, comme
shown in Figure 3, can assume different morphologies depending on the specific parameter values.

Artificial Life Volume 23, Nombre 2

173

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 3. Hardware setup of the automatic fabrication system based on a UR5 robotic manipulator equipped with HMA
supplier and gripper. The legged robots are assembled from additively fabricated parts, active modules, and passive
cubes. After evaluation, which is observed by a webcam, the manipulator removes the legged robot from the test
bed and creates a new one.

A fused deposition modeling process [29] with hot melt adhesive material (ALFA Klebstoffe,
ALFA H 5500/30) is employed to build the robot legs. A 3D voxel grid is generated from the
encoded leg shape parameters, and this grid is converted to an STL file to be processed by a slicer
program. The output from the slicer is used to define the waypoints of the robot, which guides the
nozzle to deposit material and finalize the fabrication process.

The hot melt adhesive is provided by a melting tank (Robatech, Concept B12). The hot melt material is
used for additive fabrication of leg shapes and for the bonding assembly of individual elements of the legged
robots. A piston pump provides a pressure of 4 bar, which ensures a steady supply of hot melt adhesive.

2.3.2 Modules
The legged robots are assembled as a combination of active modules, passive modules, et
additively fabricated parts. The design of the modules is as follows.

2.3.2.1 Active Modules The modules are cube-shaped, and their cases are made from acry-
lonitrile butadiene styrene (ABS). A servo motor and diverse electronic components are embedded
in it, as an adaptation from previous modules [6] with improvements in both microcontroller soft-
ware and modularity of the hardware. This ensured that modules are decoupled from the controlling
system and are able to achieve complex behavior and control. Aussi, the modularity of the hardware
allowed for faster battery change and easy reprograming of the modules.

Chiffre 4 shows the internals of an active module. The module has a cubic shape with a side
length of 60 mm. It features a LiPo rechargeable battery that is easy to change, as it is stored in
a separate compartment of the module and is connected to the power switch via a single connector.
The batteries can power the module for about 4 à 6 h reliably. The module furthermore contains a
microcontroller (Arduino, Mini Pro), which runs the control algorithm for an oscillatory movement
of the servo motor (HS-82MG). This movement is defined by a PID control loop with an amplitude
that is inversely proportional to its frequency. Aussi, the module features a Bluetooth RN-42 for
communication with a Matlab-enabled desktop PC, which communicates with the module.

2.3.2.2 Passive Cubes These modules are simple wooden cubes used as handles to manipu-
late the additively fabricated parts. Their surface sticks to the glue from the fabricated parts and
ensures that it remains flat during pickup.

174

Artificial Life Volume 23, Nombre 2

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 4. Internal components of an active module. The left side of the module contains a LiPo rechargeable battery; le
center part houses electronics and a servo motor underneath, which is connected to the shaft. The right side includes
the main switch and the power distribution.

2.3.3 Testbed Setup
The following describes the test bed and the additional elements of the hardware setup surrounding
the test bed, which are a cleaning tool and the webcam system. The test bed itself is a 67 × 63-cm
wooden plate covered with a black paper sheet to improve contrast with the white module for easier
image analysis.

2.3.3.1 Cleaning Tool The cleaning tool is a device that the robotic arm uses to remove leg-
ged robots from the test bed once the experiment has finished, hence ensuring that all experiments
start with an empty test bed. The tool is designed to push the legged robots over the edge of the test
bed (from where they drop into a padded box). The cleaning tool is stored on one side of the test
bed on small holders with a magnetic system.

2.3.3.2 Webcam System The system of two Logitech webcam C930e cameras is used to
record the experiment and evaluate the fitness of the legged robots. One camera is used to observe
the distance traveled from the top. The fitness of the legged robots (c'est à dire., the distance traveled during
the evaluation phase) is automatically extracted from this cameraʼs footage using image analysis. Le
second camera records the behavior to foster a better understanding of locomotion strategies and is
thus placed at the side of the test bed.

2.4 Experimental Method
In juxtaposition with computer simulations, real-world experiments require raw materials, mainte-
nance of parts, et, more importantly, time to assemble robots. While results from similar works in a
simulation environment [27, 4, 14] can reach thousands of individuals, the physical construction of
our robots limited the total number of individuals. The authors considered the integration between
physical and virtual experiments to expedite the process, but the reality gap associated with the highly
deformable contact of glue with the floor rendered simulations useless.

Artificial Life Volume 23, Nombre 2

175

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Our experiments are separated into three different stages: the evolution experiment, evo-devo L,

and evo-devo LT, as defined below:

(cid:129) Evolution process without development. From each genotype, one physical robot is
generated and automatically tested to evaluate its fitness for the locomotion task (voir
Figure 1a). This experiment consisted of five generations, and the first generation was
initialized with ten randomly generated robot genomes. All following generations carry over
the saved genes of two elite agents of the preceding generations and use these to produce
ten new randomized agents, with four through crossover and six through mutations.

(cid:129)

In evo-devo L, the leg length L grows in three developmental stages of each robot, alors que
all other parameters are kept constant through development. The evolutionary process can,
cependant, change the initial leg length of the robots through mutation and crossover. After the
last stage of development, the fitness value for the adult robot is recorded. Ainsi, the leg length
of the finally evaluated robot differs from the encoded value in its genotype (enfant ), as it
was grown in the developmental process. Each of the five generations consists of five agents
that undergo three development stages, with a total of 15 fitnesses analyzed per generation.
(cid:129) Evo-devo LT is similar to the previous evo-devo case, but leg thickness is included as a
development parameter, and consequently excluded from the parameters to be modified
during evolution. Ici, the leg thickness always starts at the minimum value and grows
constant thickness increments over the life span (development), comme le montre l'équation 4.
The idea behind this developmental criterion is to mimic the ontogenetic growth from
biological forms.

A comparison between evolution and evo-devo cases is not the focus of this article, as the de-
velopmental search happens over a smaller search space, which would render such comparison un-
fair. En plus, the idea behind development requires the emulation of ever-changing
morphologies, where the infant morphology is selected based on the adult behavior, and such a
process is deemed to produce different results from those observed in the evolution case.

Both evo-devo cases use the same initial seed, and these five initial genomes were randomly taken
from the initial pool adopted for the evolution-only case. The high stochasticity from the contact
between the robot and the environment yielded different fitnesses for similar cases.

Enfin, we compare the gains associated with the developmental increments by creating a metric
called yf, which compares the difference between first/second and second/third fitnesses from de-
velopmental stages (henceforth called the first and second developments). The results of these com-
parisons are needed to assess in which conditions the developmental stage is improving the
performance within evo-devo cases. The equation is as follows:
yf ¼ f n þ 1 − f n;

(6)

where n represents the developmental stage.

The results for all developmental stages and generations were compared with a two-way ANOVA
test considering a 95% Intervalle de confiance, and a Tukey HSD was chosen as a post hoc analysis to
avoid a multiple-comparisons error.

3 Results and Discussion

3.1 Evolutionary Experiments
In Figure 5 the evolved robots are shown and the diversity between legged morphologies can be
seen. The evolution case explored a host of different leg designs and provided varied adaptations to
reach a higher fitness function. Although limited by real-world constraints, wide ranges of leg angles,
lengths, symmetries, and walking gaits have been explored during this experiment.

176

Artificial Life Volume 23, Nombre 2

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 5. The legged robots evolved in the first evolutionary experiment exhibit a large variety of leg morphologies.

The results of this experiment are shown in Figure 6, with a chronological comparison of generations
advancing horizontally. The data shows that the highest fitness value observed in this experiment is
reached in generation 2, where the legged robot walked almost 40 cm, and the remaining three gener-
ations do not yield higher displacements. During this evolutionary algorithm the genetic crossovers
and mutations explored variations of the best results (blue and red lines) to reach faster robots.

The complexity of the search space, characterized by the nonlinear relationship between length,
angle, and thickness, combined with the fitness evaluation in real-world experiments, presumably
caused the relatively low maximum fitness of this evolutionary experiment.

3.2 Evo-Devo Experiments
The influences of developmental processes were tested with two additional experiments. Within these
experiments we evaluated five generations with five agents each (two generated through crossover and

Chiffre 6. Evolution of robots over 5 generations. Each circle represents a single robot, its fitness values, and its gener-
ation group. The blue line on the top shows the maximum fitness achieved up to that generation, and the red line the
second-best value. Ainsi, these lines indicate the individuals selected to form the offspring in the subsequent generation.
Notably, the fitness function does not exceed 40 cm.

Artificial Life Volume 23, Nombre 2

177

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 7. Evo-devo of robots over five generations, with development of leg length only. Within each generation, le
results of all three developmental stages are shown, but only the last stage is considered for the selection. The blue line
indicates the fittest individual up to the current generation, the red line the second best. Although every case started
with a fitness value inferior to 40 cm (infancy), the developmental process allows the system to reach higher fitnesses
(adulthood).

three through mutation), and the results from the two elite agents from the previous generation were
carried over for further reproduction. Both evo-devo cases adopt the same initial genetic seed, and all
robots from these experiments go through three developmental stages, which result in a total of 75
evaluations per experiment. The best adult fitness is chosen, in contrast to the best lifelong fitness,
to mimic the partner selection process, which only takes place at reproductive age.

3.2.1 Developing Leg Length
The results of this evo-devo L experiment are shown in Figure 7. Là, the fitnesses of all three
developmental stages for all evaluated robots are shown. The results show that the fittest robot was
found in generation 3, but developmental processes throughout all five generations improved fit-
nesses after growth. It can be seen in Table 2 that the majority of these results lead to an improve-
ment of the robotʼs fitness, especially at the third generation.

Dans l'ensemble, the evo-devo L experiment presented solutions that improve with body growth, as fit-
nesses improved with the first and second developments (Tableau 2). As the morphological growth is
not embedded in the evolution, the evolutionary part is indirectly forced to choose infant states that
will yield better adults. The nested developmental process allowed an internal parameter search (leg
length) while freezing other parameters to achieve faster locomotion (higher absolute fitness).

Tableau 2. Success rate of L developmental stages. A successful growth is characterized by the association between
increases in fitness value and morphology.

Growth type

Stage to stage

Gen. 1

5/10

Gen. 2

7/10

Gen. 3

8/10

Gen. 4

7/10

Gen. 5

7/10

Start to end of development

4/5

5/5

3/5

4/5

178

Artificial Life Volume 23, Nombre 2

D
o
w
n
o
un
d
e
d

F
r
o
m
h

t
t

:
/
/

d
je
r
e
c
t
.

je
t
.

e
d
toi
un
r
t
je
/

un
r
t
je
c
e
–
p
d

F
/

2
3
2
1
6
9
1
6
6
6
5
0
4
un
r
t
je

_
un
_
0
0
2
2
8
p
d

b
oui
g
toi
e
s
t

o
n
0
7
S
e
p
e
m
b
e
r
2
0
2
3

V. Vujovic et al.

Evolutionary Developmental Robotics

Chiffre 8. Legged robots evolved with development of leg length and thickness. All three developmental stages of the
evolved robots are shown. The robots grow from the first to the second, and lastly to the third, stage.

3.2.2 Developing Leg Length and Thickness
The results for the evo-devo LT case demonstrated the effect of both leg length and thickness on
the fitness evaluation, as seen in Figure 8. Unlike the leg length, which was grown depending on the
fitness from an initial starting value (encoded in the robotʼs genome), the leg thickness was no longer
under evolutionary control, as it always started with two layers at the first developmental stage and
grew to four layers at the third stage for all robots.

The experiment was conducted with the same parameters from the evo-devo L experiment; thus
75 legged robots were evaluated in total (5 generations × 5 robots × 3 developmental stages). figue-
ure 9 shows the results of this experiment, with generations of individuals evolving and developing
with time. The analysis of these results shows that this choice of parameters for the developmental
process has a detrimental effect on the final fitness; the vast majority of individuals perform better
during their early age (infancy) than at the latest stages of growth (adulthood), as shown in Table 3.

3.2.3 Comparison between Developments
Comparing the developmental alternatives for the evo-devo cases, a statistical analysis shows that the
evo-devo L case is superior to the evo-devo LT case ( p < 0.0001), and, overall, the second devel- opmental stage is superior to the first stage (statistically significant with p = 0.0137), as seen in Figure 10. Moreover, while the evo-devo L had a clear tendency to improve the fitness value with developmental stages, the evo-devo LT had a strong tendency to degrade the performance (yf < 0) with growth. 3.2.4 Locomotion Behavior Throughout the above experiments, a range of different leg morphologies were evolved. Based on these diverse morphologies, a large range of locomotion behaviors emerged during the fitness eval- uation, and several distinct behaviors can be identified from the trajectories, as shown in Figure 11. As one might expect, both straight and curved trajectories were observed, but also more complex Artificial Life Volume 23, Number 2 179 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 V. Vujovic et al. Evolutionary Developmental Robotics Figure 9. Evo-devo of robots over five generations, with the development of leg length and leg thickness. The results of all three developmental stages are shown within each generation, but only the last stage is considered for the fitness selection. The blue line indicates the fittest individual up to the current generation, the red line the second best. Many adult individuals perform poorer than their younger states. Table 3. Success rate of LT developmental stages. Growth type Stage to stage Gen. 1 2/10 Gen. 2 3/10 Gen. 3 1/10 Gen. 4 2/10 Gen. 5 3/10 Start to end of development 0/5 1/5 0/5 1/5 2/5 behaviors such as S-shaped trajectories or U turns. Moreover, we could observe locomotion in two different directions: forward and sideways (with respect to the active moduleʼs orientation). The elasticity introduced by the glue legs might have caused such variability, and such diverse behavior fortifies our claims about the difficulty in simulating such results within a computer envi- ronment. The use of real-world data to create new robots is a strong trend for the future, either by the invention of robots capable of changing their own morphology and adapting their control to it, or by a real-world design optimization method to guarantee better robots as final output. 3.3 Discussion The results of these three experiments demonstrated the possibility of creating legged robots through evolution and, for the first time in robotics, through an evo-devo process that contains a nested developmental search. These results show that the evo-devo system is capable of finding solutions to improve the locomotion of growing robots, which is in agreement with the seminal review by Lungarella et al. of development in simulated environments [22]. A comparison between the two evo-devo experiments provides interesting questions for discussion. 180 Artificial Life Volume 23, Number 2 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 V. Vujovic et al. Evolutionary Developmental Robotics Figure 10. The means for both evo-devo cases. The red rhombus shows the overall average fitness of each case, and **** represents p < 0.0001, ** represents p < 0.01, and n.s. represents that no statistical significance was obtained. Evo-devo L outperformed evo-devo LT, and the developmental stages of evo-devo L resulted in positive fitness increments on average, in contrast with evo-devo LT, which resulted in negative fitness increments on average. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Figure 11. The trajectories of six evolved legged robots that were extracted from the overhead camera footage of the fitness evaluation. The trajectories reveal a broad range of locomotion behaviors such as curving forward or sideways (first row), straight locomotion and straight locomotion with curve at the end (middle row), and S-shape curve and U-turn trajectory (bottom row). Each individual is identified by its evo-devo case, generation, number, and developmental stage. Artificial Life Volume 23, Number 2 181 V. Vujovic et al. Evolutionary Developmental Robotics 3.3.1 Comparison between Evo-Devo Cases As Figure 10 shows, the increase in fitness values due to development is larger in the evo-devo L experiment than in the Evo-devo LT one. A lower maximum fitness was also observed in the latter, where leg length and thickness were grown during the development. This finding is surprising, as the developmental search was added to freeze certain angle and control parameters and further increase the fitness. The question that arises is how leg length L and thickness T are coupled and how they relate to the locomotion fitness. In Blickhan et al. [2] it is suggested that leg morphologies during locomotion can be simplified as springs, and in this representation we could infer that thicker legs would represent stiffer springs. Although intuitively we would accept the idea that longer legs in- variably produce faster strides, often normalized by the Froude number for size comparisons [1], the delicate interplay between length and thickness from this work would indicate a complex relation- ship, as demonstrated in Figure 12. There, the right and left leg lengths L1 and L2 similarly showed that evo-devo LT and evo-devo L have opposing behaviors: A positive correlation between leg length and fitness for one, as opposed to a negative correlation for the other. During the evo-devo LT case the developmental process forcefully leads to an increase of both leg length and thickness from one stage to another. We hypothesize that long and thick legs do not reap the locomotive benefits described in [2], and this explains the negative influence of a forceful increase of thickness during the developmental process, corroborated by Figure 10. l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Figure 12. Fitness values for different leg lengths L1 (a, c) and L2 (b, d) with fixed thickness T = 4. The plots in the top row (a, b) show the results for leg length development only. The second row (c, d) shows the results for the combined development of leg length and thickness. The observed trends were not true for thinner legs (T ≤ 3). 182 Artificial Life Volume 23, Number 2 V. Vujovic et al. Evolutionary Developmental Robotics 3.3.2 Developmental Parameter Selection While the benefits from evo-devo have been recognized, the influence of the developmental stage can be damaging if not properly set: The evo-devo LT cases were marked by detrimental growth changes (Figure 10), and we hypothesize that the search space reduction eventually removed the good choices that development could have made by enforcing an incompatible thickness T for each stage. In the past, Bongard has found a similar influence within computer simulations [3], and our results also show that “the right kind of morphological change accelerates the discovery of success- ful behaviors.” An analysis of walking animals [2] hints at a correlation between length and thickness, which is what the evo-devo LT cases adopted. Thicker legs have higher natural frequencies, while longer legs have lower frequencies, and a combined growth of these two might create a longer and thicker leg with a similar natural frequency. Although length and thickness were not proportionally coupled, the lower performance of evo-devo LT than evo-devo L leads us to believe that higher fitnesses are not associated with one specific natural frequency. The fitness values during the evolution case might have been hindered by the presence of too many parameters, making the genotype-to-fitness mapping overly complex. Koos et al. address a similar problem while closing the reality gap [21], as the correlation between such parameters can be highly nonlinear, and Jakobi et al. emphasize the importance of wise choice for such parameters [18]. Therefore, it remains open for discussion how a suitable growth function can be designed, and how to assign design parameters to the evolutionary or developmental processes. Our aim with this work was to offer a new perspective to adaptive robotics by introducing evo-devo processes with a greater emphasis on the devo aspect of it, which is fairly unexplored in real-world environments. Despite our efforts, much still has to be done to understand how both evo and devo parameters affect the final performance. 3.3.3 Evolution and Evo-Devo Considerations A direct comparison between the evolution and evo-devo cases is counterintuitive, as evo-devo tries to find the child that will become the best adult, while evolution is solely focused on the best individual overall. However, both developmental approaches surpassed the maximum fitness value of 40 cm within the first generations and, although not specifically searching for the best individual, showed themselves useful in forcing the system to explore specific traits associated with growth while keeping the other traits untouched, as suggested in [3]. Our nested developmental approach considered a local search over a set of parameters, and there might be a resemblance between this and memetic algorithms when the local improvement aspects are considered. Evolution approaches were not forced into a growth pattern, and the search space remained open for the entirety of the experiment, while the evo-devo experiments froze part of their search space during growth stages. Within the evo-devo experiments the best genome is chosen by choosing the best adult fitness instead of the best lifelong fitness. Our intention with this choice is to mimic the partner selection process in a sexual evolution, which only takes place at reproductive age; the effects of this choice will be further investigated in future experiments. Our experiments solely considered learning by a robot that was creating smaller-legged robots, not by the legged robot itself, and our next works will approach the possibility of legged robots adapting their control method to maximize their movement. Another future consideration will be of the consequences of the hereditary transfer of optimized control genes in these robots (i.e., Lamarckian evolution). 4 Conclusion Motivated by the way plants and animals adapted to their environments through evolution and by their adjustment to external influences on an ontogenetic time scale, this article presents results from the first robot in the world that combines evolutionary and developmental processes. It can Artificial Life Volume 23, Number 2 183 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 V. Vujovic et al. Evolutionary Developmental Robotics autonomously design, build, test, and improve the morphology and control of legged robots. In order to avoid the pitfalls of simulation, all solutions are physically tested during their evaluation, as many researchers from the same field struggle in addressing the reality gap [18, 21]. To enable a large variety of shapes, the legged robots are not only assembled from a set of predefined parts, but can also be additively fabricated in real time with different lengths and stiffnesses. The comparison between an evolutionary design and the evo-devo strategy, which combines a developmental search with the evolutionary design of locomotion agents, should be formally addressed in the future. However, our experiments indicate that the addition of an internal develop- ment search (evo-devo) resulted in adult legged robots with a higher fitness by freezing certain design parameters. Additionally, a comparison between the two evo-devo experiments with different develop- mental parameters reveals the importance of choosing the right parameters to evolve and to develop. The interplay between development and evolution during this process is complex and not yet fully understood. However, the desired improvement in fitness can only emerge if the evolutionary and developmental terms are properly adjusted, as seen within our experiments. Amongst the im- portant design choices are the selection of developmental parameters and their respective growth functions. Here, two parameters were considered—leg length and leg thickness—each with a spe- cific growth function. Leg length developed depending on the fitness of the agent in the preceding stage. Fit agents achieved little growth (only minor changes to well-working robots), while slow robots achieved a higher growth of leg length. On the other hand, the leg thickness was increased according to a fixed plan, adding one layer from one stage to the next. The development of the leg length achieved the best results, while the combination with the development of leg length and leg thickness did not render strong adult robots. However, more data is required to assess whether this drop in performance is induced by too many parameters within development, where the genotype-to-fitness mapping would be too complex, or whether a different choice of developmental function could have improved performance. Thus, it remains for future work to derive the appropriate design rules and guidelines, such that the evo-devo process can be successfully applied to more complex robot design problems. Acknowledgments This research was supported by the RoboSoft–Coordination Action for Soft Robotics, funded by the European Commission under the Future and Emerging Technologies (FP7-ICT-2013-C project 619319). References 1. Alexander, R. McN. (2013). Principles of animal locomotion. Princeton, NJ: Princeton University Press. 2. Blickhan, R., Seyfarth, A., Geyer, H., Grimmer, S., Wagner, H., & Gunther, M. (2007). Intelligence by mechanics. Philosophical Transactions of the Royal Society A, 365, 199–220. 3. Bongard, J. C. (2011). Morphological change in machines accelerates the evolution of robust behavior. Proceedings of the National Academy of Sciences of the U.S.A., 108(4), 1234–1239. 4. Bongard, J. C. (2013). Evolutionary robotics. Communications of the ACM, 56(8), 74–83. 5. Bongard, J. C., Zykov, V., & Lipson, H. (2006). Resilient machines through continuous self-modeling. Science, 314(5802), 1118–1121. 6. Brodbeck, L., Hauser, S., & Iida, F. (2015). Morphological evolution of physical robots through model-free phenotype development. PLoS ONE, 10(6), e0128444. 7. Carroll, S. B. (2008). Evo-devo and an expanding evolutionary synthesis: A genetic theory of morphological evolution. Cell, 134(1), 25–36. 8. Clune, J., Beckmann, B. E., Ofria, C., & Pennock, R. T. (2009). Evolving coordinated quadruped gaits with the hyperNEAT generative encoding. In A. Tyrrell (Ed.), IEEE Congress on Evolutionary Computation (pp. 2764–2771). Piscataway, NJ: IEEE Press. 9. Cully, A., Clune, J., Tarapore, D., & Mouret, J. B. (2015). Robots that can adapt like animals. Nature, 521(7553), 503–507. 184 Artificial Life Volume 23, Number 2 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 V. Vujovic et al. Evolutionary Developmental Robotics 10. Doursat, R., & Sánchez, C. (2014). Growing fine-grained multicellular robots. Soft Robotics, 1(2), 110–121. 11. Floreano, D., & Mondada, F. (1996). Evolution of homing navigation in a real mobile robot. IEEE Transactions on Systems, Man, and Cybernetics, 26(3), 396–407. 12. Gross, R., Bonani, M., Mondada, F., & Dorigo, M. (2006). Autonomous self-assembly in swarmbots. IEEE Transactions on Robotics, 22(6), 1115–1130. 13. Guayasamin, J. M., Krynak, T., Krynak, K., Culebras, J., & Hutter, C. R. (2015). Phenotypic plasticity raises questions for taxonomically important traits: A remarkable new Andean rainfrog (pristimantis) with the ability to change skin texture. Zoological Journal of the Linnean Society, 173(4), 913–928. 14. Hiller, J. D., & Lipson, H. (2010). Evolving amorphous robots. In H. Fellermann, M. Dorr, M. M. Hanczy, L. L. Laursen, S. Maurer, D. Merkle, P.-A. Monnard, K. Stoy, & S. Rasmussen (Eds.), Proceedings of the Twelfth International Conference on Artificial Life (ALIFE XII). Cambridge, MA: MIT Press. 15. Hiller, J. D., & Lipson, H. (2012). Automatic design and manufacture of soft robots. IEEE Transactions on Robotics, 28(2), 457–466. 16. Hoekstra, H. E., & Coyne, J. A. (2007). The locus of evolution: Evo devo and the genetics of adaptation. Evolution, 61(5), 995–1016. 17. Hornby, G., Lohn, J., & Linden, D. (2011). Computer-automated evolution of an X-band antenna for NASAʼs Space Technology 5 mission. Evolutionary Computation, 19(1), 1–23. 18. Jakobi, N., Husbands, P., & Harvey, I. (1995). Noise and the reality gap: The use of simulation in evolutionary robotics. In F. Morán, A. Moreno, J. J. Merelo, & P. Chacón (Eds.), Advances in Artificial Life. ECAL 1995. Berlin: Springer. 19. Jin, Y., & Meng, Y. (2011). Morphogenetic robotics: An emerging new field in developmental robotics. IEEE Transactions on Systems, Man, and Cybernetics A, 41(2), 145–160. 20. Joachimczak, M., Suzuki, R., & Arita, T. (2014). Fine grained artificial development for body-controller coevolution of soft-bodied animats. In H. Sayama, J. Rieffel, S. Risi, R. Doursat, & H. Lipson (Eds.), Proceedings of the International Conference on Synthesis and Simulation of Living Systems (pp. 239–246). Cambridge, MA: MIT Press. 21. Koos, S., Mouret, J.-B., & Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1), 122–145. 22. Lungarella, M., Metta, G., Pfeifer, R., & Sandini, G. (2003). Developmental robotics: A survey. Connection Science, 15(4), 151–190. 23. Moczek, A. P., Sultan, S., Foster, S., Ledón-Rettig, C., Dworkin, I., Nijhout, H. F., Abouheif, E., & Pfennig, D. W. (2011). The role of developmental plasticity in evolutionary innovation. Proceedings of the Royal Society B, 278(1719), 2705–2713. 24. Nolfi, S., & Floreano, D. (2000). Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines. Cambridge, MA: MIT Press. 25. Rieffel, J., Knox, D., Smith, S., & Trimmer, B. (2014). Growing and evolving soft robots. Artificial Life, 20(1), 143–162. 26. Rieffel, J., & Sayles, D. (2010). EvoFab: A fully embodied evolutionary fabricator. In G. Tempesti, A. M. Tyrrell, & J. F. Miller (Eds.), Evolvable systems: From biology to hardware. ICES 2010. Berlin: Springer. 27. Sims, K. (1994). Evolving 3D morphology and behavior by competition. Artificial Life, 1(4), 353–372. 28. Sultan, S. E. (2000). Phenotypic plasticity for plant development, function and life history. Trends in Plant Science, 5(12), 537–542. 29. Wang, L., Brodbeck, L., & Iida, F. (2014). Mechanics and energetics in tool manufacture and use: A synthetic approach. Journal of the Royal Society Interface, 11(100). Artificial Life Volume 23, Number 2 185 l D o w n o a d e d f r o m h t t p : / / d i r e c t . m i t . e d u a r t l / / l a r t i c e - p d f / / / / 2 3 2 1 6 9 1 6 6 6 5 0 4 a r t l / _ a _ 0 0 2 2 8 p d . f b y g u e s t t o n 0 7 S e p e m b e r 2 0 2 3 Vuk Vujovic** image

Télécharger le PDF