A Systematic Literature Review of the
Successors of “NeuroEvolution
of Augmenting Topologies”

Evgenia Papavasileiou
Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel,
Bruselas, B-1050, Bélgica
imec, Lovaina, B-3001, Bélgica

epapavas@etrovub.be

Jan Cornelis
Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel,
Bruselas, B-1050, Bélgica

jpcornel@etrovub.be

Bart Jansen
Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel,
Bruselas, B-1050, Bélgica
imec, Lovaina, B-3001, Bélgica

bjansen@etrovub.be

https://doi.org/10.1162/evco_a_00282

Abstracto

NeuroEvolution (NE) refers to a family of methods for optimizing Artificial Neural Net-
obras (ANNs) using Evolutionary Computation (EC) algoritmos. NeuroEvolution of
Augmenting Topologies (LIMPIO) is considered one of the most influential algorithms in
the field. Eighteen years after its invention, a plethora of methods have been proposed
that extend NEAT in different aspects. In this article, we present a systematic literature
revisar (SLR) to list and categorize the methods succeeding NEAT. Our review protocol
identificado 232 papers by merging the findings of two major electronic databases. AP-
plying criteria that determine the paper’s relevance and assess its quality, resulted in
61 methods that are presented in this article. Our review article proposes a new cate-
gorization scheme of NEAT’s successors into three clusters. NEAT-based methods are
categorized based on 1) whether they consider issues specific to the search space or the
fitness landscape, 2) whether they combine principles from NE and another domain, o
3) the particular properties of the evolved ANNs. The clustering supports researchers
1) understanding the current state of the art that will enable them, 2) exploring new
research directions or 3) benchmarking their proposed method to the state of the art,
if they are interested in comparing, y 4) positioning themselves in the domain or
5) selecting a method that is most appropriate for their problem.

Palabras clave

NeuroEvolution, genetic algorithms, artificial neural networks, topology evolution, en-
codificación, systematic literature review.

1

Introducción

NeuroEvolution (NE) is a learning method that uses Evolutionary Algorithms (EA) a
optimize the parameters of Artificial Neural Networks (ANNs) (Stanley et al., 2019).
In early neuroevolutionary methods, evolutionary optimization methods (p.ej., Genetic

Manuscrito recibido: 25 Marzo 2020; revisado: 5 Agosto 2020 y 23 Octubre 2020; aceptado: 27 Octubre 2020.
Computación evolutiva 29(1): 1–73
© 2020 Instituto de Tecnología de Massachusetts

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

Algorithms (GAs)) were used for learning the connection weights of fixed topology
ANNs (Whitley et al., 1990; Montana and Davis, 1989; Gomez and Miikkulainen, 1997;
Moriarty and Miikkulainen, 1996; Gomez and Miikkulainen, 1998). Sin embargo, since the
definition of a network’s topology has a major effect on its performance, new methods
appeared that optimize both the weights and the topology (Maniezzo, 1994; Liu and
Yao, 1996; Yao and Liu, 1998; Moriarty and Miikkulainen, 1995; Stanley and Miikku-
lainen, 2002b). These methods are known as Topological and Weight Evolving Artificial
Neural Networks (TWEANNs).

TWEANNs offer significant advantages over fixed topology ANNs, as finding the
optimal topology of an ANN requires time-consuming evaluations of potential archi-
tectures. Especially, in complex problems, the number of neurons and connections that
are required scales with the complexity of the problem (Manning and Walsh, 2012) y
thus the manual definition of the optimal topology is even more difficult. Además, el
topology defines the size of the search space: selecting a fixed topology smaller than
the optimal means that the search is performed in a lower dimensional space and thus
the optimal solution will not be found. Por otro lado, selecting a bigger topology
than the optimal one implies that the search is performed in an unnecessarily high di-
mensional space. In TWEANNs the problem of identifying the right ANN topology is
tackled by their ability to automatically discover the optimal architecture.

En 2002, a revolutionary TWEANN method called NeuroEvolution of Augmenting
Topologies (LIMPIO) was invented by Kenneth Stanley and Risto Miikulainen (Stanley
and Miikkulainen, 2002b). The method provides solutions to problems encountered in
NE by facilitating the crossover between individuals of different length, evolving net-
works by adding new structure when necessary, and protecting structural innovations
by organizing them in species.

Hoy, 18 años después, the NEAT algorithm is still relevant as the large number of
published papers extending NEAT or applying NEAT-based methods on challenging
problems shows. To our knowledge, no systematic literature review (SLR) of NEAT-
based successors exists. Literature review papers that present an overview of NE meth-
ods in general can be found (Yao, 1993, 1999; Floreano et al., 2008; Ding et al., 2013;
Risi and Togelius, 2015; D’Ambrosio et al., 2014; Stanley et al., 2019). Sin embargo, ninguno
of them uses a systematic review protocol. En cambio, they report on the most prominent
and important papers in the field. The reviews (Yao, 1993, 1999; Floreano et al., 2008) son
very detailed papers that cover the field of NE until 2008. More recent review papers
(p.ej., Ding et al., 2013; Risi and Togelius, 2015) existir, but their scope is different than the
current article, as Ding et al. (2013) describe basic theories and methods in NE and Risi
and Togelius (2015) discuss NE’s applications in games. A chapter in Growing Adaptive
Machines by D’Ambrosio et al. (2014) gives an overview of the methods that followed
HyperNEAT within five years. Finalmente, the most recent paper (Stanley et al., 2019) gives
an overview of different aspects of recent NE techniques and discusses their potential
for application in the field of deep learning.

In the current article, we present a new categorization of the NEAT successors based
on criteria defined in this article. This article can serve as a guidance manual that allows
researchers to propose a new NEAT-based method by starting from the most appropriate
baseline method or the most recent method or to select the method that is more appro-
priate to solve their problem.

Toward this purpose, detailed tables presenting a fine-grained categorization of the
methods according to three main criteria, as well as an illustration (ver figura 1 en
pag. 13) of the trajectory of the advances in NEAT-based methods are provided. With this

2

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

new categorization scheme researchers can better position their work in the landscape
of existing methods, or select an existing method that fits best the properties of their
problem’s landscape/search space, or that evolves ANNs with particular properties or
that combines principles from different research fields. Además, researchers who are
interested in benchmarking their method to the state of the art can consult the detailed
tables in the Appendix regarding the employed datasets and performance metrics. Todo
the NEAT successors identified by the review protocol are described in detail in the
Appendix of this article. Finalmente, we discuss the findings and the shortcomings of exist-
ing approaches, the popularity of NEAT-based algorithms nowadays, and their future
relevance in the era of deep learning.

The remainder of this article is organized as follows. Sección 2 describes the method-
ology engaged for conducting the SLR and Section 3 presents the answers to the main
research questions regarding the NEAT method. An extended list with detailed an-
swers to all the research questions for all the identified methods can be found in the
Apéndice. Sección 4 presents the proposed categorization clusters and detailed tables
with the findings of the review protocol. Finalmente, Secciones 5 y 6 contain the discussion
and conclusion parts of this article.

2 Método

With this review article we want to present an objective historical account of NEAT
successors. Toward this purpose, we raise the following question: “which methods are
based on and succeeded NEAT within a period of 15 años?” A systematic literature re-
view is a type of review that satisfies this objective (Bettany-Saltikov, 2010) as it allows
for the findings to be presented in an objective and independent summary (Hemming-
way and Brereton, 2009). Since a systematic review article is characterized by the devel-
opment and use of a review protocol, we followed published guidelines (Keele, 2007;
Kofod-Petersen, 2012) on “how to conduct a systematic literature review paper.” For
a comprehensive review on modern neuroevolution, not necessarily based on NEAT
methods, but focusing on current trends, readers are advised to read a paper by Stanley
et al. (2019).

Conducting a rigorous and repeatable SLR requires a review protocol (Bettany-
Saltikov, 2010). We followed the guidelines according to Keele (2007) and Kofod-
Petersen (2012). Primero, we defined the objective and the research questions that this review
was going to address. En segundo lugar, we designed the search strategy toward answering the
research questions by defining the search terms and the inclusion and exclusion criteria for
selecting the relevant literature. After a pilot study we refined both the search terms and
the selection criteria. In the following phase, we created a Quality Assessment check-list
to evaluate the selected literature. Finalmente, we determined a data extraction form inspired
by the one in Wen et al. (2012) for documenting the main features of each method.

2.1 Research Questions

The objective of this SLR is to summarize, categorize, and clarify NEAT-based NE meth-
probabilidades. These methods will be referred to as x-NEAT.1 Toward this objective, we raised the
following Research Questions (RQ):

1. RQ1: What are the main principles and features of each method? (Principles)

1The letter “x” is used to represent the various acronyms used in the names of NEAT’s successors.

Volumen de cálculo evolutivo 29, Número 1

3

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

No.

IC1
IC2

Mesa 1: Inclusion and exclusion criteria.

Criterion

The study describes an extension of NEAT, (x-NEAT)
The study describes a hybrid model, Por ejemplo, based on NEAT and another

evolutionary or machine learning algorithm

EC1
EC2

The study is only an application of an x-NEAT method to a dataset or a new domain
The study describes an EA that is not a successor of NEAT but inspired by the NEAT

principios (Sección 3.1)

EC3

The study compares a not-NEAT-based NE method with NEAT, or existing x-NEAT

methods among each other

EC4

The study is an older/conference version of a relevant journal paper

IC: Inclusion Criterion, EC: Exclusion Criterion.

2. RQ2: How are the ANNs encoded into the genome? (Encoding) The purpose
is to identify the different encoding schemes that are used for representing the
ANNs phenotypes in the GAs’ genome.

3. RQ3: How does the x-NEAT method perform compared to others? Which per-

formance metrics are used to make this comparison? (Actuación)

2.2

Search Strategy

Search Terms

2.2.1
In order to identify the relevant papers succeeding NEAT (Stanley and Miikkulainen,
2002b), we had to define the search terms by identifying the major terms and alternative
spellings and synonyms. We used the Boolean OR to combine synonyms and alternative
spellings and the Boolean AND to link the major terms. The search was performed on
el 18 Abril 2018 and we chose to include all the papers published until the end of
2017. After refinement, the resulting search term is the following ((neuro?evolution OR
neuroevolution* OR evolv* neural networks) Y (*neat OR augment* topologies))
Y (EXCLUDE (PUBYEAR, 2018)).

Literature Resources

2.2.2
With the search term, we searched for published journal and conference papers in the
electronic databases of Web of Science (WoS) (all databases) y Scopus. The search
resulted in 225 publications from all the databases of WoS and 121 publications from
Scopus. We searched each database separately and gathered the papers together. Ex-
cluding the common papers between the two databases resulted in 232 publicaciones
which had to be evaluated on their relevance and quality.

Study Selection

2.2.3
To select the relevant papers we read all the abstracts and we applied the inclusion (IC)
and exclusion (EC) criteria defined in Table 1. Forty papers were selected based on IC1,
25 based on IC2, y 5 based on both IC1 and IC2. Sixty-two papers were excluded
because of EC1, 7 because of EC2, 42 based on EC3, 39 based on EC4, 1 based on EC1
and EC3, y 1 based on EC2 and EC3.

Study Quality Assessment

2.2.4
The quality of each relevant paper was evaluated according to the Quality Assessment
Criteria (QAC) defined in the format of questions and presented in Table 2. If a question

4

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

Mesa 2: Quality assessment questions with possible answers: “Yes,” “Partly,” or “No.”

QA1
QA2
QA3

QA4
QA5

QA6

QA7
QA8
QA9

QA questions concerning the x-NEAT’s principles

Are the aims of the research clearly defined?
Are the main aspects of the proposed method explained in detail?
If the method introduces a new encoding scheme, is it described clearly? If the same
encoding as in previous methods is used, is it understandable from the paper?

QA questions regarding the experimental procedure

Is the experimental procedure clearly described?
Is the method evaluated on sufficient number of datasets? (number of datasets ≥3: Sí,

2: partly, 1: No)

If the study involves a custom artificial dataset, is its construction method adequately
descrito? If it cannot be described for example, in case of a video game, is the task
clearly explained?

Are the parameters of the NE algorithm clearly described?
Are the metrics used for measuring the algorithm’s performance clearly defined?
Is each experiment run for an adequate number of repetitions? (Sí: ≥20, Partly:

[10,20), No: [0,10))

QA10

Is there a statistical test to test if a statistical difference in the compared performances

exists?

QA11
QA12

Is the proposed method compared to the state of the art of NE methods?
Is the proposed method compared to other machine learning/EC algorithms?
QA questions regarding the reception of the paper from the community

QA13 Does the study have an adequate number of citations per year?2 (Sí: ≥1, Partly:

[0.5,1), No: [0,0.5))

was addressed by the paper, it could take one of the answers: “yes,” “partly,” and “no,"
which received one, one-half, and zero points, respectivamente. If the question was not ad-
dressed at all by the paper, then it received the answer “non-applicable” excluding it
from the calculation of the final result. In this way, a paper’s final score was calculated
by averaging the score of each question. Nine papers had a score ∈ [0, 0.5] y ellos
were excluded from the study.

2.2.5 Data Extraction
Application of the IC, EC, and the QAC resulted in 61 papers whose acronyms and
obtained scores are presented in Table 4. A partir de estos, we collected the necessary data to
answer the RQs. To keep the most important information from each paper, we created
cards as shown in Table 3 similar to Wen et al. (2012).

3 NEAT—Answers to the Research Questions

NeuroEvolution of Augmenting Topologies (LIMPIO) (Stanley and Miikkulainen, 2002b)
is a TWEANN method that enables the learning of the structure of ANNs at the same
time it optimizes their connectivity weights. When NEAT was proposed in 2002, it pro-
vided solutions to important research questions of that time. The proficiency of the
method is attributed to the three main innovations for which ablation studies showed

2The number of citations per year (the last quality assessment criterion (QA13)) is calculated by
taking the average number of citations of the concerning paper by both the Web of Science and Scopus
and dividing by the number of years after the paper’s publication until 2017.

Volumen de cálculo evolutivo 29, Número 1

5

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

Mesa 3: Data extraction card.

Data Item

Value

Data extractor’s name and date
Data checker’s name and date
Study ID / Article type
Publication year
Names of the authors of the study
Article title
Name of the x-NEAT method
Characteristics of the version
Encoding scheme
Performance metrics used

that each of the introduced components is crucial to the NEAT’s performance. The three
main features of NEAT are described in Section 3.1.

3.1 Research Question 1: Principles

Historical Markings to Deal with the Competing Conventions Problem. One crucial
issue in NE is the Competing Conventions Problem, also known as the Permutation
Problema (Radcliffe, 1993) that appears when there is more than one way to represent an
ANN and because structures evolve independently in different networks. En particular,
the problem occurs during the crossover of two genomes that are encoded differently
even though they represent the same solution. Como consecuencia, the resulting offspring
might suffer from loss of functionality. To overcome this problem, the system should be
able to identify identical structures. NEAT deals with this issue by means of historical
markings, eso es, by assigning an increasing innovation number when a new gene is
added to the genome. A new gene is introduced in the system by structural mutations
that add a new connection or a new node in the network. The historical markings act
as chronological indicators that facilitate crossover by identifying homologous sections
between different networks. The innovation number of each gene is inherited by the
offspring, facilitating the retaining of its historical origin throughout evolution.

Speciation to Protect Innovation. NEAT protects topological innovations through spe-
ciation in order to give time to new structures to optimize. The individuals compete
within their own niche instead of the entire population. En general, when a new con-
nection is added, a random weight is assigned to it. In order to converge to its opti-
mal value, a number of generations is required. Without speciation, the new individual
would have to compete with the entire population. In that case, there is a high proba-
bility that the individual would be replaced before it gets optimized, because of its poor
performance compared with the other already more optimized networks. En el otro
mano, with speciation, the network competes within its own niche, so it is given time
to optimize its weights before having to compete with the entire population. La red-
works are grouped in species based on their topological similarities. This is defined by
aligning the genomes based on the historical markings and determining the matching,
disjoint and excess genes among the individuals. The nonmatching genes between the
two individuals that are located in the middle of the genomes are called disjoint genes,
while the nonmatching genes in the end of the genomes are called excess genes. Este

6

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

topological similarity is calculated through a measure called compatibility distance, como

δ = c1E
norte

+ c2D
norte

+ c3W ,

where E is the number of excess genes, D the number of disjoint genes, W the average
weight differences of matching genes, N the number of genes in the larger genome, y
c1, c2, c3 the coefficients which define the importance of the three factors. In practice the
division with the number of genes N is omitted and N is set to 1.

Biasing the Search toward Smaller Structures. NEAT starts the evolution with a uni-
form population of minimal structures, eso es, fully connected networks with no hid-
den nodes and it evolves more complex networks (complexification) by introducing new
nodes and connections through structural mutations. These topological innovations are
maintained only if they are found to be useful after fitness evaluation, eso es, si ellos
can increase the network’s performance. In this way, NEAT tends to evolve smaller
estructuras.

3.2 Research Question 2: Encoding

NEAT uses direct encoding to encode the phenotypes (ANNs) in the genotype. In di-
rect genetic encoding, there is a one-to-one mapping between the networks’ phenotypes
and genotypes. One genome is used for encoding the nodes and a second genome for
encoding the connections. The networks’ nodes are encoded with a set of node genes
while the connections among the nodes are encoded with a set of connection genes. El
node genes have fields that indicate the type of the node, eso es, whether the node is
una entrada, an output, or a hidden node, whereas the connection genes indicate the con-
nections established between the nodes in the node genome. Each connection gene con-
sists of 5 campos: the id of the node receiving the connection (in-node), the id of the node
from where the connection begins (out-node), the weight of the connection, an enabled
bit that specifies whether the connection is enabled or not, and an innovation number
which allows finding corresponding genes during mating.

3.3 Research Question 3: Evaluation

NEAT is tested on the XOR problem and its performance is evaluated with the average
number of generations that are required to solve it. Además, NEAT is compared to four
other NE systems on the double pole balancing problem (a benchmark Reinforcement
Aprendiendo (rl) tarea), in terms of number of generations, their ability to generalize and
the size of the evolved networks. NEAT converges faster, but its ability to generalize is
not statistically different from other systems.

4

Findings and Categorization

En esta sección, we give a summary of the findings of the review protocol. El 61 meth-
ods that fulfill the inclusion criteria and pass the quality assessment threshold are pre-
sented in Table 4 along with the obtained scores. Detailed answers to the three research
questions for all the methods are presented in the Appendix.

4.1 Categorization Based on the Encoding

To begin with, en mesa 5, we present a categorization of NEAT’s successors based on
the employed encoding scheme, eso es, the way the mapping between GA’s genotype

Volumen de cálculo evolutivo 29, Número 1

7

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

.
s
mi
i
d
tu
t
s
d
mi
t
C
mi
yo
mi
s

F
oh
s
mi
r
oh
C
s
y
t
i
yo
a
tu
q

:
4

mi
yo
b
a
t

mi
r
oh
C
S

F
mi
R

y
d
tu
t
S

mi
r
oh
C
S

F
mi
R

y
d
tu
t
S

5
6
.
0

2
9
.
0

1
8
.
0

5
8
.
0

5
8
.
0

8
5
.
0

8
8
.
0

2
6
.
0

8
8
.
0

7
7
.
0

5
8
.
0

9
6
.
0

2
6
.
0

7
7
.
0

5
8
.
0

1
8
.
0

2
9
.
0

2
9
.
0

3
7
.
0

3
7
.
0

3
7
.
0

3
8
.
0

1
8
.
0

9
7
.
0

5
7
.
0

8
5
.
0

3
6
.
0

0
9
.
0

8
8
.
0

8
5
.
0

)
3
1
0
2
,
s
a
k
t
i

METRO
d
norte
a
tu
oh
i
r
t
i

i

metro
d
i
z
t
a
h
C

(

)
2
1
0
2
,

norte
mi
norte
i
a
yo
tu
k
k
i
i

METRO
d
norte
a

yo

h
oh
k

(

)
3
1
0
2
,
.
yo
a

t
mi
norte
á
r
tu
D
–
oh
gramo
mi
yo
yo
a
GRAMO

(

)
3
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a
h
gramo
tu
PAG
(

)
3
1
0
2
,
.
yo
a

t
mi

mi
s
r
oh
METRO

(

)
2
1
0
2
,
.
yo
a
t
mi
norte
mi
d
norte
I
(

)
4
1
0
2
,
.
yo
a

t
mi

a
gramo
norte

i
z
i
tu
h

(

)
3
1
0
2
,
.
yo
a
t
mi
gramo
norte
a
W.

(

)
3
1
0
2
,
.
yo
a
t
mi
norte
mi
d
norte
I
(

)
3
1
0
2
,
.
yo
a
t
mi
norte
a
t
(

2
v
t
A
mi
norte
r
mi
pag
y
h
mi
v
i
t
pag
a
d
A

.
7
3

t
A
mi
norte
r
mi
pag
y
h
GRAMO
PAG
Ud.
S

–

.
5
3

t
A
mi
norte
r
mi
pag
y
h
–
S
S
METRO

.
6
3

t
A
mi
norte
–
PAG
A
norte
S

.
4
3

s
d
yo
mi
fi
t
A
mi
norte

.
3
3

s
d
yo
mi
i
F
t
A
mi
norte
–
S
norte
F

.
0
4

t
A
mi
norte
d
mi
r
mi
y
a
l

.
9
3

t
A
mi
norte
d
mi
s
a
h
PAG

.
1
4

t
C
C
–
t
A
mi
norte
r
mi
pag
y
h

.
2
4

R
A
mi
norte

.
8
3

)
b
2
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a

i
s
i
R

(

t
A
mi
norte
r
mi
pag
y
h
–
S
mi
mi
v
i
t
pag
a
d
A

.
2
3

)
5
1
0
2
,
s
s
mi
tu
gramo
r
a
h
d
norte
a

s
C
i
s
C
norte
a
b
r
mi
V

(

)
5
1
0
2
,
.
yo
a
t
mi

s
i
t
i

norte
mi
h
t
mi
METRO

(

–

t
A
mi
norte
norte
PAG
PAG
C
–
mi
F
–
S
norte

.
4
4

t
A
mi
norte
r
mi
pag
y
h
pag
mi
mi
D

.
5
4

)
4
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a

i
s
i
R

(

t
A
mi
norte
r
mi
pag
y
h

)
mi
v
i
t
pag
a
d
A

(
d
mi
d
mi
mi
S

.
3
4

)
6
1
0
2
,

norte
mi
norte
i
a
yo
tu
k
k
i
i

METRO
d
norte
a
yo
a
w
a
R

(

METRO

I
–

METRO
t
S
l
–
t
A
mi
norte

,

METRO
t
S
l
–
t
A
mi
norte

.
2
5

)
5
1
0
2
,
.
yo
a

t
mi
oh
z
yo
a
C
s
oh
l
(

)
5
1
0
2
,
.
yo
a

t
mi
norte

i
mi
t
S
(

)
5
1
0
2
,
.
yo
a

t
mi
a
v
yo
i
S
(

)
6
1
0
2
,
.
yo
a

t
mi
a
v
yo
i
S
(

)
6
1
0
2
,
.
yo
a

t
mi
oh
ñ
a
metro
a
a
C

(

)
6
1
0
2
,
.
yo
a

t
mi

mi
r
oh
pag
a
r
a
t
(

norte
PAG
PAG
C
s
mi
t
i
yo

mi
–
PAG
A
METRO

.
1
5

t
A
mi
norte
–
S
F
PAG

.
7
4

norte
oh
mi
GRAMO
I
PAG

.
6
4

t
A
mi
norte
d
oh

.
8
4

2
v
t
A
mi
norte
d
oh

.
9
4

t
A
mi
norte

–
t

.
0
5

)
6
1
0
2
,

norte
mi
norte
i
a
yo
tu
k
k
i
i

METRO
d
norte
a
metro
tu
r
h
C
S
(

)
7
1
0
2
,
.
yo
a

t
mi
h
t
i

metro
S
–
k
C
i
w
d
r
a
h

(

)
7
1
0
2
,
.
yo
a

t
mi
oh
yo
yo

tu
z
r
a
METRO

(

)
6
1
0
2
,
.
yo
a
t
mi
metro
tu
r
h
C
S
(

)
7
1
0
2
,

norte
r
oh
D
d
norte
a
i
C
s
i
r
GRAMO

(

)
7
1
0
2
,
.
yo
a

t
mi
a
v
yo
i
S
(

)
a
7
1
0
2
,
yo
yo
mi
s
mi
D

(

)
7
1
0
2
,
.
yo
a

t
mi
gramo
gramo
a
h

(

)
7
1
0
2
,
.
yo
a
t
mi
gramo
norte
mi
PAG
(

t
A
mi
norte
r
mi
pag
y
h
–
B
METRO

.
4
5

l
t
–
t
A
mi
norte
–
S
PAG
l
PAG

.
5
5

t
A
mi
norte
r
mi
pag
y
h

–
t

.
7
5

t
A
mi
norte
oh
METRO
t

–

.
6
5

t
A
mi
norte
METRO
METRO

–

.
3
5

S
GRAMO
PAG
–
C
A
R
–
t
A
mi
norte

.
0
6

X
mi
l
F
–
t
A
mi
norte

.
1
6

t
A
mi
norte
A
h

–

.
9
5

t
C
A
X
mi

.
8
5

2
9
.
0

1
8
.
0

1
8
.
0

1
7
.
0

3
7
.
0

8
5
.
0

1
8
.
0

5
6
.
0

4
5
.
0

5
6
.
0

2
6
.
0

9
6
.
0

5
8
.
0

9
6
.
0

2
9
.
0

4
5
.
0

2
6
.
0

1
8
.
0

1
8
.
0

3
7
.
0

3
7
.
0

5
8
.
0

5
8
.
0

7
7
.
0

5
8
.
0

7
5
.
0

8
8
.
0

5
8
.
0

2
6
.
0

2
9
.
0

3
7
.
0

)
4
0
0
2
,

norte
mi
norte

i
a
yo
tu
k
k
i
i

METRO
d
norte
a
y
mi
yo
norte
a
t
S
(

t
A
mi
norte
y
r
a
norte
oh
i
t
tu
yo
oh
v
mi
oh
C

)
a
6
0
0
2
,
mi
norte
oh
t
S
d
norte
a
norte
oh
s
mi
t
i
h
W.

(

)
6
0
0
2
,
.
yo
a

t
mi
y
oh
r
norte
oh
METRO

(

)
6
0
0
2
,

norte
oh
oh
k
a
h
a
yo
A
d
norte
a
norte
mi
h
C

(

)
7
0
0
2
,
.
yo
a
t
mi

r
mi
gramo
norte
i
s
i
mi
R

(

)
8
0
0
2
,
.
yo
a

t
mi

yo
mi
tu
gramo
i
METRO

(

)
9
0
0
2
,
.
yo
a

t
mi
y
mi
yo
norte
a
t
S
(

)
7
0
0
2
,
.
yo
a

t
mi
oh
a
h
z

(

)
4
0
0
2
,
.
yo
a
t
mi

r
mi
gramo
norte
i
s
i
mi
R

(

)
5
0
0
2
,
.
yo
a
t
mi
norte
oh
s
mi
t
i
h
W.

(

)
5
0
0
2
,
.
yo
a

t
mi
y
mi
yo
norte
a
t
S
(

)
5
0
0
2
,
.
yo
a
t
mi

a
v
yo
i
S
'
D

(

)
6
0
0
2
,

y
mi
yo
norte
a
t
S
(

)
8
0
0
2
,

norte
mi
norte

i
a
yo
tu
k
k
i
i

METRO
d
norte
a

i
C
mi
ç
h
a
B
(

)
8
0
0
2
,

y
mi
yo

norte
a
t
S
d
norte
a
oh
i
s
oh
r
b
metro
A
D

'

(

)
9
0
0
2
,
.
yo
a

t
mi

s
gramo
norte
i
t
s
a
h

(

)
9
0
0
2
,
i
yo
yo
mi
metro
mi
GRAMO
d
norte
a
t
h
gramo
i
r

W.

(

)
9
0
0
2
,
.
yo
a

t
mi
norte
a
t
(

q
+
t
A
mi
norte

)
–
mi
norte

i
yo

norte
oh

(

–

t
A
mi
norte
A
C
PAG
A
l

t
A
mi
norte
norte
PAG
PAG
C

–

t
A
mi
norte
–
l

t
A
mi
norte
r
a
yo
tu
d
oh
METRO

t
A
mi
norte
–
S
F

t
A
mi
norte

t
r

2
v
t
A
mi
norte

t
r

.

1

.

2

.

3

.

4

.

5

.

6

.

7

.

8

.

9

t
A
mi
norte
r
mi
pag
y
h

t
norte
mi
gramo
a
i
t
yo
tu
METRO

.
5
1

norte
norte
R
t
C
–
t
A
mi
norte

.
2
1

t
A
mi
norte
r
mi
pag
y
h

.
3
1

–

t
A
mi
norte
norte
PAG
PAG
C
–
l
t

.
4
1

yo
mi
z
a
h
gramo
r
norte
norte

.

.
0
1

t
A
mi
norte
oh
k

–

.
1
1

mi
norte
A
S
–
l
R

.
7
1

t
A
mi
norte
D
F

–

.
8
1

t
A
mi
norte
gramo
C

.
6
1

)
9
0
0
2
,

norte
mi
norte
i
a
yo
tu
k
k
i
i

METRO
d
norte
a

yo
h
oh
k

(

t
A
mi
norte
–
mi
d
a
C
s
a
C
/
F
B
R

.
9
1

)
1
1
0
2
,

d
r
a
gramo
norte
oh
B
d
norte
a
h
C
a
b
r
mi
tu
A

(

t
A
mi
norte
norte
PAG
PAG
C

–

t
norte
mi
r
r
tu
C
mi
R

.
3
2

)
9
0
0
2
,

tu
h
C
d
norte
a
t
t
mi
gramo
gramo
a
h

(

)
0
1
0
2
,
.
yo
a

t
mi

mi
norte
oh
metro
a
d
r
a
C

(

)
0
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a

i
s
i
R

(

t
A
mi
norte
r
mi
pag
y
h
mi
v
i
t
pag
a
d
A

.
1
2

t
A
mi
norte

)
–
t
r
(

mi
norte

i
yo

norte
oh

.
2
2

t
A
mi
norte
oh
METRO

–

.
0
2

)
1
1
0
2

,
.
yo
a

t
mi
oh
i
s
oh
r
b
metro
A
D

'

(

2
v
t
A
mi
norte
r
mi
pag
y
h

t
norte
mi
gramo
a
i
t
yo
tu
METRO

.
4
2

)
1
1
0
2
,

y
mi
yo

norte
a
t
S
d
norte
a

s
C
i
s
C
norte
a
b
r
mi
V

(

oh
mi
l
–
t
A
mi
norte
r
mi
pag
y
h

.
5
2

)
a
1
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a
norte
a
metro
h
mi
l
(

)
2
1
0
2
,

h
s
yo
a
W.
d
norte
a
gramo
norte
i
norte
norte
a
METRO

(

)
1
1
0
2
,
.
yo
a

t
mi

mi
norte
tu
C

yo

(

)
2
1
0
2
,
.
yo
a

t
mi

t
h
gramo
i
r

W.

(

t
A
mi
norte
y
t
yo
mi
v
oh
norte

.
7
2

D
I
r
b
y
h
h
C
t
i

w
S

.
8
2

t
A
mi
norte
–
F
F
METRO

.
9
2

t
A
mi
norte
–
mi
S
F
I

.
6
2

)
a
2
1
0
2
,

y
mi
yo
norte
a
t
S
d
norte
a

i
s
i
R

(

)

oh
mi
l
–
(
t
A
mi
norte
r
mi
pag
y
h
–
S
mi

.
0
3

)
2
1
0
2

,

h
a
ˇc
r
k

(

t
A
mi
norte
norte
y
D

.
1
3

8

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

Mesa 5: Encoding used by each method.

Direct Encoding

(Stanley and Miikkulainen, 2002b, 2004; Reisinger et al., 2004; Whiteson

et al., 2005; Stanley et al., 2005; D’Silva et al., 2005; Stanley, 2006;
Whiteson and Stone, 2006a; Monroy et al., 2006; Chen and Alahakoon,
2006; Reisinger et al., 2007; Zhao et al., 2007; Miguel et al., 2008; Bahçeci
and Miikkulainen, 2008; Hastings et al., 2009; Wright and Gemelli, 2009;
Tan et al., 2009; Kohl and Miikkulainen, 2009; Haggett and Chu, 2009;
Cardamone et al., 2010; Auerbach and Bongard, 2011; Wright et al., 2012;
Lehman y Stanley, 2011a; Manning and Walsh, 2012; Krˇcah, 2012; Kohl
and Miikkulainen, 2012; Chatzidimitriou and Mitkas, 2013; Wang y cols.,
2013; Inden et al., 2013; Tan et al., 2013; Methenitis et al., 2015; Stein et al.,
2015; Loscalzo et al., 2015; Silva et al., 2015, 2016; Caamaño et al., 2016;
Rawal and Miikkulainen, 2016; Schrum and Miikkulainen, 2016;
Hardwick-Smith et al., 2017; Marzullo et al., 2017; Desell, 2017a; Hagg
et al., 2017; Peng et al., 2017; Grisci and Dorn, 2017)

(Stanley et al., 2009; D’Ambrosio and Stanley, 2008; Risi and Stanley, 2010;
Auerbach and Bongard, 2011; D’Ambrosio et al., 2011; Verbancsics and
Stanley, 2011; Risi and Stanley, 2012a, 2012b; Inden et al., 2012; Morse
et al., 2013; Pugh and Stanley, 2013; Gallego-Durán et al., 2013; Huizinga
et al., 2014; Risi and Stanley, 2014; Verbancsics and Harguess, 2015;
Tarapore et al., 2016; Schrum et al., 2016; Silva et al., 2017)

Indirect Encoding

Hybrid

(Clune et al., 2011)

and ANN’s phenotype happens. We identify three types of encoding: directo, indirect,
and hybrid.

Direct encoding refers to a one-to-one mapping between the genotype and the phe-
notype, such as NEAT’s node and connection genes described in Section 3.2. Sobre el
other hand, in indirect encoding a set of rules is used to map the genotype to the
phenotype. Compositional Pattern Producing Networks (CPPNs) (Stanley, 2007), de-
tails presented in the Appendix, are an example of indirect encoding (Stanley et al.,
2019). CPPNs, which behave as ANNs, can be evolved by NEAT to output a spatial
pattern in the hyperspace which corresponds to a connectivity pattern in a substrate of
nodos. Hybrid encoding refers to an alternation between direct and indirect encoding.
Analyzing how the ANNs are encoded in the genotype, we found that approxi-
mately two-thirds of the methods use direct encoding, while one method proposes a hy-
brid encoding. Most of the methods with direct encoding use NEAT’s encoding without
modifications. Sin embargo, there are methods that modify it by adding new fields accord-
ing to the characteristics of the proposed method. The methods using indirect encoding
either propose a new type of encoding or use HyperNEAT’s encoding. When necessary,
modifications on HyperNEAT’s encoding are made to meet the requirements of the pro-
posed method, Por ejemplo, by changing the initial topology of the CPPNs to include
more input, producción, or hidden nodes.

4.2

Proposed Categorization Scheme

Although NE methods can be traditionally categorized based on the encoding used
for mapping the genotype to the phenotype, as presented in Section 4.1, in this ar-
ticle we perform a more fine-grained classification. Based on three criteria presented
next, the NE methods are classified into three clusters, each of which consists of a

Volumen de cálculo evolutivo 29, Número 1

9

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

number of subclusters that result in 18 subclusters in total. We started the categorization
from subclusters that considered research questions we wanted to address and then we
regrouped them in a higher level of clusters to increase the readability and get a more
structured cluster description.

• Cluster 1: methods that consider issues relevant to the search space or the fitness
landscape. This cluster is about properties of the search space and the fitness
landscape and not about descriptive features of a certain method.

• Cluster 2: hybrid methods, eso es, methods that employ principles from NE and

another field of EC or Machine Learning (ML).

• Cluster 3: methods that evolve ANNs with particular properties.

4.2.1 Cluster 1: Issues Specific to the Search Space and the Landscape
NE methods are classified into the following subclusters based on which search
space/landscape issues they solve, which include:

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

• a problem/search space with multiple objectives. A multiobjective optimization
task is a domain where the simultaneous optimization of multiple conflicting
objectives is required (Branke et al., 2008).

• a search space characterized by many irrelevant or redundant features. A feature
is a measurable property or characteristic that is used to describe a task (obispo,
2006). In this way, the instances of a problem are represented as multidimen-
sional points in an n-dimensional space (where n is the number of features
describing the problem), which is difficult to search.

• a deceptive landscape. A landscape is deceptive when the population of the GA
is misguided away from the objective (Horn and Goldberg, 1995). En este caso,
a fitness function deceives the search by pointing the wrong way.

• a landscape characterized by uncertainty. An uncertain landscape is one where
the fitness functions’ optimum changes rapidly between generations (Krˇcah,
2012).

• a search space of an open-ended problem, es decir. a problem whose final solu-
tion is not finite (Stanley and Miikkulainen, 2004) and for which more com-
complejo (Maley, 1999) and novel (Standish, 2003) solutions are continuously
generated.

• a space where evolution takes place online or in real time. In online evolution
an ANN is evolved spontaneously without having prior information of the
whole environment or task. This is in contrast with offline evolution when all
the information is available from the beginning and the ANN is evolved before
its final application to the target task. In real-time evolution, a constraint of the
time for generating a solution exists. According to Cardamone et al. (2010),
online evolution focuses on maximizing an agent’s performance during the
whole learning process, whereas real-time evolution aims to find a group of
agents that perform well as fast as possible.

10

Volumen de cálculo evolutivo 29, Número 1

A Systematic Review of NEAT’s Successors

4.2.2 Cluster 2: Hybrid NE Methods
These methods combine NE principles with methods from the following fields:

• Evolutionary Computation (EC)

• Backpropagation (BP)

• Reinforcement Learning (rl)

• Unsupervised Learning (UL)

4.2.3 Cluster 3: Evolving ANNs with Particular Properties
NE methods that can evolve ANNs with particular characteristics belong to this cluster.
These properties include:

• modularity. A network is modular if it contains “highly connected clusters of
nodes that are sparsely connected to nodes in other clusters” (Clune et al.,
2013). A modular network consists of independent functional structures that
can be separately optimized (Kashtan and Alon, 2005), can operate on separate
inputs to perform a sub-task and are organized by an intermediary to produce
the network’s output (Azam, 2000).

• plasticity. An ANN is plastic when its connections’ weights do not remain static
during their lifetime but can change in response to the changing activation
levels in the neurons they connect (Risi and Stanley, 2010).

• transfer learning ability, eso es, transferring knowledge that is learned on one

task to another one (Bahçeci and Miikkulainen, 2008).

• automatic ANN substrate configuration. This refers to methods that evolve not
only the weights of a large scale ANN substrate, but also its topography, eso
es, the density and placement of neurons (Risi and Stanley, 2012a). Substrate is
a specific terminology introduced in HyperNEAT (Stanley et al., 2009). Para
detailed description please see Section A.13 in the Appendix.

• different types of nodes, eso es, ANNs with nodes that are different than the

sigmoid neuron units.

• large scale topologies, evolving large ANNs that define a high-dimensional
search space. In this category we also include ANNs with a large number of in-
put and output nodes to solve problems with high-dimensional input/output
espacio.

• deep architectures. We refer to methods optimizing parameters of deep ANNs

(DNNs).

• memory capacity. Augmenting an ANN with memory is essential for sequential

problems requiring long-term memory.

4.3

The x-NEAT Methods

The classification of the x-NEAT methods found by the review protocol is presented in
Mesa 6; only two methods, TMO-NEAT (Marzullo et al., 2017) and Cascade-NEAT (Kohl

Volumen de cálculo evolutivo 29, Número 1

11

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

.
s
d
oh
h
t
mi
metro
t
A
mi
norte
–
X
mi
h
t

F
oh
norte
oh
i
t
a
z
i
r
oh
gramo
mi
t
a
C
d
mi
s
oh
pag
oh
r
PAG

:
6

mi
yo
b
a
t

12

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 1: The trajectory of evolution of x-NEAT methods.

and Miikkulainen, 2009), could not be categorized into any of the proposed clusters. Él
should be noted that this table is not an exhaustive list of all the methods in the field,
just the ones that are returned by the inclusion and selection criteria of the protocol.
The table is preferably read column-wise: a single tick in a column does not at all mean

Volumen de cálculo evolutivo 29, Número 1

13

mi. Papavasileiou, j. Cornelis, y B. Jansen

that the indicated paper is the only paper addressing the category. In the Appendix,
we present detailed answers of the three RQs (principios, encoding, and evaluation) para
each method. Cifra 1 presents the trajectory of the x-NEAT methods’ evolution. Este
is a graphical illustration of how x-NEAT methods emerged from the original NEAT
método, eso es, which method is being extended by another.

4.4 Reference Tables for Benchmarking

Tables 8, 9, y 10 in the Appendix present the domains/datasets/tasks on which the x-
NEAT methods of this review article are tested and the used performance metrics. Cada
dataset has one or more references that correspond to the x-NEAT method that employs
él. En algunos casos, the original source of the dataset was available and this is included as
the last citation in the list of provided citations. These tables can help researchers to pick
up the method that is most suitable for their problem and find its detailed description
in the Appendix. También, it facilitates comparing and benchmarking new or existing NE
methods by choosing the dataset and the performance metric used in the state of the
arte.

5 Discusión, Open Issues, y perspectivas futuras

5.1 Why NEAT Inspired the Different Extensions

After the invention of NEAT, many methods have appeared that extend its function-
ality in various ways. We believe that the reason for this expansion is because NEAT
is an easily understandable algorithm that works well on difficult problems and many
researchers made its implementation publicly available. Además, we attribute this ex-
pansion to the fundamental importance of the NEAT principles, detailed in Section 3.
NEAT incorporates specific biological principles (Stanley and Miikkulainen, 2002b) eso
have contributed to its success. The evolutionary process in NEAT resembles the pro-
cess of natural evolution, as the evolved networks become more complex during their
optimization. As in NEAT, the genome in nature does not have a fixed length, but new
genes have been added during evolution through a process known as gene amplifi-
catión (Darnell and Doolittle, 1986; watson, 2004). Además, the solution that NEAT
provides to the competing conventions problem is also inspired by natural evolution
and in particular by the synapsis process, eso es, the process of lining up homologous
genes before crossover (Radding, 1982; Sigal and Alberts, 1972). Similarmente, the specia-
tion mechanism is inspired from nature’s speciation mechanism that groups together
individuals that share a common characteristic and implicitly protects innovation since
the different species compete within their own niche (Stanley and Miikkulainen, 2002b).
Although speciation as a notion was known in GAs, it was NEAT which introduced it in
the TWEANNs, since NEAT offered a solution to the competing conventions problem
with the use of historical markings. Before this solution, the definition of a compati-
bility metric, necessary for assigning individuals into species, was difficult to define.
As far as the notion of evolving minimal architectures is concerned, before NEAT there
were already attempts to control the size of the evolved topologies by including penal-
ties in the fitness function. Sin embargo, these attempts had the disadvantages of having
to predefine the penalty and risking to have a different performance from the original
non-penalized fitness function (Stanley and Miikkulainen, 2002b). By keeping the
topologies minimal throughout evolution, NEAT tends to minimize the search space
and to speed up the learning process (Stanley and Miikkulainen, 2002b). Before NEAT,
starting with a uniform minimal population was not possible. En cambio, TWEANNs

14

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

started the evolution with topologically diversified networks because without speci-
ation topological innovations could not survive (Stanley and Miikkulainen, 2002b). On
the other hand, speciation in NEAT enabled the evolution to start minimally and di-
versify along evolution, since topological innovations would be protected. In conclu-
sión, NEAT tackled problems known in the community in a unified and fundamentally
strong manner.

A clear indication of how fundamentally important and globally applicable the
NEAT principles are, can be found in the existence of a large number of papers whose
method is inspired by the NEAT principles in, Por ejemplo, NE with ontogeny (Inden,
2008), Genetic Programming (Drchal and Šnorek, 2013; Drchal and Snorek, 2012; Tru-
jillo et al., 2016), and molecular programming (Dinh et al., 2014). Neat-GP (Trujillo et al.,
2016) is an example of a method that adopts NEAT’s principles (minimal topology ini-
tialization, speciation with fitness sharing and crossover by identifying the matching
topologies) for evolving programs in Genetic Programming (médico de cabecera) for regression and
classification tasks. Neat-GP was able to achieve or improve the test fitness performance
of standard GP while evolving smaller programs and reducing the computational cost
and it shows that NEAT principles are fundamental and can be extended beyond neural
redes.

5.2

Findings of the Review and Recommendations

From the tables provided in the Appendix of this review article as well as from Table
7, we can conclude that comparing the performance of different x-NEAT methods is
difficult, as even methods belonging to the same subcluster are not all evaluated on the
same datasets and do not employ the same performance metrics.

Different Performance Metrics. The majority of the methods report on the algorithm’s
performance using the obtained fitness value. Similar metrics include the accuracy, el
average error and the absolute error. Most of the methods obtain the mean value of
these metrics calculated over the different algorithmic runs. Another metric is the ratio
of runs when the algorithm was successful in solving the problem. También, the majority
of the methods report metrics to describe the algorithm’s efficiency such as the number
of generations and the computational time. Metrics describing the size of the evolved
networks are also very common, Por ejemplo, the number of evolved nodes and con-
nections. Some methods are evaluated based on qualitative metrics, es decir. based on the
user’s interpretation of the evolved patterns and the agents’ evolved behavior or mor-
phology. Other papers evaluate the generalization ability of the proposed algorithm by
testing the best ANN on a different task with different initial conditions. Finalmente, métrica
to evaluate a specific characteristic of a method are proposed, such as the ratio between
relevant and irrelevant features or the sum of absolute weights for methods performing
feature selection, the number of agents that are dominated for coevolutionary methods
y otros.

Performance Comparisons. Even though a newly proposed method that brings new
insights in the field has a great value in its own, we observed that many research papers
also compare their methods to other ones using a wide set of datasets and performance
métrica. We believe that in order to facilitate comparisons, a good practice would be
to report performance on a minimal set of universal performance metrics. Acerca de
the algorithms’ performance, the accuracy, precisión, recordar, and confusion matrix for
classification tasks and the accumulated reward for identical reinforcement learning

Volumen de cálculo evolutivo 29, Número 1

15

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

Mesa 7: Table summarizing an x-NEAT method’s categorization to clusters, the method
it extends, and its comparisons to the state of the art (either to an existing x-NEAT al-
gorithm or to an algorithm from the fields of ML/NE/EC).

Cluster

Extends

x-NEAT

ML/NE/EC

Compared to

Método

RBF-NEAT

Different node types

LIMPIO

LIMPIO

SNAP-NEAT

Different node types

RBF-NEAT
Cascade-NEAT

NoveltyNEAT

Deceptive landscape NEAT
Open ended

LIMPIO
RBF-NEAT
Cascade-NEAT

LIMPIO
NEAT-CTRNN

DynNEAT

Fitness uncertainty

LIMPIO

LIMPIO

NEAT-LSTM-IM

Deceptive landscape NEAT-LSTM
UL
Different node types
Memory Capacity

NEAT-LSTM
NEAT-RNN

Coevolutionary NEAT Open ended

LIMPIO

Fixed Topology
Coevolutionary NEAT
Simplifying
Coevolutionary NEAT

NEAT-HOF

LAPCA-NEAT

Open ended

Open ended

Multiple objectives
Plastic NNs

Multiple objectives
Modular NNs

LIMPIO

LIMPIO

LIMPIO

LIMPIO

nnrg.hazel

MO-NEAT

MM-NEAT

FS-NEAT

FD-NEAT

Irrelevant features

LIMPIO

LIMPIO

Irrelevant features

LIMPIO

IFSE-NEAT

Irrelevant features

LIMPIO

Layered NEAT

Irrelevant features

LIMPIO

PFS-NEAT

Irrelevant features

LIMPIO

Phased NEAT

Irrelevant features

LIMPIO

LIMPIO
FS-NEAT

LIMPIO
FS-NEAT

LIMPIO
FD-NEAT

LIMPIO
FS-NEAT
FD-NEAT
SAFS-NEAT

LIMPIO
FD-NEAT

√

√

√

√

√

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

rtNEAT

rtNEATv2

Online NEAT+Q

Online/Real time

LIMPIO

Online/Real time

rtNEAT

rtNEAT

NEAT+Q

Online/Real time
EC
BP
rl

LIMPIO
NEAT+Q
softmax NEAT+Q
softmax NEAT

16

Volumen de cálculo evolutivo 29, Número 1

A Systematic Review of NEAT’s Successors

Método

KO-NEAT

Mesa 7: Continuado.

Cluster

Extends

x-NEAT

ML/NE/EC

Compared to

Online/Real time
EC
rl

LIMPIO

LIMPIO

cgNEAT

Online/Real time

Online NEAT

Online rtNEAT

Online/Real time
rl
NNs with Transfer Learning

Online/Real time
rl
NNs with Transfer Learning

LIMPIO

LIMPIO

rtNEAT

LIMPIO
rtNEAT
Online rtNEAT

LIMPIO
rtNEAT
Online NEAT

odNEAT

Online/Real time

LIMPIO

rtNEAT

odNEATv2

Online/Real time

odNEAT

odNEAT

HyperNEAT

Large Topologies

CPPN-NEAT

PNEAT

Switch HybrID

Large Topologies

NEAT+Q

NEAR

FNS-NEATFields

NS-FE-CPPN-NEAT

EC
BP
rl

EC
rl
Different node types
NNs with memory

Deceptive landscape
EC
Large Topologies

Deceptive landscape
EC
Different node types

PIGEON

L-NEAT

EC

BP

DeepHyperNEAT

BP
Deep NNs

EXACT

RL-SANE

NEAT-RAC-PGS

NEAT-FLEX

ES-HyperNEAT

BP
Different node types
Deep NNs

rl

rl

UL

Automatic Substrate
Configuration
Large Topologies

FT-NEAT
HyperNEAT

FT-NEAT
HyperNEAT

LIMPIO

LIMPIO

LIMPIO

LIMPIO

NEATFields

NEATFields

CPPN-NEAT NS-CPPN-NEAT

CPPN-NEAT

LIMPIO
PSO

LIMPIO

LIMPIO

LIMPIO

HyperNEAT HyperNEAT

LIMPIO

EXACTv1

BS-NEAT

RL-SANE

LIMPIO

LIMPIO

LIMPIO

HyperNEAT HyperNEAT

√

√

√

√

√

√

√

√

√

√

Volumen de cálculo evolutivo 29, Número 1

17

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

Método

Cluster

Extends

x-NEAT

ML/NE/EC

Mesa 7: Continuado.

Compared to

ES-HyperNEAT-LEO

Adaptado

ES-HyperNEAT

MSS-HyperNEAT

Modular NEAT

HyperNEAT-LEO

Automatic Substrate
Configuration
Large Topologies
Modular NNs

Automatic Substrate
Configuration
Large Topologies
Plastic NNs

Automatic Substrate
Configuration
Large Topologies

Modular NNs
Large Topologies

Modular NNs
Large Topologies

HyperNEAT-LEO
ES-HyperNEAT

HyperNEAT-

LEO

Adaptive HyperNEAT
ES-HyperNEAT

ES-HyperNEAT

HyperNEAT

HyperNEAT

LIMPIO

LIMPIO

HyperNEAT

HyperNEAT

MFF-NEAT

Modular NNs

LIMPIO

LIMPIO

HyperNEAT-CCT

MB-HyperNEAT

Modular NNs
Large Topologies

HyperNEAT-LEO

Modular NNs
Large Topologies

HyperNEAT
MM-NEAT

NEAT-CTRNN

τ -NEAT

τ -HyperNEAT

NEAT-LSTM

TL-CPPN-NEAT

NNs with memory
Different node types

LIMPIO

NNs with memory

LIMPIO

NNs with memory
Large topologies

τ -NEAT

HyperNEAT

NNs with memory
Different node types

LIMPIO

NEAT-RNN
NEAT-LSTM-IM

NNs with
Transfer Learning
Different node types

CPPN-NEAT

CPPN-NEAT

PLPS-NEAT-TL

NNs with
Transfer Learning

LIMPIO

Phased NEAT
GPS-NEAT
BS-NEAT

Adaptive HyperNEAT

Plastic NNs
Large Topologies

HyperNEAT

Adaptive HyperNEATv2 Plastic NNs

Adaptive HyperNEAT Adaptive

Large Topologies

Seeded Adaptive
HyperNEAT

Plastic NNs
Large Topologies

Adaptive HyperNEAT
Seeded HyperNEAT

HyperNEAT

Seeded

HyperNEAT

HyperNEAT

CPPN-NEAT

Different node types NEAT

Recurrent CPPN-NEAT

Different node types CPPN-NEAT

CPPN-NEAT

HyperNEAT
HyperNEAT-GS

Multiagent

HyperNEATv2

HyperNEAT

LIMPIO

LIMPIO

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

√

√

√

18

Volumen de cálculo evolutivo 29, Número 1

A Systematic Review of NEAT’s Successors

Método

HA-NEAT

SUPG-HyperNEAT

MAP-Elites CPPN

Mesa 7: Continuado.

Compared to

Cluster

Extends

x-NEAT

ML/NE/EC

Different node types NEAT

LIMPIO

Different node types
Large Topologies

Deceptive landscape
Different node types
EC

HyperNEAT

HyperNEAT

CPPN-NEAT

direct encoding
MAP-Elites

Multiagent HyperNEAT

Large Topologies

HyperNEAT

HyperNEAT

Multiagent

HyperNEATv2

Large Topologies

Multiagent

HyperNEAT

Multiagent

HyperNEAT

NEATFields

Large Topologies

HyperNEAT

HyperNEAT

Seeded HyperNEAT

Large Topologies

HyperNEAT

Seeded Adaptive
HyperNEAT

HyperNEAT

√

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

tasks should be reported and be accompanied by statistical tests to indicate whether
differences in performance are significant or not. In the papers reviewed in this study,
a statistical hypothesis test had been conducted in around 60% of the cases. Además,
although it is a computationally demanding task, another good practice would be to
evaluate the generalization ability of an algorithm on an independent test, either by
performing k-fold cross validation in supervised learning tasks or testing on different
versions of a reinforcement learning task and/or with different initial conditions. Re-
porting on the values of a fitness function might be valuable in specific cases but it
is not necessarily useful in all scenarios, since different definitions of fitness functions
can result in different outcomes. Regarding the efficiency of a method, the number of
generations until convergence or until the best solution is found (maximal or mini-
mal fitness value) should be reported as well. Reporting on the computational time
can be informative but depends on the computational power of a computing machine
as well as on the employed processing schemes and it does not allow for direct com-
parisons among different methods. x-NEAT methods that evolve the topology of an
ANN, like the ones extending NEAT and the ones that automatically configure the sub-
strate of the HyperNEAT-based methods should be additionally evaluated on the size
of evolved networks, eso es, the number of connections and hidden nodes. Agreeing
on a set of performance metrics does not exclude the use of other extra metrics if neces-
sary for characterizing the performance of a specific property of an algorithm. A pesar de
we stress the importance of comparisons and benchmarking, this does not mean that
this is always the most important aspect of an algorithm (Stanley, 2018). Papers that
propose new research directions or provide insights are of great value as well. As it
is stated in Stanley (2018), “While a good algorithm is sometimes one that performs
Bueno, sometimes a good algorithm is instead one that leads to other algorithms and new
frontiers.”

Datasets. Although some datasets are commonly used among x-NEAT methods of
the same or of different subclusters such as the XOR, the pole balancing, the retina

Volumen de cálculo evolutivo 29, Número 1

19

mi. Papavasileiou, j. Cornelis, y B. Jansen

classification problems, etc., in general x-NEAT methods belonging to the same clus-
ter are compared on different problems. Combined with the different employed perfor-
mance metrics as illustrated above, it is evident that direct comparisons among different
methods become very difficult. More standardization and the use of benchmark prob-
lems is required. This article provides an overview of the datasets used in the methods
included in the review but an analysis of the datasets and the proposal of a minimal set
that is suitable for benchmarking should be addressed in future work.

Parameter Configuration. All x-NEAT methods depend on a large number of parame-
ters. The majority of the methods presented in this review article report on the employed
parámetros, while the rest of the papers either include a small portion (p.ej., the size of
the population or the allowed number of generations) or do not report them at all. Cómo-
alguna vez, these parameters are usually configured by trial and error or by reusing previously
defined default parameter settings and not by systematically searching the parameter
space using intelligent optimization methods. Another issue concerns how parameters
should be configured when a method is compared to the state of the art. Optimizando el
parameters for the proposed method and then comparing to the old method may cause
a bias toward favoring the new method. Optimizing the parameters of the old method
and using these on the new method may cause less bias. Por otro lado, optimiz-
ing the parameters of the two methods separately and comparing the methods on their
best performance seems to be the right option. Sin embargo, it is not very often observed
in literature.

Comparison to the State of the Art. During the evaluation process of each paper, nosotros
observed that only one-quarter of the methods perform a (partial) comparison of the
proposed NEAT-based method to another algorithm from EC or an algorithm from an-
other field of computer science, such as ML. This is evident from Table 7. It clearly shows
the trend that most of the papers follow: omitting the performance comparison of the
proposed algorithm to other methods that may exist in other fields. Además, cuando
a method is proposed to solve a problem from a specific domain or a specific dataset,
it should be compared to existing algorithms even though they belong to another cat-
egory of methods, Por ejemplo, to a classifier from ML or a GA from EC. Además,
although a method is proposed to solve a specific task, it would be interesting to see
how the method performs on other datasets. One of the main questions is how to select
the appropriate baseline methods for comparison. The clustering proposed in this ar-
ticle might give an answer. Future studies that propose a new x-NEAT method should
therefore use Figure 2 and Table 6 to position their new method in an existing subcluster
and compare against the state-of-the-art methods of that subcluster. If a new x-NEAT
method does not belong to an existing cluster, un nuevo (sub)-cluster should be created. Es
evident from Table 7 that until now many methods are proposed to deal with a problem
that is tackled by another x-NEAT algorithm of the same subcluster, but no exhaustive
comparisons are performed against the rest of the methods of this subcluster, only to
NEAT or to the NEAT-based algorithm that is extended. This shows how important it is
to be aware of the state of the art and of the clustering approach tackled by our article.

5.3 Current Perspectives

The main advantage of a systematic review is that it is performed based on a proto-
col that is reproducible. Además, deciding which papers should be included is based

20

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Cifra 2: Histogram of the number of papers per year in WoS and Scopus.

on clearly defined criteria and evaluation scores and not on the authors’ opinions.
Sin embargo, one limitation is that the decision of whether a paper should be excluded is
based on its abstract. Por lo tanto, papers whose abstracts or keywords did not include a
reference to NEAT or even the fact that they are extending NEAT, are not included in this
revisar. Además, some other methods can exist in databases other than the Web of Sci-
ence or Scopus, although most of the high-quality papers are included in these two. Nosotros
should therefore note that Table 6 does not include an exhaustive list of all the methods
that belong to each cluster. Other methods may exist that can either be based on NEAT
or employ algorithms independent of NEAT, but they are not included because they do
not fulfill the criteria raised by the protocol. Additional methods have been published
after the day we last searched the databases on 18 Abril 2018 and these are not included
in this review article. A search performed in Web of Science and Scopus on the 18 Julio
2019 for papers published after January 2018, resulted into 35 publicaciones. Applying
only the inclusion/exclusion criteria and not the evaluation criteria resulted in 17 rele-
vant papers. These concern x-NEAT methods belonging to the three clusters defined in
the paper, Por ejemplo, multiobjective optimization (Ihara and Kato, 2017; Künzel and
Meyer-Nieberg, 2018; Chidambaran et al., 2018), evolving ANNs for sequential tasks
with memory requirements (Merrild et al., 2018; ElSaid et al., 2019), hybrid NE meth-
probabilidades (Nadkarni and Neves, 2018; Peng et al., 2018; McDonnell et al., 2018), optimizing
ANNs with different types of nodes (Papavasileiou and Jansen, 2017; Sboev et al., 2018),
optimizing deep learning structures (Desell, 2018), etc..

In the graph of Figure 2 we present the number of published papers in WoS and
Scopus following NEAT in 2002. We chose to show not only the number of meth-
ods proposed to extend NEAT in the ways we present in this article and in the Ap-
pendix, but also the number of papers that are applications of an x-NEAT method to a

Volumen de cálculo evolutivo 29, Número 1

21

mi. Papavasileiou, j. Cornelis, y B. Jansen

challenging problem. Eighteen years after NEAT we can still see its relevance as the
number of published papers extending or applying x-NEAT methods seems to increase.

5.4

Future Perspectives

We believe that especially nowadays in the era of deep learning, NEAT-based algo-
rithms can contribute significantly to the configuration of the DNN. This potential is al-
ready evident in literature as hybrid methods of deep learning and NE have emerged.
The papers (Schrum, 2018; Miikkulainen et al., 2019; Costa et al., 2019) are examples
of x-NEAT methods for evolving DNNs. Especially, DeepNEAT and CoDeepNEAT
(Miikkulainen et al., 2019) are two very promising x-NEAT methods (305 citations in
Google Scholar) for optimizing the structure of DNNs. DeepNEAT follows the prin-
ciples of standard NEAT, eso es, an initial population of chromosomes, represented
by graphs is complexified during evolution by gradually adding structure (edges and
nodos) through mutation operators. Crossover is facilitated by historical markings and
speciation is applied to protect innovation. The differences between NEAT and deep-
NEAT are: each gene in the chromosome represents a layer of a DNN, with a table de-
scribing its hyperparameters, instead of representing a node of an ANN. Además, el
edges in the chromosome do not include weights but only indications of the layers’
conectividad. During fitness evaluation the chromosome is decoded into a DNN that is
trained for a fixed number of epochs. Coevolution DeepNEAT (CoDeepNEAT) is an al-
gorithm optimizing the structure of DNNs inspired by principles of SANE (Moriarty,
1997), ESP (Gomez and Miikkulainen, 1999), and CoSyNE (Gomez et al., 2008). CoDeep-
NEAT generates repetitive modular structure by evolving two populations of modules
and blueprints with coevolution. Each blueprint is a pointer to a module representing
a small DNN. These DNNs are combined to create a larger network. We have the opin-
ion that coevolution of two populations or within the individuals of one population is
a powerful technique that NEAT-based successors should consider in the future, especialmente-
cially when aiming into reaching open-ended evolution (Stanley et al., 2017).

Quality Diversity (QD) algorithms also have potential applications to open-ended
problemas (Pugh et al., 2016; Wang y cols., 2019). QD algorithms search a behavioral space
to find a large set of solutions that are simultaneously diverse and high-performing
(Cully and Demiris, 2017). We believe that novelty search with local competition (NSLC)
(Lehman y Stanley, 2011b) and multidimensional archive of phenotype elites (MAP-
Elites) (Mouret and Clune, 2015) are two QD algorithms that constitute another inter-
esting research area where NEAT-based successors could focus on in the future.

NEAT is a complex method coming from intertwining three principles. Ablation
studies showed that each component of NEAT is crucial to its performance (Stanley
and Miikkulainen, 2002b). Sin embargo, since then, research has provided many power-
ful works such as quality diversity methods (Lehman y Stanley, 2011b; Mouret and
Clune, 2015), age fitness Pareto optimization (Schmidt and Lipson, 2011), and others that
might raise the question of revisiting these ablation studies and investigating whether
each component is really necessary when NEAT is applied in these types of cases. Para
ejemplo, in Mouret and Doncieux (2012), a NE algorithm that employs only the encod-
ing part of NEAT, NEAT’s structural mutation operations (no crossover) and NSGA-II
(Deb et al., 2002) with an objective rewarding exploration, performs better.

6 Conclusión

Eighteen years after NEAT’s invention, a plethora of successors have appeared. Nuestro
review protocol identified 60 methods that met the inclusion criteria and passed the

22

Volumen de cálculo evolutivo 29, Número 1

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

A Systematic Review of NEAT’s Successors

quality assessment threshold. These methods allow us to identify relevant features,
perform real time and/or online evolution, optimize multiple objectives, accelerate the
computational performance, increase the interpretability of the evolved networks, y
allow applications on search spaces with issues including high dimensionality and
deceptive landscapes. NEAT is extended by modifications that include: presentando
new mutation operators, evolving different types of neural nodes, adding new fields
in NEAT’s direct encoding, introducing new types of encoding (p.ej., developmental),
applying different selection strategies, changing the way by which the population is
evolved (p.ej., evolving subnets, coevolving two populations), etc.. As supplementary
material, readers can find a graphical tool on our website3 that uses the 18 subclusters
defined in this article to guide them in choosing a method that satisfies their needs.
This article and the graphical tool can be therefore used as an index to the Appendix of
this article where readers can find the description of the method and its reference to the
original paper.

To facilitate benchmarking and comparisons, we believe that methods belonging
to the same subcluster should be evaluated on the same tasks and should report on
the same performance metrics. As a future work, it would be relevant to categorize
the datasets/tasks of each subcluster to propose a set of benchmark tasks. Combined
with the outcome of this article, researchers will then have all the relevant information
to benchmark and compare a new method using the same state-of-the-art benchmark
test sets and the same performance metrics. Finalmente, we hope that with this article we
can stimulate similar review papers in other areas of EC, Por ejemplo, in multiobjective
optimization for NSGA-II (Deb et al., 2002) or in Evolutionary Strategy for CMA-ES
(Hansen and Ostermeier, 2001).

Acknowledgment

Evgenia Papavasileiou was funded by a PhD grant from the Research Foundation Flan-
ders (FWO) (scholarship numbers 1S34016N, 1S34018N).

Referencias

Auerbach, j. MI., and Bongard, j. C. (2010). Dynamic resolution in the co-evolution of morphol-
ogy and control. In Artificial Life XII: Proceedings of the Twelfth International Conference on the
Synthesis and Simulation of Living Systems, páginas. 451–458.

Auerbach, j. MI., and Bongard, j. C. (2011). Evolving complete robots with cppn-NEAT: The utility
of recurrent connections. In Proceedings of the 13th Annual Conference on Genetic and Evolution-
ary Computation, páginas. 1475–1482.

Azam, F. (2000). Biologically inspired modular neural networks. PhD thesis, Virginia Tech.

Bahçeci, MI., and Miikkulainen, R. (2008). Transfer of evolved pattern-based heuristics in games.

En 2008 IEEE Symposium on Computational Intelligence and Games, páginas. 220–227.

Bettany-Saltikov, j. (2010). Learning how to undertake a systematic review: Parte 1. Nursing Stan-

dard, 24(50):47–55.

obispo, C. METRO. (2006). Pattern recognition and machine learning. Berlina: Saltador.

3www.neatreview.online

Volumen de cálculo evolutivo 29, Número 1

23

yo

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

/

/

mi
d
tu
mi
v
C
oh
a
r
t
i
C
mi
–
pag
d

yo

F
/

/

/

/

/

2
9
1
1
1
8
8
8
4
8
6
mi
v
C
oh
_
a
_
0
0
2
8
2
pag
d

.

F

b
y
gramo
tu
mi
s
t

t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

mi. Papavasileiou, j. Cornelis, y B. Jansen

Box, GRAMO. MI., Jenkins, GRAMO. METRO., Reinsel, GRAMO. C., and Ljung, GRAMO. METRO. (2015). Time series analysis: Forecasting

and control. Hoboken, Nueva Jersey: John Wiley & Sons.

Boyan, j. A., and Moore, A. W.. (1995). Generalization in reinforcement learning: Safely approxi-
mating the value function. In Advances in neural information processing systems, páginas. 369–376.
Cambridge, MAMÁ: CON prensa.

Branke, J., Branke, J., Deb, K., Miettinen, K., and Slowi ´nski, R. (2008). Multiobjective optimiza-
ción: Interactive and evolutionary approaches, volumen. 5252. Berlina: Ciencia Springer & Negocio
Media.

Caamaño, PAG., Salgado, r., Bellas, F., and Duro, R. j. (2016). Introducing synaptic delays in
the NEAT algorithm to improve modelling in cognitive robotics. Neural Processing Letters,
43(2):479–504.

Cardamone, l., Loiacono, D., and Lanzi, PAG. l. (2009). On-line neuroevolution applied to the
open racing car simulator. En 2009 IEEE Congress on Evolutionary Computation, páginas. 2622–
2629.