焦点功能: - 麻省理工学院人工智能研究专业

焦点功能:
Bridging Scales and Levels

NeuroCave: A web-based immersive visualization
platform for exploring connectome datasets

Johnson J. G. Keiriz
Alex D. Leow

1,4

1,3,4

, Liang Zhan

, Olusola Ajilore

2,4

3,4

, and Angus G. Forbes

1,4,5

1计算机科学系, University of Illinois at Chicago, 芝加哥, 伊尔, 美国
2Department of Engineering and Technology, University of Wisconsin–Stout Menomonie, WI, 美国
3Department of Psychiatry, University of Illinois at Chicago, 芝加哥, 伊尔, 美国
4Collaborative Neuroimaging Environment for Connectomics, University of Illinois Chicago, 芝加哥, 伊尔, 美国
5Computational Media Department, 加州大学, Santa Cruz, Santa Cruz, CA, 美国

关键词: Connectome visualization, Immersive analytics, Intrinsic geometry, Network analysis

开放访问

杂志

抽象的

We introduce NeuroCave, a novel immersive visualization system that facilitates the visual
inspection of structural and functional connectome datasets. The representation of the
human connectome as a graph enables neuroscientists to apply network-theoretic
approaches in order to explore its complex characteristics. With NeuroCave, 脑
researchers can interact with the connectome—either in a standard desktop environment or
while wearing portable virtual reality headsets (such as Oculus Rift, Samsung Gear, 或者
Google Daydream VR platforms)—in any coordinate system or topological space, 也
cluster brain regions into different modules on-demand. 此外, a default side-by-side
layout enables simultaneous, synchronized manipulation in 3D, utilizing modern GPU
hardware architecture, and facilitates comparison tasks across different subjects or diagnostic
groups or longitudinally within the same subject. Visual clutter is mitigated using a
state-of-the-art edge bundling technique and through an interactive layout strategy, 尽管
modular structure is optimally positioned in 3D exploiting mathematical properties of
platonic solids. NeuroCave provides new functionality to support a range of analysis tasks
not available in other visualization software platforms.

作者总结

NeuroCave is a web-based software application that facilitates the exploration of and
comparison between connectome datasets in virtual reality environments. This article
describes the visualization features provided by NeuroCave, enabling clinical neuroscientists
to investigate connectome datasets in a ﬂexible manner, to detect patterns within
connectome data, and to generate hypotheses about these observations. NeuroCave is open
source software freely available via our GitHub code repository at https://github.com/
CreativeCodingLab/NeuroCave.

介绍

现代的, noninvasive neuroimaging techniques provide a means with which to understand
structural and functional brain networks, or connectomes (斯波恩斯, 托诺尼, & Kötter, 2005). Dif-
fusion MRI–derived white matter interconnectivity between different brain regions yields the
structural connectome, and BOLD signal correlations generate the functional connectome.
从数学上来说, a connectome can be modeled as a graph by representing the different brain

引文: Keiriz, J. J. G。, Zhan, L。,
Ajilore, 奥。, Leow A. D ., & Forbes, A. G.
(2018). NeuroCave: A web-based
immersive visualization platform for
exploring connectome datasets.
网络神经科学, 2(3), 344–361.
https://doi.org/10.1162/netn_a_00044

DOI:
https://doi.org/10.1162/netn_a_00044

支持信息:
https://dx.doi.org/10.7910/DVN/PQ6ILM

已收到: 24 七月 2017
公认: 10 一月 2018

利益争夺: 作者有
声明不存在竞争利益
存在.

通讯作者:
Angus G. Forbes
angus@ucsc.edu

处理编辑器:
奥拉夫·斯波恩斯

版权: © 2018
麻省理工学院
在知识共享下发布
归因 4.0 国际的
(抄送 4.0) 执照

麻省理工学院出版社

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

regions as nodes. Such models enable neuroscientists to apply network-theoretic methods
and metrics, revealing important properties of the brain, such as small-worldness (Achard,
Salvador, Whitcher, Suckling, & 布莫尔, 2006; Salvador et al., 2005), clustering and modu-
larity (莫尼耶, 兰比奥特, & 布莫尔, 2010), and rich-club conﬁguration (Van Den Heuvel
& 斯波恩斯, 2011), and embeddedness (叶, Zhan, 等人。, 2015) 除其他外.

We introduce NeuroCave, a visual analytics tool to investigate the connectome (数字 1).
Although NeuroCave facilitates a range of explorations, it was initially developed to support
clinical neuroscience investigations, primarily related the following analysis tasks:

T1 Identify regions responsible for speciﬁc cognitive functions and study their interactions

with other regions.

T2 Compare individual networks to the mean or group average connectome, or compare

differences between two group average connectomes.

NeuroCave enables researchers to load in different representations of a dataset in order
to examine a speciﬁc region and to reason about its relationship to other brain regions. 我们

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

数字 1. The user interface of NeuroCave, presenting multiple views to investigate connectome
数据. A researcher can compare different datasets or the same dataset from different perspectives.
这里, (A) shows a high-resolution functional connectome, 和 (乙) shows the same connectome
from a different orientation. When viewing multiple copies of the same dataset, actions can be
synchronized so that interacting with one connectome updates the other. Users can choose an atlas
to label brain regions.
(C) shows the color/glyph atlas, and a user can toggle on or off speciﬁc
brain regions or classiﬁcations by clicking next to items in the atlas. 这里, the right parietal lobe
has been turned off, making it easier for the researcher to interactively explore the speciﬁc areas of
兴趣. 在 (D) a slider controls the opacity of all selected connectivities; 在 (乙), users can turn edge
bundling on or off for selected brain regions; 并在 (F), users can set the minimum or maximum
threshold values and number of hops to determine which edges to display. Users can change the
glyph size of individual nodes or selected brain regions in order to highlight relevant information.
这里, the user has selected and enlarged a node (G) and is investigating edges emanating from the
left parietal lobe above a threshold of 1.35518. NeuroCave is a web application that runs in both
desktop and mobile environments, and users can switch seamlessly between the standard and VR
modes on-demand. The inset image (bottom center) shows a neuroscientist (author ADL) exploring
the 3D dataset in virtual reality using Oculus Rift VR with Touch controllers.

网络神经科学

345

NeuroCave: Immersive visualization platform

explore an example of this in Use Case 1, in which NeuroCave is used to identify a strong
functional coupling between the left and right hemispheres mediated by callosal connections
during the resting state. In group studies, individual variations as well as joint network char-
acteristics are studied in order to identify commonalities or differences between groups, 在-
cluding how these change over time. Use Case 2 explores neurological gender differences
in connectome datasets and observes how these differences relate to various psychological
学习.

NeuroCave enables a wide range of interactive methods to support these tasks, 包括
ﬂexible data loading and data transformations, enabling comparisons within and between con-
nectome datasets; user deﬁned coloring scheme (based on lobar information, modular and/or
community afﬁliation, ETC。); a coordinate system that can be deﬁned interactively by users
or automatically determined via a modular layout based on platonic solids; adjustable glyph
size and transparency of nodes and edges; adjustable connectivity threshold for displaying
边缘; shortest path between two nodes; on-demand edge bundling and edge coloring; 和
on-demand labeling of nodes and edges. Details about these interactive functionalities are
provided below (in the Methods section), and two use cases present examples of the range of
insights that can be generated when using these techniques in NeuroCave to support tasks T1
and T2 (in the Results section).

相关工作

Many tools exist to generate and visualize the connectome in 2D and 3D (马古利斯, Böttger,
Watanabe, & Gorgolewski, 2013). Three-dimensional visualization tools most often repre-
sent the connectome as node-link diagrams, in which nodes are positioned relative to their
corresponding anatomical locations, and links represent the connectivity between nodes. Ex-
amples of such tools include the Connectome Visualization Utility (LaPlante, Douw, 唐, &
施弗尔比姆, 2014), BrainNet Viewer (Xia, 王, & 他, 2013), and the Connectome Viewer
Toolkit (Gerhard et al., 2011). 一般来说, node-link diagrams provide an effective overview
of the entire graph, which makes it easy to observe relationships between both directly and
indirectly connected nodes. 然而, excessive visual clutter is introduced as the number of
edge crossings increases, affecting the readability of the graph.

Representations of the connectome in 2D are also common. In certain cases, adjacency
matrices can better manage large connectome datasets than node-link diagrams (Alper, 巴赫,
Henry Riche, Isenberg, & Fekete, 2013; Ma et al., 2015). 然而, some visual analysis
tasks are difﬁcult to perform using matrix representations (Ghoniem, Fekete, & Castagliola,
2005; 凯勒, Eckert, & Clarkson, 2006), such as detecting graph alterations in group studies.
A popular 2D technique to highlight relevant brain connectivity patterns is the connectogram
(Irimia et al., 2012). In a connectogram, the names of each brain region are presented along
the perimeter of a circle, and the regions are positioned in two different halves according to
the hemisphere they belong to. 此外, each hemisphere is broken down into differ-
ent lobes, subcortical structures, and the cerebellum. The inner space of the circle is divided
into multiple-colored nested rings, where each ring shows a heat map representing a spe-
ciﬁc metric. Interconnections between the regions are illustrated inside the circle by means
of curved lines. As with NeuroCave, a goal of the connectogram is to more effectively repre-
sent densely connected networks, as is the case for the human connectome. Cacciola et al.
(2017) demonstrate that the intrinsic geometry of a structural brain connectome relates to its
brain anatomy, noting that the hyperbolic disk seems a congruous space of representation
for structural connectomes, one in which it is possible to design brain latent-geometry-based

346

Edge bundling:
techniques that reduce visual clutter
in a network by grouping together
edges that follow similar trajectories
between nodes or clusters of nodes.

Visual clutter:
the state in which the representation
of data leads to a degeneration of
performance at some visual analysis
任务. 例如, networks with
many edge crossing introduce visual
clutter because it becomes more
difﬁcult to identify paths between
two nodes.

Intrinsic geometry:
the very high dimensional nonlinear
geometric space natively induced
by the brain’s interconnectivity as
encoded by the connectome, 任何一个
structural or functional, instead of the
neuroanatomical space.

网络神经科学

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

markers for differential connectomic analysis of healthy and diseased. Although connec-
tograms help to prevent some of the clutter that occurs when visualizing networks containing
a large number of edges, it can be challenging to correlate anatomical structures with connec-
活力, and users may ﬁnd it difﬁcult to make sense of connectograms with many layers of inner
and outer circles (伯奇 & Weiskopf, 2014). 而且, it can be time-consuming to produce a
connectogram by using the popular Circos software (Krzywinski et al., 2009), which requires
the preparation of nine distinct conﬁguration ﬁles. 最后, lacking a graphical user interface,
it is generally used as a presentation tool rather than as a means to interact with connectome
数据. Although NeuroCave focuses on supporting the analysis tasks deﬁned above, 它可以是
used to represent data in an analogous way. 数字 2 shows an example of similarly dense
datasets represented in 2D using a connectogram and in 3D by using NeuroCave.

Although most commonly used visualization tools are dedicated desktop applications, 网络-
based implementations, such as Slice:Drop (Haehn, 2013) or BrainBrowser (Sherif, Kassis,
Rousseau, Adalat, & 埃文斯, 2015), free the user from being attached to a speciﬁc operating
系统 (Pieloth, Pizarro, Knosche, Maess, & Fuchs, 2013). 为此, NeuroCave is a web-
based application and runs in any modern browser, both on desktop and mobile computers.
Rojas et al. (2014) ﬁnds that the use of stereoscopic techniques can provide a more immer-
sive way to explore brain imaging data, and Hänel, Pieperhoff, Hentschel, Amunts, & Kuhlen
(2014) show that healthcare professionals perceive the increased dimensionality provided by
stereoscopy as beneﬁcial for understanding depth in the displayed scenery. 而且, Ware
& 米切尔 (2008) ﬁnd that the use of stereographic visualizations reduces the error rate in
graph perception for large graphs with more than 500 节点. Alper, Hollerer, Kuchera-Morin,
& Forbes (2011) observe that when coupled with a highlighting technique, stereoscopic rep-
resentations of 3D graphs outperform their nonimmersive counterpart. NeuroCave harnesses
the visualization capabilities of virtual reality (VR) 环境, which can facilitate spa-
tial manipulation, 识别, and classiﬁcation of objects and imagery, and aid users in
understanding complex scenes (Bohil, Alicea, & Biocca, 2011; Forbes, Villegas, Almryde, &
Plante, 2014; Marai, Forbes, & 约翰逊, 2016). Other tools that make use of VR for visualizing

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

数字 2. An example of a 2D connectogram (左边), taken from the Circos tutorial website
(http://circos.ca/tutorials/), versus a 3D platonic solid representation of a connectome and its mod-
ularity using NeuroCave (正确的). With NeuroCave, users can interactively select particular nodes
or groups of nodes to explore connectivity on-demand, and alternative layouts based on clustering
parameters can be generated as required for a particular analysis task.

网络神经科学

347

NeuroCave: Immersive visualization platform

桌子 1. A survey of neuroimaging connectomic software. This table categorizes each software in terms of whether or not it supports structural
or functional connectomes, 或两者, or if the software is accessed online via a browser. 此外, we indicate whether or not the software
visualizes connectomes as a volume, a surface, or as a graph.

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

网络神经科学

348

NeuroCave: Immersive visualization platform

connectomes include AlloBrain (Thompson et al., 2009), BrainX3 (Arsiwalla et al., 2015;
Betella et al., 2014), and BRAINtrinsic (Conte et al., 2016; Conte, 叶, Forbes, Ajilore, & Leow,
2015). Similar to BRAINtrinsic, NeuroCave emphasizes the ability to switch between anatom-
ical representations and low-dimensional embeddings of connectome datasets. 虽然
NeuroCave includes some of the virtual reality functionality available in these previous con-
nectome visualization tools, it also enables users to move seamlessly between desktop and VR
environments for interactively exploring 3D connectomes in a range of topological spaces,
supports larger connectome datasets, includes novel layout strategies for presenting clusters
of data in 3D space, and introduces a hardware-accelerated edge bundling technique for
reducing link clutter.

桌子 1 provides an overview of popular tools used for visualizing connectome datasets.
Although each of the visualization software tools listed in Table 1 may partially address the
visualization tasks delineated in the introduction, none provides a visualization that can di-
rectly facilitate tasks involving various types of comparison between datasets, since they all
lack the ability to simultaneously load and synchronize a comparative visualization of multi-
ple connectomes. 反而, the user needs to open multiple instances of the application, 哪个
usually requires the use of multiple monitors in order to visually compare the structural or
functional connectomes of the same subject, or two subjects belonging to different groups.
清楚地, with two instances of the software running, user actions will not be synchronized,
making it more difﬁcult to assess visual differences. Some of the applications implemented
in scripting languages, such as R and MATLAB, do provide the user with the ﬂexibility to
customize views (例如, to present multiple connectomes simultaneously). 然而, this re-
quires additional efforts as well as programming expertise. By introducing a side-by-side layout,
NeuroCave enables neuroscientists and researchers to efﬁciently execute tasks that involve
comparative analyses, and to simultaneously spot changes occurring within and across sub-
项目. NeuroCave does not target tractography-related usages, 哪个, although an important
area of connectomics visualization, are not usually a requirement for clinical neuroscientists
(who are the intended audience for our visualization software).

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

方法

NeuroCave is implemented as a web-based application that makes use of three.js, a JavaScript
graphics library for real-time rendering of 3D scenes. It runs on all major web browsers, 和
is thus platform independent. The default view is formed of two side-by-side rendering views
(见图 1 和图 3). Each view enables the interactive visualization of a connectome
as a node-link diagram.

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

Group Visualization

NeuroCave loads connectome data from a user-speciﬁed folder. This folder must contain all
adjacency matrices as well as the corresponding topological and clustering information of
the subjects within the study. An index ﬁle states the subject ID and its corresponding data
ﬁles. Each study or analysis session requires a predeﬁned Atlas that provides numerical labels
and their corresponding anatomical names to each node. NeuroCave currently supports three
Atlases by default: FSL-based parcellation, which consists of 82 labels from FreeSurfer
(菲舍尔, 2012); the Brain Hierarchical Atlas (BHA), comprised of 2,514 labels (Diez et al.,
2015); and the Harvard-Oxford Atlas, which uses 177 labels (Makris et al., 1999). 额外的
Atlases can be created and existing ones can be customized easily, simply by using a preexist-
ing Atlas as a template for deﬁning a new one. (See the online instruction manual available at

网络神经科学

349

NeuroCave: Immersive visualization platform

数字 3. Anatomical versus intrinsic geometry, as ﬁrst proposed by Ye et al. (2015). 这里, 我们
see a screen capture of NeuroCave showing a high-resolution 2514-ROI functional connectome in
the anatomical space (左边) and the intrinsic space (正确的), in which nodes that make up a module
or “community,” as determined via Q-maximization, clearly form a cluster. Our tool facilitates the
simultaneous exploration of multiple connectome datasets in a variety of conﬁgurations, 使能
researchers to make meaningful comparisons between them and to reason about their differences.

https://github.com/CreativeCodingLab/NeuroCave for more details of how to load in datasets
and customize Atlases; Keiriz, Zhan, Ajilore, Leow, & Forbes, 2018a.)

A common task in disease studies involves the comparison of two groups of subjects—for
例子, a healthy control group versus a disease group—in order to derive conclusions about
alterations due to the disease. 现在, no existing connectome visualization application
effectively facilitates real-time simultaneous comparison for two or more datasets. NeuroCave
enables neuroscientists to visualize connectome datasets via a synchronized “side-by-side”
layout, making it easier to explore differences between groups of subjects, or the same group
represented using different spaces, 方式, or in different coordinate systems.

Topology Visualization

NeuroCave positions nodes according to the provided topological information. 可用的
topologies include the anatomical positioning or any of number of applied transformations
that reformulate this positioning into an abstract space. These topologies are automatically
identiﬁed by the application, and ongoing development aims to enable the transformation of
anatomical datasets into a range of topological spaces on-demand. 现在, we have applied
a range of dimensionality reduction techniques to connectome datasets, including Isomap
(Tenenbaum, De Silva, & Langford, 2000) and t-SNE (Maaten & 欣顿, 2008), and we make
use of these methods to help identify patterns in the “intrinsic geometry” (IE。, the geometry
as determined by the brain connectivity itself, either structural or functional) of a connectome
dataset (Cacciola et al., 2017; 叶, Ajilore, 等人。, 2015). Once these intrinsic datasets are loaded,
users can switch between anatomical and these abstract topological spaces as needed to sup-
port particular analyses, making it possible to see the same data transformed in various ways
in order to investigate the connectome from a range of different perspectives. 数字 3 节目
a comparison of the same connectome dataset in an anatomical versus a topological space.

网络神经科学

350

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

数字 4. Modular representation on our platonic solid layout can be used to effectively visualize
multilevel hierarchical clustering. 在这个例子中, each level is a bifurcation of the previous one:
等级 1 with two clusters (左边), 等级 2 with four clusters (中间), 和水平 3 with eight clusters
(正确的). 此外, this ﬁgure illustrates how our force-directed edge bundlin (FDEB) algorithm can
be used to simplify the visualization of edges; here a node corresponding to the left hippocampus
is selected and all emerging edges connected to other brain regions are plotted.

Clustering Visualization

NeuroCave also supports the visualization of clusters of nodes (IE。, modular or community
结构), either embedded within a topological space, or simply as groups of related points
(where the spatial positioning of a cluster of nodes within the cluster has no meaning). Cluster-
ing information is input as a vector of integer values, where each value represents a different
module or cluster. When there is no meaningful spatial positioning provided for clusters (或者
when we choose to exclude this information), NeuroCave makes use of a layout technique
that exploits the geometrical properties of platonic solids. In brief, a platonic solid is a regular,
convex polyhedron constructed by congruent regular polygonal faces with the same num-
ber of faces meeting at each vertex. Five platonic solids exist: tetrahedron, cube, octahedron,
dodecahedron and icosahedron, 和 4, 6, 8, 12, 和 20 faces, 分别. Based on how
many clusters are generated, assuming there are less than 20, a suitable platonic solid is chosen
such that its number of faces is greater than the number of these clusters. The glyphs of each
cluster are then equally distributed according to the sunﬂower algorithm (沃格尔, 1979), 覆盖-
ing the corresponding face of a platonic solid embedded in a sphere. This enables the user to
“enter” into the geometry (IE。, into the “NeuroCave”) via one of the unpopulated faces, 亲-
viding a more immersive experience of the data, especially when the VR mode is activated.
Users can interactively rearrange the position of the clusters within the platonic solid in or-
der to more easily see particular clusters, 例如, those that are densely interconnected,
or that are relevant for a particular analysis session. When applicable, clustering can be re-
computed on-demand, with the user specifying the number of clusters, which in turn updates
the platonic solid that is generated. Connections between individual nodes within clusters or
between clusters can be visualized as well, 如下所述. 数字 4 shows an example
of visualizing multilevel hierarchical clustering using our platonic solids approach. Finding
appropriate visualizations that are useful for desktop and VR environments (或两者) is an on-
going topic in visualization (Liluashvili et al., 2016; 唐, 刘, 张, & Mei, 2016). 我们的
above approach is of course just one of the many ways that connectome datasets could be
visualized in 3D, and we plan to introduce additional layouts in the future.

Node and Edge Visualization

By default, we utilize two different glyphs (spheres and cubes) to differentiate between left and
right hemisphere afﬁliation. Nodes can be colored according to lobar or modular information.

网络神经科学

351

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

Controlling nodal transparency is also possible according to their color scheme and colors can
be interactively assigned to different amounts of transparency modes as desired. 例如,
a brain region that has been assigned a particular color can be toggled on or off (made visible or
invisible), or the transparency of one or more regions or clusters can be increased or decreased
in order to emphasize or de-emphasize them (见图 1). The glyph size of individual nodes
or groups of nodes is also interactively adjustable by the user via our interface, both in desktop
mode or in VR mode. Text labels identifying the nodes can be displayed for all nodes or for
user-selected nodes as desired.

NeuroCave includes a range of features to visualize edges efﬁciently and effectively. 网-
work visualizations that have an excessive amount of overlapping edges, common in dense
node-link diagrams, can introduce unwanted visual clutter, which makes it more difﬁcult to
read and interpret the network. We implement two ways to mitigate this problem. 第一的, 我们
provide the option to hide all edges by default (IE。, to show only the nodes), and then enable
a user to interactively add edges as desired.
In this mode, a user can select any node as a
“root” node, causing all connected edges stemming from this node to be displayed. 第二,
to minimize the clutter occurring from edge crossings, we use the force-directed edge bundling
(FDEB) algorithm to group edges going in the same direction (Holten & Van Wijk, 2009).

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

Standard implementations of edge bundling are too slow for the large numbers of edges that
can appear in some connectome datasets visualized in NeuroCave, reducing the frame rate
of the application and preventing an effective real-time experience. 所以, we introduce
an enhanced WebGL texture-based implementation, extending previous work by Wu, 于, &
于 (2015). In our approach, we can increase the maximum number of edges between nodes
through the use of multiple GPU textures. Our texture-based implementation can bundle the
closest 1,000 edges to the selected node at interactive rates on a desktop computer (see Imple-
mentation Details below). Although this is sufﬁcient for the datasets we explored, we also en-
able users to choose threshold values that limit only connections above or below speciﬁed
strengths to be computed, both to improve performance in situations where the data contains
very dense interconnections, and to assist in analyses focused on particular connectivity
重量.

Each edge can be colored using a gradient, whose two colors are chosen according to the
colors of the source and target nodes that it connects, and where the gradient is skewed toward
the node possessing the higher nodal strength. This enables the user to quickly recognize the
strength of the selected node with respect to its interconnected neighbors, which can help in
identifying important nodes or clusters of nodes, as well as to highlight the reason for modular
changes when they occur in group studies. 数字 5 和图 7 show examples of this edge
coloring approach. (Further details about the layout algorithm, the edge bundling algorithm,
and the gradient coloring can be found in the supplementary material available at the project
网站; Keiriz, Zhan, Ajilore, Leow, & Forbes, 2018a.)

Virtual Reality

NeuroCave can be viewed on a normal desktop or mobile environment, or via a VR sys-
TEM. Currently we support the Oculus Rift and the Samsung Gear VR platforms, with explicit
support for additional platforms planned in the near future.
In addition to the standard 3D
manipulations of panning, rotating, and zooming, NeuroCave supports the advanced interac-
tion features available on the Oculus Rift via the use of Oculus Rift Touch controllers. 这
Touch controllers are a pair of VR input devices that track each hand, enabling an effective
gesture-based manipulation of the VR environment. The user selects the preview area to be

网络神经科学

352

NeuroCave: Immersive visualization platform

数字 5. Connectivity emerging from the anterior (red ring) and posterior (yellow ring) parts of the
precuneus in anatomical space (left panel) and an “intrinsic” space generated using Isomap (中间
panel). Right panel: Connectivity emerging from the posterior part of the precuneus visualized in
the intrinsic space. The color code represents the modular structure of the connectome consisting of
four modules. Note that the orange community contains the default mode network. The bottom-left
inset shows another view of the left panel with all nodes enlarged in order to better see the modular
结构. The top-center inset plot shows the residual geodesics for the ﬁrst 10 dimensions of the
Isomap dimensionality reduction algorithm.

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

explored in VR and then uses the “thumbsticks” on the Touch devices to navigate the visual-
ized connectome. Nodal selection is enabled via a two-step procedure: 首先, pressing the grip
button lets the user point at and highlight a node; 第二, pressing the index button selects
the highlighted node. We can mimic some of this functionality in other VR platforms (例如, 任何
platform that supports WebVR, such as Google Cardboard or Daydream), but node selection
is not as effortless if the controllers do not contain tracking sensors. Users can enter and leave
VR mode as often as they like in order to support investigations of connectome datasets.

From our initial qualitative observations, we ﬁnd that users are very engaged in exploring
the data while in VR mode, and enjoy switching between the different available layouts based
on the different clustering and dimensionality reduction techniques. Users also indicate that
they appreciate the ability to bring up different datasets on-demand while immersed in the VR
mode, especially to see if patterns discovered in one connectome (例如, a connectome dataset
representing average healthy subjects) were present in another (例如, a connectome dataset
representing average diseased subjects). Users also readily move between the desktop display
and the VR display without complaint. 在实践中, users tend to use the VR environments to
make initial explorations of the data and to generate hypotheses about the connectomes, 和
then switch to a desktop view once more nuanced investigations are required. 然而, 我们
believe this is partly due to users not being as familiar with navigating in VR (尤其是
with making ﬁne-grained selections with the VR controllers), and also because of the need to
use additional applications during an analysis (IE。, for web search, taking notes, ETC。) 那是
not readily available when wearing a portable VR headset.

结果

NeuroCave has been tested in a wide range of contexts using a variety of datasets. Through
using the various features in combination in order to interact with connectome datasets, Neuro-
Cave supports a range of analysis tasks, including identifying relevant brain regions, 比较
variations between individual and average connectomes, investigating relationships between
structural and functional connectomes, and analyzing group changes in connectomes, 之中
其他的. The rich set of visualization features provided by NeuroCave makes it possible for

网络神经科学

353

NeuroCave: Immersive visualization platform

Sensemaking:
the process of building a mental
model by exploring and organizing
数据, facilitating decision making
and problem solving.

Q-modularity:
a scalar metric that measures a
network’s community or modular
结构. It is computed as the
fraction of edges that fall within the
given modules minus the expected
fraction if edges were randomly
分散式. Dense nodal connections
within the same modules and sparse
connections between nodes in
different modules are found in
networks with high modularity.

Default mode network, or DMN:
a network that comprises different
brain regions jointly active during the
resting state.

users to explore connectome datasets in a ﬂexible manner, to make observations about con-
nectome data, to generate hypotheses about these observations, and then to dive in more
deeply to support or invalidate hypotheses. 这里, we brieﬂy describe two use cases in which
neuroscientists use NeuroCave to analyze connectome datasets. Although these use cases
show only preliminary research results, they provide evidence indicating that NeuroCave
facilitates useful explorations of complex datasets. 那是, NeuroCave supports the process of
generating and querying visual representations in order to answer task-speciﬁc questions, 或者
what Russell, Steﬁk, Pirolli, and Card (1993) have described as “sensemaking.”

Use Case 1

Our ﬁrst use case explores a resting-state fMRI high-resolution dataset (Keiriz, Zhan, Ajilore,
Leow, & Forbes, 2018乙) consisting of 2,514 regions-of-interest publicly available at NITRC
(http://www.nitrc.org/frs/?group_id=964) and demonstrates the feasibility of visualizing the
intrinsic geometry of the resting state. Extending the PACE procedure introduced by Zhan
等人. (2017) for an N × N functional connectome, this intrinsic geometry is reconstructed by
ﬁrst estimating the probability that an edge eij is positive (nodes i and j are co-activating) 或者
negative (nodes i and j are anti-activating) using a group of subjects. The resulting probabil-
实体, the edge positivity positivity EPij and edge negativity ENij form a complementary pair,
since EPij + ENij = 1, and thus can be jointly coded using the angle of a unit-length vector:
θij = arctan(
ENij/EPij) that varies from 0 到 90 degrees. Using dissimilarity graph em-
bedding (Xing et al., 2016), each node, 我, is then embedded in a 2,514-dimensional space at
the coordinates (θi1, θi2, …θi2514)时间
. 这里, we picked the classic Isomap algorithm because of
its quasi-isometry property (特南鲍姆等人。, 2000) that aims to preserve geodesic distances
in a lower dimensional space (IE。, the “intrinsic space”), but other type of low-dimension
embedding approaches can be used. Separately, we determined the community structure by
maximizing the Q-modularity metric, yielding four communities in this case (纽曼, 2006).

(西德:2)

From the inset plot in Figure 5, it is clear that the Isomap-derived intrinsic geometry of the
resting-state functional connectome is achieved with four dimensions, a novel ﬁnding that
merits further research. To enable 3D visualization of the transformed topology, we retained
the ﬁrst three dimensions of Isomap and visualized the modular structure of the brain in both
the anatomical space as well as this intrinsic space. As the community afﬁliation is separately
determined using Q-maximization, it is thus validating to see that the nodes assigned to the
same community according to Q are positioned close to one another in the intrinsic space. 到
illustrate how neuroscientists explore this complex topological space and gain further insight
into the brain, we selected two nodes that belong to the anterior and posterior part of the
precuneus. Although the nodes are anatomically close to each other, they are known to be
functionally distinct (and thus belong to different modules). 的确, the anterior part of the
precuneus is an important region of the default mode network known to be responsible for
self-referential imagery (thinking about self) and is involved in autobiographical tasks and self-
consciousness, thus activated during “resting consciousness” (Cavanna, 2007). 如图 5
说明, in this intrinsic space the anterior part of the precuneus, while assigned to the orange
module that contains the default mode network, also exhibits diverse connections with various
regions of the brain: the blue cluster that contains the sensorimotor module and the red module
that contains the frontoparietal executive or task-positive system. 相比之下, the posterior
precuneus is part of the visual system (绿色的) and has a relatively restricted pattern (compared
with its more anterior counterpart) of connectivity with the rest of the brain. 尤其, 这样的
connectivity differences only become visually apparent when visualized in this novel space.
The intrinsic geometry of the mean structural connectome from the same dataset reveals a

网络神经科学

354

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

数字 6. This ﬁgure shows an exploration of the structural connectome in intrinsic (左边) 和
anatomical (正确的) geometry for the same connectome dataset gathered from the same participants
as the functional connectome shown in Figure 5.

completely different topology, shown in Figure 6, 左边. 这里, note that visually there exists
a strong left-right symmetry, which is not present in the case of functional connectome. Put
一起, these two intrinsic geometries suggest strong functional coupling between the left
and right hemispheres, likely mediated by the callosal connections, during the resting state.
This ﬁnding is consistent with recent results reported by Cacciola et al. (2017), who indicate a
clear matching between intrinsic geometry and neuroanatomy.

This use case highlights our visualization system successfully supporting task T1, 使能
neuroscientists to explore high-density connectomics data comprising a large number of re-
gions of interest (ROIs) in order to identify and further understand the speciﬁc functionality of
different brain regions. 而且, the side-by-side visualization enables users to reason about
the relationships between the anatomical and the intrinsic topology, facilitating further insight
into how the same brain region can take part in different tasks.

Use Case 2

Our second use case investigates the sex-speciﬁc resting-state functional connectomes in the
F1000 repository, 一个大的 986 subject publicly available resting-state fMRI connectome dataset
available at https://www.nitrc.org/projects/fcon_1000/. Recognizing that negative edges be-
tween nodes in connectome datasets may be neurobiologically relevant (斯波恩斯 & 贝策尔,
2016), Zhan et al. (2017) introduce an approach that utilizes negative signal correlations be-
tween two nodes. The following postprocessing steps were performed on the F1000 dataset: (A)
第一的, to eliminate the potential confounding effect of age, we only included subjects between
20 到 30 years old (319 women at 23.25 ± 2.26 years of age and 233 men at 23.19 ± 2.35); 和
(乙) using PACE (probability-associated community estimation), we constructed a hierarchical
modularity of the resting-state functional connectome followed by a rigorous permutation test-
ing procedure that established novel sex differences, statistically signiﬁcant across all levels of
modular hierarchy, starting at the ﬁrst level ( p = 0.0378), for the following regions, as indicated
图中 7: left precentral gyrus (A), bilateral most posterior segment of the frontal pole near
the frontotemporal junction (B and C), right inferior frontal gyrus pars opercularis (D), and right
hippocampus (乙). These observations, which were initially noted in explorations using Neuro-
Cave, were then further analyzed using other statistical tools, as discussed in Zhan et al. (2017).

Probability-associated community
estimation, or PACE:
a noncorrelation-based
computational connectomics
approach to the modularity of the
functional connectome, 其中
共- versus antiactivations naturally
occur between brain regions.

网络神经科学

355

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

数字 7. This ﬁgure shows two annotated screen captures taken during a visual analysis session
using NeuroCave in which a researcher interactively explores sex-speciﬁc resting-state functional
connectomes in the F1000 repository, exploring differences between average female (左边) and male
(正确的) connectomes. 这里, the color code represents the hierarchical modularity of the connectome,
represented as a dendrogram at the most global level (2 模块). All edges are turned on and nodes
that exhibited switching in modularity are selected (indicated with the larger marker size). Node
A is from the left precentral gyrus, B and C are nodes from the bilateral most posterior segment
of the frontal pole near the fronto-temporal junction, D is from the right inferior frontal gyrus pars
opercularis, and E is from the right hippocampus. Contrasting the top image with the bottom image
(where edges were thresholded at a higher value of 0.33), sex-speciﬁc patterns reveal themselves,
showing a tightly interconnected cluster in A–D, whereas E exhibits an opposite switching pattern.

NeuroCave enables us to visually interpret these sex differences as follows. 第一的, we exam-
ine the connectivity patterns of the aforementioned regions in the anatomical (ﬁrst hierarchical
等级) space by selecting them in NeuroCave (数字 7, 顶部) and using the threshold sliders
available in NeuroCave to restrict edges to values with absolute correlation values for fMRI
BOLD signals larger than 0.33 (数字 7, 底部). In the bottom image of Figure 7, it is clear
that A–D forms a tightly interconnected cluster, whereas E (the right hippocampus) exhibits an
opposite switching pattern. 实际上, afﬁliation patterns for the right hippocampus differ between
sexes. In women they are clustered with other regions that collectively form the default mode
网络, or DMN, the orange module, whereas in males they are clustered with other parietal
and occipital non-DMN ROIs. 尤其, the afﬁliation differences are in the opposite direction
for the left and right frontal poles, the left precentral gyrus, and the right pars opercularis, 这样的
that they are assigned the blue module (IE。, non-DMN) in the average female connectome.

网络神经科学

356

NeuroCave: Immersive visualization platform

The left and right frontotemporal junctions form the larger language system, with the right
pars opercularis (functionally coupled with its homologous area on the left that forms the
Broca’s language area) linked to the processing of semantic information (Heim et al., 2005),
and the superior temporal gyrus involved in the comprehension of language, as well as in the
perception of emotions in facial stimuli (Bigler et al., 2007). 因此, the observed connectivity
differences are likely related to well-known sex differences in language and emotion/affect
加工, as well as differences in self-referential/autobiographical information retrieval. 经过
对比, the hippocampus is known to play an important role in the formation of new memory,
retrieval of declarative long-term memory, and the management and processing of spatial and
spatiotemporal working memory. The modular afﬁnity between the right hippocampus and
other non-DMN regions in the parietal and occipital lobes in men may thus be related to
their postulated advantage in visuospatial tasks, including spatial visualization, 洞察力,
and mental rotation (Linn & 彼得森, 1985). (The visual system is heavily composed of the
occipital lobe, which is responsible for ﬁrst-level visual processing, although part of the parietal
lobe is instrumental for visuospatial skills.)

此外, examining the same dataset in NeuroCave, we ﬁnd a clear visually indication
that the right hippocampus exhibits a stronger contralateral connectivity with the left frontal
cortex in women than in men. 有趣的是, in ovariectomized female rates there is a decrease
in dendritic spine in the prefrontal cortex (全氟碳化物) and hippocampus (华莱士, Luine, Arellanos,
& 法兰克福, 2006), whereas in human studies estrogen has been shown to activate the same
地区 (Berman et al., 1997; 真木, 2005; 真木 & Resnick, 2000; Stevens, 克拉克, & Prestwood,
2005) and estrogen infusion in a group of post-menopausal women increased the connectivity
两者之间 (Ottowitz et al., 2008).

The discovery of such subtle differences is dependent on an iteratively explorative visual-
ization process, only possible with the comprehensive suite of tools, such as those we have
implemented in NeuroCave. 的确, NeuroCave provided researchers with the ability to visu-
ally explore the sex-speciﬁc resting-state functional connectomes, leading to initial hypotheses,
some of which have been analyzed much more thoroughly in Zhan et al. (2017). This use case
demonstrates the effectiveness of NeuroCave in supporting task T2, enabling neuroscientists to
better understand neurological differences between connectome datasets and to observe how
these differences relate to various psychological studies.

讨论

As demonstrated by these two use cases, NeuroCave enables brain researchers to conduct
sophisticated explorations of large datasets interactively. It provides users with the ability to
navigate connectomes in a ﬂexible manner, using the available interactive techniques to switch
between different views, to change clustering metrics and layouts, to highlight meaningful
connections and to ﬁlter out edges or nodes not relevant to a speciﬁc inquiry, to see data in
different topological spaces, and to move seamlessly in and out of VR mode.
In both use
cases described above, researchers made use of each of these techniques in order to home in
on interesting features of the connectome data, and to then validate initial hypotheses about
these features.

Implementation Details

Within NeuroCave, node selection and the edge display are synchronized between the two
side-by-side views: selecting a node in one view activates the node and displays its corre-
sponding edges across both viewing areas, independent of the chosen conﬁguration for each

网络神经科学

357

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

边.
In order to make real-time manipulation of large datasets possible, we use hardware-
accelerated graphics and extend a texture-based implementation of the FDEB algorithm (吴
等人。, 2015). Our implementation harnesses the computational power of the graphical pro-
cessing unit (GPU) in order to perform the required computations, and is at least 50 times faster
than its CPU counterpart, enabling real-time edge bundling of over 1,000 edges at interactive
费率 (tested on a desktop computer with the following hardware: Intel Core i7, 3.4 GHz CPU,
a Nvidia GTX 1070 GPU card, and 32GB RAM). The other features presented in NeuroCave
do not require a dedicated GPU, and the real-time edge bundling does not need to be active
to use the software. 然而, systems with a dedicated GPU are able to render much larger
networks without any noticeable lag between frames. The datasets investigated in Use Case 1,
where each panel displays 2,514 节点, are the largest that we have so far tested, 和我们的
system successfully facilitates pair-wise comparisons between two groups or the same group
presented in different topological spaces. Although only two panels are displayed simulta-
neously, a user can swap out the dataset for each panel on-demand. The ability to do this is
limited only by the memory available to the browser. 在实践中, we have loaded over a dozen
datasets into NeuroCave at the same time, with no loss in performance.

结论

In this paper we presented NeuroCave, a web-based, VR-compatible visualization system that
enables researchers to explore the human connectome in a range of immersive, 交互的
环境, as well as within traditional desktop or mobile environments. NeuroCave sup-
ports the comparison of two datasets in a side-by-side layout in order to facilitate the discov-
ery of connectivity (or disconnectivity) patterns in group studies. Our software makes use of
the GPU to greatly improve the rendering speed of large connectome datasets and to enable
real-time user interactions. As shown in the two illustrative cases, we believe that Neuro-
Cave is a valuable tool for a wide range of structural and functional connectome analyses.
Future work will give users the option to make use of popular community detection algo-
rithms, such as Louvain (Blondel, Guillaume, 兰比奥特, & Lefebvre, 2011) and Infomap
(Rosvall & 伯格斯特罗姆, 2008), which were found to perform well across a range of benchmarks
(Lancichinetti & Fortunato, 2009). We also will adapt our system to support temporally varying
dynamic datasets (Crossley et al., 2016; Forbes et al., 2018; Ma, Forbes, Llano, Berger-Wolf,
& Kenyon, 2016; Purgato, Santambrogio, Berger-Wolf, & Forbes, 2017), and investigate how
NeuroCave can facilitate comparisons between structural and functional connectomes in or-
der to reveal the complex mappings between them (布莫尔 & 斯波恩斯, 2009; C. Honey et al.,
2009; C. J. 蜂蜜, Kötter, Breakspear, & 斯波恩斯, 2007). Another future goal is to quantitatively
assess the impact of VR mode on analysis tasks and empirically investigate the current Neuro-
Cave workﬂow, which encourages moving between VR and desktop modes. 此外, 我们
will investigate the use of collaborative VR environments in which multiple users can interact
with the same datasets simultaneously (Marrinan et al., 2016, 2017).

SUPPORTING INFORMATION

The NeuroCave application and open source code, along with detailed instructions and
examples, are freely available for use and for download at our GitHub repository: https://github.
com/CreativeCodingLab/NeuroCave. All ﬁgures in this article were made using NeuroCave,
and the datasets supporting the use cases can be explored via the online version of NeuroCave
在https://creativecodinglab.github.io/NeuroCave/.

网络神经科学

358

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

作者贡献

Johnson Keiriz: 软件; 可视化; Writing – original draft; 写作——复习 & 编辑.
Liang Zhan: 数据管理; 验证. Olusola Ajilore: 概念化; 调查;
方法; 项目管理; 资源; 监督; 验证; 写作——复习 &
编辑. Alex Leow: 概念化; 形式分析; 资金获取; 调查;
方法; 项目管理; 资源; 监督; 验证; Writing –
original draft; 写作——复习 & 编辑. Angus G. Forbes: 概念化; 调查;
方法; 项目管理; 资源; 软件; 监督; 验证; Visualiza-
的; Writing – original draft; 写作——复习 & 编辑.

资金信息

Alex Leow, 美国国立卫生研究院 (http://dx.doi.org/10.13039/100000002), 奖项ID:
R21AG056782.

参考

Achard, S。, Salvador, R。, Whitcher, B., Suckling, J。, & 布莫尔,
乙. (2006). A resilient, low-frequency, small-world human brain
functional network with highly connected association cortical
枢纽. 神经科学杂志, 26(1), 63–72.

Alper, B., 巴赫, B., Henry Riche, N。, Isenberg, T。, & Fekete, J.-D.
(2013). Weighted graph comparison techniques for brain con-
在: Proceedings of the ACM SIGCHI confer-
nectivity analysis.
ence on human factors in computing systems (PP. 483–492).
Alper, B., Hollerer, T。, Kuchera-Morin, J。, & Forbes, A. G. (2011).
Stereoscopic highlighting: 2D graph visualization on stereo
IEEE Transactions on Visualization and Computer
显示.
Graphics, 17(12), 2325–2333.

Arsiwalla, X. D ., Zucca, R。, Betella, A。, Martinez, E., Dalmazzo, D .,
Omedas, P。, . . . Verschure, 磷. F. (2015). Network dynamics with
BrainX3: A large-scale simulation of the human brain network
with real-time interaction. 神经信息学前沿, 9, 02.
Berman, K. F。, 施密特, 磷. J。, Rubinow, D. R。, Danaceau, 中号. A。,
Van Horn, J. D ., Esposito, G。, . . . 温伯格, D. 右. (1997). Mod-
ulation of cognition-speciﬁc cortical activity by gonadal steroids:
A positron-emission tomography study in women. 会议记录
of the National Academy of Sciences, 94(16), 8836–8841.

Betella, A。, Cetnarski, R。, Zucca, R。, Arsiwalla, X. D ., Martinez, E.,
Omedas, P。, . . . Verschure, 磷. F. (2014). BrainX3: Embodied ex-
ploration of neural data. 在: 诉讼程序 2014 virtual re-
ality international conference (p. 37).

Bigler, 乙. D ., Mortensen, S。, Neeley, 乙. S。, Ozonoff, S。, Krasny, L。,
约翰逊, M。, . . . Lainhart, J. 乙. (2007). Superior temporal gyrus, lan-
guage function, and autism. Developmental Neuropsychology,
31(2), 217–238.

Blondel, V. D ., Guillaume, J.-L., 兰比奥特, R。, & Lefebvre, É.
(2011). The louvain method for community detection in large net-
作品. Journal of Statistical Mechanics: Theory and Experiment,
10, P10008.

Bohil, C. J。, Alicea, B., & Biocca, F. A. (2011). Virtual reality in neu-
roscience research and therapy. 自然评论神经科学,
12(12), 752–762.

布莫尔, E., & 斯波恩斯, 氧. (2009). 复杂的大脑网络: 图形
结构和功能系统的理论分析. 自然
评论神经科学, 10(3), 186–198.

伯奇, M。, & Weiskopf, D. (2014). On the beneﬁts and drawbacks
of radial diagrams. In W. 黄 (埃德。), Handbook of human-
centric visualization (PP. 429–451). 纽约: 施普林格.

Cacciola, A。, Muscoloni, A。, Narula, 五、, Calamuneri, A。, Nigro, S。,
Mayer, 乙. A。, . . . Cannistraci, C. V. (2017). Coalescent embed-
ding in the hyperbolic space unsupervisedly discloses the hidden
geometry of the brain. arXiv 预印本 arXiv:1705.04192. https://
arxiv.org/pdf/1705.04192.pdf

Cavanna, A. 乙. (2007). The precuneus and consciousness. CNS

Spectrums, 12(07), 545–552.

Conte, G。, 叶, A。, Almryde, K., Ajilore, 奥。, Leow, A。, & Forbes,
A. G.
(2016). Intrinsic geometry visualization for the inter-
active analysis of brain connectivity patterns. 在: 会议记录
electronic imaging, visualization and data analysis (PP. 481-1–8).
Springﬁeld, VA: Society for Imaging Science and Technology.
Conte, G。, 叶, A. Q., Forbes, A. G。, Ajilore, 奥。, & Leow, A. (2015).
BRAINtrinsic: A virtual reality-compatible tool for exploring in-
trinsic topologies of the human brain connectome. In Y. Guo,
K. 弗里斯顿, F. Aldo, S. 爬坡道, & H. 彭 (编辑。), Brain informatics
和健康 (PP. 67–76). 施普林格.

Crossley, 氮. A。, Marques, 时间. R。, 泰勒, H。, Chaddock, C。,
Dell’Acqua, F。, Reinders, A. A。, . . . Kapur, S. (2016). Connec-
tomic correlates of response to treatment in ﬁrst-episode psy-
chosis. Brain, 140(2), 487–496.

Diez, 我。, Bonifazi, P。, Escudero, 我. N。, Mateos, B., Muñoz, 中号. A。,
Stramaglia, S。, & 科尔特斯, J. 中号. (2015). A novel brain partition
highlights the modular skeleton shared by structure and function.
科学报告, 5, 10532.

菲舍尔, 乙. (2012). Freesurfer. 神经影像, 62(2), 774–781.
Forbes, A. G。, Burks, A。, 李, K., 李, X。, Boutillier, P。, Krivine, J。,
& 丰塔纳, 瓦. (2018). Dynamic inﬂuence networks for rule-
based models. IEEE Transactions on Visualization and Computer
Graphics, 24(1), 184–194.

网络神经科学

359

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

Forbes, A. G。, Villegas, J。, Almryde, K. R。, & Plante, 乙.

(2014). A
stereoscopic system for viewing the temporal evolution of brain
activity clusters in response to linguistic stimuli. 在: 会议记录
of SPIE: 卷. 9011. (p. 90110我).

Gerhard, S。, Daducci, A。, Lemkaddem, A。, 买, R。, Thiran, J.-P.,
& 哈格曼, 磷. (2011). The Connectome Viewer Toolkit: 一个
open source framework to manage, analyze, and visualize con-
nectomes. 神经信息学前沿, 5, 3.
J.-D., & Castagliola, 磷.

(2005). 上
readability of graphs using node-link and matrix-based rep-
resentations: A controlled experiment and statistical analysis.
Information Visualization, 4(2), 114–135.

Ghoniem, M。, Fekete,

Haehn, D. (2013). Slice:Drop: Collaborative medical imaging in the
browser. In ACM SIGGRAPH 2013 computer animation festival.
Hänel, C。, Pieperhoff, P。, Hentschel, B., Amunts, K., & Kuhlen,
时间.
Interactive 3D visualization of structural changes in
the brain of a person with corticobasal syndrome. Frontiers in
神经信息学, 8, 42.

(2014).

Heim, S。, Alter, K., Ischebeck, A. K., Amunts, K., Eickhoff, S. B.,
Mohlberg, H。, . . . Friederici, A. D. (2005). The role of the left
Brodmann’s areas 44 和 45 in reading words and pseudowords.
Cognitive Brain Research, 25(3), 982–993.

Holten, D ., & Van Wijk, J. J. (2009). Force-directed edge bundling
for graph visualization. 在: Proceedings of the eurographics
conference on visualization (EuroVis) (PP. 983–998).

蜂蜜, C。, 斯波恩斯, 奥。, 卡蒙, L。, 巨人, X。, Thiran, J.-P.,
买, R。, & 哈格曼, 磷. (2009). Predicting human resting-state
functional connectivity from structural connectivity. 会议记录
of the National Academy of Sciences, 106(6), 2035–2040.

蜂蜜, C. J。, Kötter, R。, Breakspear, M。, & 斯波恩斯, 氧. (2007). 网-
work structure of cerebral cortex shapes functional connectivity
on multiple time scales. 美国国家科学院院刊
科学, 104(24), 10240–10245.

Irimia, A。, Chambers, 中号. C。, Torgerson, C. M。, Filippou, M。, Hovda,
D. A。, Alger, J. R。, . . . 其他的 (2012). Patient-tailored connec-
tomics visualization for the assessment of white matter atrophy
in traumatic brain injury. Frontiers in Neurology, 3, 10.

Keiriz, J. J. G。, Zhan, L。, Ajilore, 奥。, Leow, A. D ., & Forbes, A. G.
(2018A). CreativeCoding Lab. https://github.com/CreativeCoding
Lab/NeuroCave

Keiriz, J. J. G。, Zhan, L。, Ajilore, 奥。, Leow, A. D ., & Forbes, A. G.
(2018乙). 数据集. https://creativecodinglab.github.io/NeuroCave/
凯勒, R。, Eckert, C. M。, & Clarkson, 磷. J. (2006). Matrices or node-
link diagrams: Which visual representation is better for visualis-
ing connectivity models? Information Visualization, 5(1), 62–76.
Krzywinski, M。, Schein, J。, Birol, 我。, Connors, J。, Gascoyne, R。,
Horsman, D ., . . . Marra, 中号. A. (2009). Circos: An information
aesthetic for comparative genomics. Genome Research, 19(9),
1639–1645.

Lancichinetti, A。, & Fortunato, S. (2009). Community detection algo-
rithms: A comparative analysis. Physical Review E, 80(5), 056117.
LaPlante, 右. A。, Douw, L。, 唐, W., & 施弗尔比姆, S. 中号. (2014).
The connectome visualization utility: Software for visualization
of human brain networks. 公共图书馆一号, 9(12), e113838.

Liluashvili, 五、, Kalayci, S。, Flouder, E., Wilson, M。, Gabow, A。, &
Gümü¸s, Z. H. (2016). iCAVE: An open source tool for immersive

3D visualization of complex biomolecular interaction networks.
bioRxiv, 061374.

Linn, 中号. C。, & 彼得森, A. C. (1985). Emergence and characteri-
zation of sex differences in spatial ability: A meta-analysis. Child
发展, 56, 1479–1498.

Ma, C。, Forbes, A. G。, Llano, D. A。, Berger-Wolf, T。, & Kenyon, 右. V.
(2016). SwordPlots: Exploring neuron behavior within dynamic
communities of brain networks. Journal of Imaging Science and
技术, 60(1), 10405-1–13.

Ma, C。, Kenyon, 右. 五、, Forbes, A. G。, Berger-Wolf, T。, Slater, 乙. J。,
& Llano, D. A. (2015). Visualizing dynamic brain networks using
an animated dual-representation. 在: Proceedings of the euro-
graphics conference on visualization (EuroVis) (PP. 73–77).

Maaten, L. V. D ., & 欣顿, G. (2008). Visualizing data using t-SNE.

Journal of Machine Learning Research, 9, 2579–2605.

真木, 磷. 中号. (2005). Estrogen effects on the hippocampus and frontal
lobes. International Journal of Fertility and Women’s Medicine,
50(2), 67–71.

真木, 磷. M。, & Resnick, S. 中号. (2000). Longitudinal effects of estro-
gen replacement therapy on pet cerebral blood ﬂow and cogni-
的. Neurobiology of Aging, 21(2), 373–383.

马克里斯, N。, 迈耶, J. W., Bates, J. F。, Yeterian, 乙. H。, 肯尼迪, D. N。,
& Caviness, V. S. (1999). MRI-based topographic parcellation of
human cerebral white matter and nuclei: 二. Rationale and appli-
cations with systematics of cerebral connectivity. 神经影像,
9(1), 18–45.

Marai, G. E., Forbes, A. G。, & 约翰逊, A. (2016). Interdisciplinary
immersive analytics at the Electronic Visualization Laboratory:
Lessons learned and upcoming challenges. 在: IEEE VR workshop
on immersive analytics (PP. 54–59).

Marrinan, T。, Renambot, L。, Leigh,

马古利斯, D. S。, Böttger, J。, Watanabe, A。, & Gorgolewski, K. J. (2013).
Visualizing the human connectome. 神经影像, 80, 445–461.
琼斯,
S。, & 约翰逊, A. (2016). Synchronized mixed presence data-
conferencing using large-scale shared displays. 在: 会议记录
of the 11th ACM international conference on interactive surfaces
and spaces (ISS) (PP. 355–360). Niagara Falls, 加拿大.

J。, Forbes, A. G。,

Marrinan, T。, Renambot, L。, Leigh, J。, Forbes, A. G。, 琼斯, S。, &
约翰逊, A. (2017). Mixed presence collaboration using scalable
visualizations in heterogeneous display spaces. 在: 会议记录
the 20th ACM Conference on Computer Supported Cooperative
工作 & Social Computing (CSCW) (PP. 2236–2245). Portland,
Oregon.

莫尼耶, D ., 兰比奥特, R。, & 布莫尔, 乙. 时间. (2010). Modular and
hierarchically modular organization of brain networks. Frontiers
in Neuroscience, 4, 200.

纽曼, 中号. 乙. (2006). Modularity and community structure in
网络. 美国国家科学院院刊,
103(23), 8577–8582.

Ottowitz, 瓦. E., Siedlecki, K. L。, Lindquist, 中号. A。, Dougherty, D. D .,
Fischman, A. J。, & 大厅, J. 乙.
(2008). Evaluation of prefrontal–
hippocampal effective connectivity following 24 hours of estro-
gen infusion: An FDG-PET study. Psychoneuroendocrinology,
33(10), 1419–1425.

Pieloth, C。, Pizarro, J. M。, Knosche, T。, Maess, B., & Fuchs, 中号. (2013).
An online system for neuroelectromagnetic source imaging.

网络神经科学

360

我

D
哦
w
n
哦
A
d
e
d

F
r
哦
米
H

t
t

:
/
/

d
我
r
e
C
t
.

米

我
t
.

e
d
你
n
e
n
A
r
t
我
C
e
–
p
d

我

F
/

2
3
3
4
4
1
0
9
2
1
1
9
n
e
n
_
A
_
0
0
0
4
4
p
d

乙
y
G
你
e
s
t

哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3

NeuroCave: Immersive visualization platform

在: IEEE International Conference on Intelligent Data Acquisition
and Advanced Computing Systems (IDAACS) (PP. 270–274).
Purgato, A。, Santambrogio, M。, Berger-Wolf, T。, & Forbes, A. G.
(2017). Interactive visualization for brain spatiotemporal net-
作品. 在: Proceedings of the IEEE International Conference on
Biomedical and Health Informatics (BHI) (PP. 21–24).

Rojas, G. M。, Gálvez, M。, Vega Potler, N。, Craddock, 右. C。,
马古利斯, D. S。, Castellanos, F. X。, & Milham, 中号. 磷. (2014).
Stereoscopic three-dimensional visualization applied to multi-
modal brain images: Clinical applications and a functional con-
nectivity atlas. Frontiers in Neuroscience, 8, 328.

Rosvall, M。, & 伯格斯特罗姆, C. 时间. (2008). Maps of random walks on
complex networks reveal community structure. 会议记录
the National Academy of Sciences, 105(4), 1118–1123.

拉塞尔, D. M。, Steﬁk, 中号. J。, Pirolli, P。, & Card, S. K. (1993). The cost
在: Proceedings of the ACM INTER-
structure of sensemaking.
ACT and CHI conference on human factors in computing systems
(PP. 269–276).

Salvador, R。, Suckling, J。, Coleman, 中号. R。, Pickard, J. D ., Menon,
D ., & 布莫尔, 乙. (2005). Neurophysiological architecture of
functional magnetic resonance images of human brain. Cerebral
Cortex, 15(9), 1332–1342.

Sherif, T。, Kassis, N。, Rousseau, M.-É., Adalat, R。, & 埃文斯, A. C.
(2015). BrainBrowser: Distributed, web-based neurological data
可视化. 神经信息学前沿, 8, 89.

斯波恩斯, 奥。, & 贝策尔, 右. F. (2016). 模块化大脑网络. Annual

Review of Psychology, 67(1), 613–640.

斯波恩斯, 奥。, 托诺尼, G。, & Kötter, 右. (2005). The human con-
nectome: A structural description of the human brain. 公共科学图书馆
计算生物学, 1(4), e42.

Stevens, 中号. C。, 克拉克, V. P。, & Prestwood, K. 中号. (2005). Low-dose es-
tradiol alters brain activity. Psychiatry Research: Neuroimaging,
139(3), 199–217.

唐, J。, 刘, J。, 张, M。, & Mei, 问. (2016). Visualizing large-scale
and high-dimensional data. 在: Proceedings of the 25th interna-
tional conference on world wide web (PP. 287–297).

Tenenbaum, J. B., De Silva, 五、, & Langford, J. C. (2000). A global
geometric framework for nonlinear dimensionality reduction.
科学, 290(5500), 2319–2323.

汤普森, J。, Kuchera-Morin, J。, Novak, M。, Overholt, D ., Putnam,
L。, Wakeﬁeld, G。, & 史密斯, 瓦. (2009). The AlloBrain: An inter-
积极的, stereographic, 3D audio, immersive virtual world. 国际米兰-
national Journal of Human-Computer Studies, 67(11), 934–946.
Van den Heuvel, 中号. P。, & 斯波恩斯, 氧. (2011). Rich-club organiza-
tion of the human connectome. 神经科学杂志, 31(44),
15775–15786.

沃格尔, H. (1979). A better way to construct the sunﬂower head.

Mathematical Biosciences, 44(3-4), 179–189.

华莱士, M。, Luine, 五、, Arellanos, A。, & 法兰克福, 中号. (2006).
Ovariectomized rats show decreased recognition memory and
spine density in the hippocampus and prefrontal cortex. Brain
研究, 1126(1), 176–182.

Ware, C。, & 米切尔, 磷. (2008). Visualizing graphs in three dimen-
西翁. ACM Transactions on Applied Perception (TAP), 5(1), 2.

吴, J。, 于, L。, & 于, H.

(2015). Texture-based edge bundling:
A web-based approach for interactively visualizing large graphs.
在: IEEE international conference on big data (PP. 2501–2508).

Xia, M。, 王, J。, & 他, 是.

(2013). BrainNet Viewer: A network
visualization tool for human brain connectomics. 公共图书馆一号,
8(7), e68910.

Xing, M。, Ajilore, 奥。, Wolfson, 奥。, Abbott, C。, MacNamara, A。,
Tadayonnejad, R。, . . . Leow, A. (2016). Thought chart: Tracking
dynamic EEG brain connectivity with unsupervised manifold
学习. In G. Ascoli, 中号. Hawrylycz, H. Ali, D. Khazaranchi,
(编辑。), Brain informatics and health (卷. 9919
& 是. Shi
of Lecture Notes in Artiﬁcial Intelligence, PP. 149–157). 占婆:
施普林格.

叶, A. Q., Ajilore, 奥。, Conte, G。, GadElkarim, J。, Thomas-Ramos,
G。, Zhan, L。, . . . Leow, A. D. (2015). The intrinsic geometry of
the human brain connectome. Brain Informatics, 2(4), 197–210.
叶, A. Q., Zhan, L。, Conrin, S。, GadElKarim, J。, 张, A。, 哪个, S。,
. . . Leow, A. (2015). Measuring embeddedness: 分层的
scale-dependent information exchange efﬁciency of the human
脑连接组. 人脑图谱, 36(9), 3653–3665.
Zhan, L。, Jenkins, L. M。, Wolfson, 奥。, GadElkarim, J. J。, Nocito, K.,
汤普森, 磷. M。, . . . Leow, A. D. (2017). The signiﬁcance of ne-
gative correlations in brain connectivity. Journal of Comparative
Neurology, 525(15), 3251–3265.