Office of Naval Research “Computational Neuroscience, Sensory Augmentation” Arlington, VA, April 25-26, 2007
Large-Scale Biologically Realistic Models of
Cortical Mesocircuit Dynamics Philip H. Goodman1 and René Doursat1,2 1Brain
Computation Laboratory, School of Medicine, University of Nevada, Reno
2Institut
des Systèmes Complexes, CNRS & Ecole Polytechnique, Paris, France
Contributors • Graduate Students Brain models & Robotics Oscar Sessions Milind Zirpe Membrane Ion Channels Del Ray Jackson Sermsak Buntha Cluster Optimization John Kenyon • Post-Doc Quan Zou • Investigators & Collaborators Fred Harris, Jr. Monica Nicolescu Phil Goodman AIBO
April 25, 2007
René Doursat Henry Markram Jim King
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
2
Large-Scale Cortical Mesocircuit Dynamics
1. Scope of Work
PFC AV
AS
MC
SC
2. Neurodynamics & Mesocircuit Theory 3. RAIN Concepts & Progress to Date 4. Future Directions April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
3
1. Scope of Work ¾ Assumptions: • • •
The brain is an analog system (with some digital-like features) The brain obeys classical, non-quantum physics, operating in biological time The brain can be reverse-engineered
¾ Obstacles to understanding relevant brain processing: • • • •
No consensus on algorithmic basis of cognition, memory, or consciousness Biological brain models have an extreme no. of unconstrained parameters Extracting biological parameters from behaving brains is very difficult Emulating genetic/developmental neural cell biology doesn’t “explain” it
¾ Hypotheses: • •
Phenomenologically-detailed brain mesocircuit models (to be described) could bridge this gap in our understanding (and give rise to novel hybrid analog-digital “AI”) Robots based on mesocircuit theory and through social interaction can provide strong behavioral constraints on the cortical modeling (and result in immediate applications)
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
4
1. Scope of Work ¾ Simulation Software and Cluster Development 9 200-CPU cluster (Xeon/Opteron), Myrinet, 2 TB memory 9 NCS software written in C/C++ with MPI architecture
9 100 columns x 10,000 cells = 1 million multicompartmental neurons, massively interconnected through 1 billion synapses April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
5
Large-Scale Cortical Mesocircuit Dynamics
1. Scope of Work
PFC AV
AS
MC
SC
2. Neurodynamics & Mesocircuit Theory 3. RAIN Concepts & Progress to Date 4. Future Directions April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
6
2. Neurodynamics ¾ AI: symbols, syntax → production rules 9 logical systems define high-level symbols that can be composed together in a generative way → they are lacking a “microstructure” needed to explain the fuzzy complexity of perception, categorization, motor control, learning
¾ Neural networks: neurons, links → activation rules 9 in neurally inspired dynamical systems, the nodes of a network activate each other by association → they are lacking a “macrostructure” needed to explain the systematic compositionality of language, reasoning, cognition
¾ Missing link: “mesoscopic” level of description 9 cognitive phenomena emerge as complex systems from the underlying neurodynamics, via intermediary spatiotemporal patterns April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
7
2. Neurodynamics macrolevel:
×
TT × Tt
×
→ Tt × Tt → TT, Tt, tT, tt
molec. biology
mesolevel:
genetics
microlevel: atoms
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
8
2. Neurodynamics “John gives a book to Mary”
“molec. congition”
Mary givG e John
mesolevel:
symbols
O
R
→
O
G
h Jo
n O
bal l
Ge giv
nsO ow P
R
ry Ma
bo
give
ok
owns
O
P
R
“Mary is the owner of the book”
ba ll
book
macrolevel:
John Mary
after Elie Bienenstock (1995, 1996)
microlevel: neurons
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
9
2. Neurodynamics ¾ Laws of neurodynamics: from rate to temporal coding 9 more than mean rates → temporal correlations among spikes high activity rate rate coding
high activity rate high activity rate low activity rate low activity rate low activity rate temporal coding
¾ zero-delays: synchrony
(1 and 2 more in sync than 1 and 3)
¾ nonzero delays: rhythms
(4, 5 and 6 correlated through delays)
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
10
2. Neurodynamics ¾ From digital to analog temporal coding
9 neurons receive a great amount of background activity from close or remote cortical areas 9 this activity is irregular but somewhat rhythmic and has a critical influence on the neurons’ responsiveness → this suggests a new form of analog binding, instead of spike synchrony April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
11
Destexhe & Pare, Neurocomputing, 2000
S. M. Bohte, 2003
9 more than discrete spikes → continuous membrane potential
2. Neurodynamics ¾ Cognitive Neurodynamics 9 Springer journal: “CN is a trend to study cognition from a dynamic view that has emerged as a result of the rapid developments taking place in nonlinear dynamics and cognitive science.” focus on the spatiotemporal dynamics of neural activity in describing brain function contemporary theoretical neurobiology that integrates recent and rapid advances in nonlinear dynamics, complex systems and statistical physics often contrasted with computational and modular approaches of cognitive neuroscience
2. Neurodynamics ¾ Cognitive Neurodynamics. . . and beyond 9 CN also distinguishes three levels of organization (W. Freeman): microscopic − multiple spike activity (MSA) “mesoscopic” − local field potentials (LFP), electrocorticograms (ECoG) macroscopic − electroencephalograms (EEG)
9 however, in the CN view, upper levels are generally based on neural fields: continuum approximation of discrete neural activity by spatial and temporal integration of lower levels → loss of spatial and temporal resolution
→ in our view, the mesoscopic level of description should retain the fine details of spiking (and subthreshold) patterns: what truly matters are the spatiotemporal “shapes” of mesoscopic objects
2. Neurodynamics ¾ Mesoscopic entities are spatiotemporal patterns (STPs) 9 large-scale, localized dynamic cell assemblies that display complex, reproducible digital-analog regimes of neuronal activity
mesoscopic neurodynamics
9 these regimes of activity are supported by specific, ordered patterns of recurrent synaptic connectivity
9 paradigm shift: construing the brain as a pattern formation machine
2. Neurodynamics ¾ Theories populating the mesolevel: a zoology of STPs 9 polychronous groups
mesolevel
mesolevel
9 synfire chains & braids
Abeles (1982), Bienenstock (1995), Doursat (1991)
Izhikevich (2006)
9 RAIN & mesocircuits
mesolevel
mesolevel
9 BlueColumn
Markram (2006) April 25, 2007
3200 EXC
800 INH
Brunel (2000), Vogels & Abbott (2006), Goodman & Doursat (2006) Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
15
2. Neurodynamics ¾ A building-block game of mental representations
dot_wave2
9 STPs can bind, interact and/or assemble at several levels, forming complex structures from simpler ones in a hierarchy
dot_wave1
9 binding by temporal correlations and fast synaptic plasticity synchronization delayed correlations, waves analog induction, resonance, etc. April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
16
2. Mesocircuit Theory ¾ Inter-pattern perturbation and coherence induction 9 cell assemblies can interact through weak coupling contacts and influence each other’s dynamical activity modes
mesoscopic neurodynamics
9 an input “key” stimulating a “lock” pattern induces a transient regime change and a greater or lesser response in the lock
Lock
Key
9 the lock has key-specific recognition/representation abilities; it is similar to a “template”, except not a copy of, or analogous to the key similar to an “attractor”, except does not need the key to be active April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
17
2. Mesocircuit Theory ¾ New neural dynamics: perturbation by coupling 9 subtle but fundamental distinction between activating and perturbing/influencing 9 old paradigms (input/output processing chain): a lower area literally activates a higher area, initially silent
9 new paradigm: subnetworks already possess endogenous modes of activity, perturbed by coupling interactions, possibly two-way
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
18
2. Mesocircuit Theory ¾ “Molding” locks by Hebbian learning 9 a lock’s ability to respond more to certain types of keys relies on its specific distribution of synaptic contacts
mesoscopic neurodynamics
9 these contacts could be learned through Hebbian modification, for example by presenting certain types of keys repeatedly
Lock
Key
loc kl rn ea ing
9 during the training phase, undifferentiated locks are transformed into specialized locks April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
19
2. Mesocircuit Theory ¾ Lock diversification and inter-lock competition 9 both training and functional phases could be carried out on multiple locks in parallel
April 25, 2007
dot_wave1
mesoscopic neurodynamics
9 inter-lock competition by inhibitory feedback could create diversification and specialization in the context of a “society of locks”
Locks
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
Various Stimulus-Keys
20
2. Mesocircuit Theory ¾ Preliminary lock & key experiments 9 lock L: 100 neurons (R) driven by 20 source cells (S) via different combinations of synaptic weights (W), which make L’s specificity 9 S cells are bursting at various frequencies and phases, creating complex potential landscapes in R cells with low spike frequency ...
mesoscopic neurodynamics
W
...
S
R
L
K
9 key K is a random train of spikes: when L is stimulated by K, the R cells’ firing rates increase more or less, depending on the match between their endogenous temporal structure and K’s structure April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
21
2. Mesocircuit Theory ¾ Lock excited by key 9 example of strongly resonant L-cell → 6 extra spikes when stimulated by K: blue signal = intrinsic L potential red signal = potential when excited by K
9 example of nonresonant L-cell → 0 extra spikes when stimulated by K:
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
22
2. Mesocircuit Theory ¾ Lock excited by key 9 other examples of L-cells strongly excited by the K spikes
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
23
2. Mesocircuit Theory ¾ Next stages in the model 9 driven mode – L modeled as a subthreshold SNN driven by source cells, and K as spike trains − endogenous mode – add to L recurrent connections (possibly delayed) and inhibitory neurons → RAIN Concepts − Hebbian learning – add STDP synaptic plasticity and devise training procedure, so that lock L “molds” around specific keys K − brain module – integrate a society of L networks into the association module (AS) of the brain architecture
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
24
Large-Scale Cortical Mesocircuit Dynamics
1. Scope of Work
PFC AV
AS
MC
SC
2. Neurodynamics & Mesocircuit Theory 3. RAIN Concepts & Progress to Date
3200 EXC
800 INH
4. Future Directions April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
25
3. RAIN Concepts ¾ Evoked vs. ongoing activity
(Weliky M, Bosking WH, Fitzpatrick D. Nature 1996;379:725-728)
April 25, 2007
(Tsodyks et al. Science 1999; 286, 1943 -1946)
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
26
3. RAIN Concepts ¾ Ongoing activity -transient coherence
(Kenet et al. Spontaneously emerging cortical representations of visual attributes. Nature 2003;425:954-956)
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
27
3. RAIN Concepts ¾ in vivo awake spiking (identical cell, repeated trials)
Rat Thalamus LGN (Erik Flister, Pam Reinagel): 2006, UCSD online course
Monkey Lateral Prefrontal Cortex (Shima et al. Nature 2007; 445:315-318) April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
28
3. RAIN Concepts ¾ First, we stimulate a recurrently connected excitatory network... Pconnect Gexc
April 25, 2007
800 excitatory neurons
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
29
3. RAIN Concepts ¾ Next, we add recurrent inhibition via E:I and I:E synapses... Pconnect Gexc
800 excitatory neurons
Pconnect Gexc 200 inhibitory neurons
Ginh
Pconnect
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
30
3. RAIN Concepts ¾ How could we get sustained firing? Pconnect Gexc
Rand Noise
800 excitatory neurons
Pconnect Gexc 200 inhibitory neurons
Ginh
Pconnect
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
31
3. RAIN Concepts ¾ But what if we instead add recurrent inhibition of inhibition? Pconnect Gexc
800 excitatory neurons
Pconnect Gexc 200 inhibitory neurons
Ginh
Pconnect
April 25, 2007
Ginh Pconnect
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
32
3. RAIN Concepts ¾ Work of others 9 van Vreeswijk, C. & Sompolinsky, H. (1996) Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 315: 973–976
Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J. Physiol. (Paris) 94: 445– 463
April 25, 2007
Ext Stim
9 Brunel, N. (2000)
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
Inh:Exc Cond
33
3. RAIN Concepts ¾ Work of others (cont’d) 9 Vogels, T. P. & Abbott, L. F. (2005) Signal propagation and logic gating in networks of integrate-and-fire neurons. J. Neurosci. 25: 10786-95.
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
34
3. RAIN Concepts ¾ “Negative-pulse ignited” firing phase diagram: Gexc vs. Ginh
based on a combination of firing statistics, 4 consistent domains are discovered as excitatory and inhibitory conductances are covaried in separate experiments
extensive domain of self-sustained asynchronous irregular firing
R
April 25, 2007
N
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
35
3. RAIN Concepts ¾ Effect of Membrane Ion Channels
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
36
3. RAIN Concepts ¾ Firing characteristics
Cell 1 50 0
0
100 200 Instant. Freq (Hz) Cell 8
0
100 200 Instant. Freq (Hz) Cell 15
50 0
100 50 0
100 50 0
40 20 0
200 100 0
0
100 200 Instant. Freq (Hz) Cell 22
Cell 2 40 20 0
1000 500 0
100 50 0
0
100 200 Instant. Freq (Hz) Cell 9
0
100 200 Instant. Freq (Hz) Cell 16
0
100 200 Instant. Freq (Hz) Cell 24
100 200 Instant. Freq (Hz) Cell 31
0
0
100 200 Instant. Freq (Hz) Cell 32
50 0
0
100 200 Instant. Freq (Hz) Cell 39
100 200 Instant. Freq (Hz)
0
100 50 0
0
0
100 200 Instant. Freq (Hz) Cell 40
100 200 Instant. Freq (Hz)
0
100 50 0
200 100 0
100 50 0
100 50 0
Cell 4 50
0
100 200 Instant. Freq (Hz) Cell 10
0
100 200 Instant. Freq (Hz) Cell 17
500
50 0
Cell 3 40 20 0
0
0
100 200 Instant. Freq (Hz) Cell 11
0
100 200 Instant. Freq (Hz) Cell 18
50
0
0
0
0
100 200 Instant. Freq (Hz) Cell 25
100 200 Instant. Freq (Hz) Cell 33
100 200 Instant. Freq (Hz) Cell 41
100 200 Instant. Freq (Hz)
0
40 20 0
1000 500 0
100 50 0
100 50 0
Cell 5 100 50 0
0
100 200 Instant. Freq (Hz) Cell 12
0
100 200 Instant. Freq (Hz) Cell 19
50
0
100 200 Instant. Freq (Hz) Cell 27
0
40 20 0
0
100 200 Instant. Freq (Hz) Cell 34
100 200 Instant. Freq (Hz) Cell 42
0
100 50 0
0
100 200 Instant. Freq (Hz) Cell 28
0
0
100 200 Instant. Freq (Hz) Cell 35
100 200 Instant. Freq (Hz) Cell 43
500 0
100 200 Instant. Freq (Hz)
0
100 50 0
0
40 20 0
100 50 0
0
100 200 Instant. Freq (Hz) Cell 13
0
100 200 Instant. Freq (Hz) Cell 20
0
0
100 200 Instant. Freq (Hz) Cell 29
100 200 Instant. Freq (Hz) Cell 37
0
100 200 Instant. Freq (Hz) Cell 45
0
100 200 Instant. Freq (Hz)
50 0
100 200 Instant. Freq (Hz)
0
Cell 7 50
50
50 0
Cell 6 100 50 0
0
100 50 0
100 50 0
100 50 0
40 20 0
0
100 200 Instant. Freq (Hz) Cell 14
0
100 200 Instant. Freq (Hz) Cell 21
0
100 200 Instant. Freq (Hz) Cell 30
0
100 200 Instant. Freq (Hz) Cell 38
0
100 200 Instant. Freq (Hz)
3. Progress to Date
sync
mean rate
spike raster
¾ Subthreshold correlation complexity (eigenvalue distribution):
subthresh complexity “Hellow World” @ 200-700 ms into 4 of 800 exc cells
3. Progress to Date ¾ Multi-RAIN “Transient Winner-Take-Most”
200 inhibitory neurons
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
39
3. Progress to Date ¾ Multi-RAIN Discriminate Hebbian Learning: clustering
Frontal expection
RAIN A dot_wave1
2 distinct sensory patterns
April 25, 2007
RAIN B
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
40
3. Progress to Date ¾ Multi-RAIN Discriminate Hebbian Learning: clustering Frey, B.J., Delbert, D. (2007) Clustering by Passing Messages Between Data Points Science 315: 973–976 200 cell RAIN network
1
cluster
4 6 7
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
“Key X” “Key Y” “Lock”
]
41
3. Progress to Date ¾ Multi-RAIN Discriminate Hebbian STDP Learning Rate Diffs (Hz) with 95% Confidence Intervals Baseline RAIN A & B activity:
No Pattern Injection 0.054 [0.027 0.081]
Response to Patt X Patterns into RAIN A & B, pre-STDP:
-1.81 [-1.85 -1.77]
Response to Patt Y -1.67 ]-1.71 -1.63]
Patterns into RAIN A & B, post-STDP: 10.29 [10.23 10.34]
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
-7.47 [-7.53 -7.41]
42
3. Progress to Date ¾ Robot’s visual input 9 coarse Gabor filters transduced into raster of spikes
VC
3. Progress to Date
PFC buffer
¾ At the core: visual-association-motor triad MC
9 the visual cortex (VC) provides the stimulus patterns, behavior 1 or “keys”
dot_wave1
AS
Locks
dot_wave1
behavior 2
....
do t_ w av e3
....
VC
Key
behavior N
SC reward
inhib. cells
9 the association area (AS) holds the complex neurodynamics in “locks” 9 the motor cortex (MC) is the AS “readout”: it coalesces the activity into winner-take-all
3. Progress to Date ¾ RAIN drives Robot’s behavioral output MC stand up
bark
9 raster of spikes transduced into stereotyped actions
present paw step back lie down growl
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
45
3. Progress to Date ¾ Completed sensorimotor loop between cluster and robot 9 initial attempt to implement a real-time, embedded neural robot c) a robot (military sentry, industrial assistant, etc.) interacts with environment and humans via sensors & actuators a) NeoCortical Simulator (NCS) software runs on computer cluster; contains the brain architecture for decision-making and learning b) “brainstem” laptop brokers WiFi connection: transmits multimodal sensory signals to NCS; sends actuator commands to robot
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
46
3. Progress to Date ¾ Social robotics paradigm 9 Typical robot/human interaction and learning scenario
a. resting
b. alerted, observing
c. reacting by offering a paw April 25, 2007
d. receiving reward (stroking)
e. angry barking and walking (no reward!) 47 Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
Large-Scale Cortical Mesocircuit Dynamics
1. Scope of Work
PFC AV
AS
MC
SC
2. Neurodynamics & Mesocircuit Theory 3. RAIN Concepts & Progress to Date
3200 EXC
800 INH
4. Future Directions April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
48
4. Future Directions Remote-RAIN Brain control of MDS (Mobile/Dexterous/Social) Robot Developed by a collaboration of: 1. Cynthia Breazeal (MIT Media Lab) 2. Xitome (MIT spinoff) 3. Rod Grupen (U Mass Amherst) Expressive face design to support the robot’s social function (Xitome)
2 DoF hands with tactile sensing developed at MIT (based on SDM technique developed by Aaron Dollar and Rob Howe at Harvard)
uBot-5 chassis from U Mass—1/2 meter tall • 11 DoF inverted pendulum mobility base • Trunk rotation and two 4 DoF arms • 11-channel embedded FPGA controller • Force and position feedback & series-elastic actuators • API in C++
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
49
4. Future Directions ¾ NCS development & robotics
optimization of RAIN mesocircuit network design and STDP
addition of realistic dopamine-reward basal ganglion (subcortical SC) module for reinforcement learning & planning
humanoid social interactive robotics
mesocircuit application to other domains (eg, speech & threat recognition)
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
50
4. Future Directions DURIP 2007: optimizing architecture for neural simulation Additional CPUs: 1.
Distributed, inexpensive cluster
2.
New “less expensive” shared-memory architectures
Faster inter-CPU communication options: 1.
Infiniband (switched, theoretically up to 96 Gbit/s throughput)
2.
LightFleet’s “Corowave” direct optical interconnect (32-cpu this summer)
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
51
4. Future Directions Collaboration with Mind Brain Institute, EPFL, Lausanne •
•
Basic science: ongoing biological extraction of neural parameters
Computer science: 1.
Use of IBM BlueGene L: 8096-CPU cluster, 22 Trillion Flops
2.
Runs NCS and NEURON (and a hybrid)
3.
Plan to run remote-brain AIBO this summer
4.
& MDS robot by summer 2008
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
52
Q&A
April 25, 2007
Goodman, P. H. & Doursat, R. - Cortical Mesocircuit Dynamics
53
