Revealing functional organization by modeling fMRI responses to natural stimuli Sam Norman-Haignere HHMI Postdoctoral Fellow of the Life Sciences Research Foundation École Normale Supérieure Marseille, 03/14/2018
Standard Approach Example of standard approach: “Pitch-responsive” regions fMRI Pitch Responses in Example Human Subject
Average fMRI Response of Human Pitch Region
R
50
0.6 0.4 0.2 0
10-6
oi
p = 10-3
se
s
1.2 2.4 Frequency (Hz)
0.8
N
60
N=12
1
% Signal Change
Power (dB)
Tones > Noise Harmonic Tones Freq-Matched Noise
To ne
Simulated Excitation Pattern
Norman-Haignere et al., 2013, J Neurosci
Limitations of Standard Approach Only test small number of human-intuitive hypotheses (e.g. pitch)
Þ
Many possible ways cortex could be organized
Þ
Organization could be counter-intuitive
Alternative Data-Driven Approach 1.
Measure responses to large set of 165 natural sounds
Þ 2.
Vary on many ecologically relevant dimensions
Use statistical criteria to search large space of possible response patterns
Þ
Not dependent on prior functional hypotheses
Norman-Haignere et al., 2015, Neuron
50 of the 165 Sounds in Experiment (each 2-seconds) 1.Man speaking 20. Zipper 39. Crumpling paper 2.Flushing toilet 21. Cellphone vibrating 40. Siren 3.Pouring liquid 22. Water dripping 41. Splashing water 4.Tooth-brushing 23. Scratching 42. Computer speech 5.Woman speaking 24. Car windows 43. Alarm clock 6.Car accelerating 25. Telephone ringing 44. Walking with heels 7.Biting and chewing 26. Chopping food 45. Vacuum 8.Laughing 27. Telephone dialing 46. Wind 9.Typing 28. Girl speaking 47. Boy speaking 10. Car engine starting 29. Car horn 48. Chair rolling 11. Running water 30. Writing 49. Rock song 12. Breathing 31. Computer startup sound 50. Door knocking 13. Keys jangling 32. Background speech 14. Dishes clanking 33. Songbird 15. Ringtone 34. Pouring water Scanned 10 human subjects 16. Microwave 35. Pop song English speakers not active musicians 17.All Dognative barking 36.and Water boiling 18. Walking (hard surface) 37. Guitar
Voxel Responses For each voxel, we measure it’s average response to each sound 3 2 1 0 20
60
100
140
20
60
100
140
20
60
100
140
3 2 1 0
Response % Signal Change
•
3 2 1 0
All 165 Sounds
Data Matrix
165 Sounds
11065 Voxels
Response Magnitude
How can we discover structure from this matrix?
Data Matrix
165 Sounds
11065 Voxels
Response Magnitude
Two assumptions motivating a simple linear model:
1. Neuronal populations with distinct responses to the sound set 2. fMRI signal reflects summed response of neuronal populations in each voxel
Linear Model of Voxel Responses r1
Sounds
v
=
w1 +
r2
w2 +
r3
w3 +
…
Response Magnitude
Voxel responses modeled as weighted sum of response profiles
v=
N X i=1
ri w i
Matrix Factorization
⇡ Response Magnitude
Response Profile 1 Response Profile 2 …
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Voxel Weight Vector 1 Voxel Weight Vector 2 …
Number of Components
Factor response matrix into set of components, each with:
1. Response profile to all 165 sounds 2. Voxel weights specifying contribution of each component to each voxel
Factor response matrix into set of components, each with:
1. Response profile to all 165 sounds 2. Voxel weights specifying contribution of each component to each voxel
Matrix Factorization
⇡ Response Magnitude
Response Profile 1 Response Profile 2 …
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Voxel Weight Vector 1 Voxel Weight Vector 2 …
Number of Components
Factor response matrix into set of components, each with:
1. Response profile to all 165 sounds 2. Voxel weights specifying contribution of each component to each voxel
Matrix Factorization
⇡ Response Magnitude
Response Profile 1 Response Profile 2 …
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Voxel Weight Vector 1 Voxel Weight Vector 2 …
Number of Components
1. No information about sounds or anatomy used 2. Consistent functional or anatomical structure must reflect structure in data
Matrix Factorization
⇡ Response Magnitude
Response Profile 1 Response Profile 2 …
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
How Many Components?
Voxel Weight Vector 1 Voxel Weight Vector 2 …
Voxel Prediction Accuracy vs Number of Components Correlation (r)
0.5
N=6 Over-fitting
0.4
0.3
3 6 9 12 15 # Principal Components
Prediction accuracy best with top 6 principal components
Þ Higher-order components driven by noise Þ Explain >80% of replicable response variance across natural sounds Þ Stimulus set high-dimensional but fMRI responses low-dimensional
Matrix Factorization
⇡
Response Profile 1 … Response Profile 6
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Voxel Weight Vector 1 … Voxel Weight Vector 6
6 Components Response Magnitude
Matrix approximation still ill-posed (many equally good solutions)
Þ Used two methods to constrain the solution via statistics of the voxel weights
Matrix Factorization
⇡
Response Profile 1 … Response Profile 6
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Voxel Weight Vector 1 … Voxel Weight Vector 6
6 Components Response Magnitude
Method 1: Non-parametric algorithm
Method 2: Probabilistic model
Search for components with maximally non-Gaussian weights, quantified using a non-parametric estimate of entropy
Find components which maximize likelihood of the data given a parametric prior (Gamma distr.) on voxel weights
(Hyvarinen, 2000)
(Liang, Hoffman, Mysore, 2014)
Probing the Discovered Components We now have 6 dimensions, each with:
1.
A response profile (165-dimensional vector)
2.
A weight vector, specifying its contribution to each voxel
Walking on Hard Surface stim399 walking on hard surface
stim108 telephone dialing
7204
5331
5331
3930
3930
3930
2882
2882
2882
2882
1073 745
2098 1512 1073 745
2098 1512 1073 745
500
500
316
316
316
179
179
179
76
76
76
1 Time (sec)
2
500
1 Time (sec)
2
stim33 bluegrass
2098 1512 1073 745
179 76 2
stim387 typing 9707
7204
7204
7204
7204
5331
5331
5331
5331
3930
3930
3930
3930
2882
2882
2882
2882
Frequency (Hz)
1512 1073 745
2098 1512 1073 745
Frequency (Hz)
9707
Frequency (Hz)
9707
2098
2098 1512 1073 745
1073 745
500
500
500
316
316
316
179
179
179
76
76
76
0
1 2 Time (s)
1 Time (sec)
2
Blender
1512
316
2
2
2098
500
1 Time (sec)
1 Time (sec)
stim31 blender
9707
10 1 0.1
2
316
Typing
stim150 finger tapping
1 Time (sec)
Crowd Cheering
500
1 Time (sec)
Finger taps
Bluegrass
Frequency (Hz)
9707
7204
5331
3930
Frequency (Hz)
9707
7204
5331 Frequency (Hz)
9707
7204
1512
179 76 1 Time (sec)
0.25
0.25
v
-0.7 0.7 Normalized Correlation (r)
stim98 crowd cheering
9707
2098
1
Temporal Modulation (Hz)
1512
316
Bigband Music
1
2098
500
76
2
stim254 orchestra music
9707
7204
2098
1 Time (sec)
Orchestra Music
9707
Frequency (Hz)
Frequency (Hz)
1073
2098
500
stim384 trumpet jazz solo
Frequency (Hz)
1512
2
1 Time (sec)
2
7 Impulse Response 5 3 1 1 Time (seconds)
2
Frequency (octaves)
1073
1512
2098
Frequency (octaves)
1512
2098
64
2882
2098
16
3930
2882
4
5331
3930
2882
1
7204
5331
3930
2882
64
7204
5331
3930
4
16
7204
5331
4
7204
v
4
1
9707
Frequency (Hz)
9707
Frequency (Hz)
9707
Trumpet
Frequency (Hz)
stim51 car accelerating
9707
1 Time (sec)
Freq (kHz)
stim399 walking on hard surface
stim401 walking with heels
Frequency (Hz)
Frequency (Hz)
stim394 violin
Component 4
Component 3
Walking on Car Hard Surface Accelerating
Walking with Heels
Violin
Correlation of Full Response Profile with Spectrotemporal Energy Measures Spectral Mod (cyc/oct)
Component 3 Highest Lowest Response Response
7 5 3 1 1 Time (seconds)
2
Tonotopy Measured with Pure Tones
R
Component 3
Component 4
R
R
L
L
L
0.2 6.4 Best-Frequency (kHz)
0.5 28.8 -2.4 46.9 Significance of Voxel Weight (-log10[p])
Fine spectral modulations / pitch (Comp 3) mapped more anteriorly Rapid temporal modulations (Comp 4) mapped more posteriorly
Component 5
Component 6 Music
Sound Categories Instr. Music Vocal Music English Speech Foreign Speech
Response Magnitude
Speech 2
2
1
1
0
0
NonSpeech Vocal Animal Vocal
Nature Mechanical Env. Sounds
Response Magnitude
Human NonVocal Animal NonVocal
All Sounds Tested
All Sounds Tested 2
Foreign Speech
2
English Speech 1
Music with vocals
Instrumental Music Music with Vocals
1
Vocalizations 0
0 Sound Categories
Sound Categories
Single Instrument (e.g. violin)
Multi-Instrument Ensembles (e.g. “Blues Band”)
Drum Solo
Tonotopy Measured with Pure Tones
Component 5
Component 6
R
R
L
L
R
L
0.2 6.4 Best-Frequency (kHz)
-5.9
76.3
-2.2
21.6
Significance of Voxel Weight (-log10[p])
Speech-selectivity lateral to primary auditory cortex Music-selectivity anterior and posterior to primary auditory cortex
Þ Suggests distinct non-primary pathways for speech and music
Can speech and music-selectivity be explained by tuning for modulation? Natural Sound
Model-Matched Sound
783 220
1.5
Frequency (Hz)
783 220
0.5
1.0
1.5
783 220
9.9 3.02792 0.8783 0.2220 .06620.0 0
2.0
Time (s)
2792
62 0.0 9948
2.0
Frequency (Hz)
Frequency (Hz) Frequency (Hz)
1.0
Time (s)
0.5
1.0
1.5
2.0
Time (s)
Frequency (Hz)
Freq (kHz)
Keyboard Typing
0.5
2792
62 0.0 9948
Walking in Heels
Frequency (Hz)
2792
62 0.0 9948
Violin
9948
Frequency (Hz)
Woman Speaking
Frequency (Hz)
9948
0.5
1.0
1.5
1 Time (sec) Time (s)
2.0
2
2792 783
Group
220 62 0.0 9948
0.5
1.0
1.5
2.0
1.5
2.0
1.5
2.0
Time (s)
2792 783 220 62 0.0 9948
0.5
1.0
Time (s)
2792 783 220 62 0.0 9948
0.5
1.0
Time (s)
LH
2792
RH
783
Normalized Squared Error (Noise-C
220 62 0.0
0.5
1.0
Time (s)
1.5
2.0
Measured
?
Known
Data Matrix
Response Matrix
Weight Matrix
Same Voxels/Subjects
=
Model-Match Stim
Model-Matched Stim
Model-Matched Stimuli
D = RW D WT(WWT)-1 = R
6 Comp
Same Voxels/Subjects
Model-Matched Stimuli
Speech Selective Component 4 Component 5
Model-Matched (au)
Tuning for Standard Acousitic Features Component 1
Component 2
Component 3
3 1.5 0 0 1.5 3 Natural Sounds (au)
Instr. Music Vocal Music NonSpeech Vocal Env. Sounds Mechanical
English Speech Foreign Speech Human NonVocal Animal NonVocal
Music Selective Component 6
Model-Matched Stimuli
Speech Selective Component 4 Component 5
Model-Matched (au)
Tuning for Standard Acousitic Features Component 1
Component 2
Component 3
3 1.5 0 0 1.5 3 Natural Sounds (au)
Instr. Music Vocal Music NonSpeech Vocal Env. Sounds Mechanical
English Speech Foreign Speech Human NonVocal Animal NonVocal
Music Selective Component 6
Model-Matched Stimuli
Speech Selective Component 4 Component 5
Model-Matched (au)
Tuning for Standard Acousitic Features Component 1
Component 2
Component 3
Music Selective Component 6
3 1.5 0 0 1.5 3 Natural Sounds (au)
Speech and music selectivity not explained by modulation model
Why has music selectivity not been clearly observed before?
Music
2
2
Instrumental Music Music with Vocals
1
1
0
0 All Sounds Tested
Sound Categories
Speech-Selective Component 5
0 Voxel Weight Selectivity/Purity
Music-Selective Component 6
0.6
0 Voxel Weight Selectivity/Purity
0.6
Music-Selective Component
% Signal Change
% Signal Change
Music-Selective Voxels
1
0.5
0
2
1
0 Instr. Music Vocal Music Nature
English Speech Foreign Speech Mechanical
NonSpeech Vocal Animal Vocal Env. Sounds
Human NonVocal Animal NonVocal
Conclusions from Part 2 1. Novel approach for discovering functional organization Þ Voxel decomposition of responses to natural sounds
=
Response Magnitude
Response Profile 1 Response Profile 2 …
11065 Voxels
165 Sounds
165 Sounds
11065 Voxels
Number of Components
Voxel Weight Vector 1 Voxel Weight Vector 2 …
Conclusions from Part 2 1. Novel approach for discovering functional organization Þ Voxel decomposition of responses to natural sounds 2. Six components explain responses throughout auditory cortex
Þ Each with interpretable response selectivity, despite lack of constraints Speech 2
1
0
All Sounds Tested
Music
Conclusions from Part 2 1. Novel approach for discovering functional organization Þ Voxel decomposition of responses to natural sounds 2. Six components explain responses throughout auditory cortex Þ Each with interpretable response selectivity, despite lack of constraints Þ Localized to specific anatomical region Component 3
Component 4
Component 5
Component 6
ral
Component 2
La te
Component 1
An t&
po st
Conclusions from Part 2 1. Novel approach for discovering functional organization Þ Voxel decomposition of responses to natural sounds 2. Six components explain responses throughout auditory cortex Þ Each with interpretable response selectivity, despite lack of constraints Þ Localized to specific anatomical region 3. Revealed music-selective component, not evident with standard methods Component 6 Music
2
1
0 All Sounds Tested
Conclusions from Part 2 1. Novel approach for discovering functional organization Þ Voxel decomposition of responses to natural sounds 2. Six components explain responses throughout auditory cortex Þ Each with interpretable response selectivity, despite lack of constraints Þ Localized to specific anatomical region 3. Revealed music-selective component, not evident with standard methods 4. And distinct non-primary pathways for music and speech Speech
Oct 19, 1998 - problem. Footprints in the Sand ..... active but rather to show the temporal characteristics ... first describes the properties of the VEPEG and com-.
apace at regional or global scales across a wide range of ecosystems, due to ... RLQ analysis: a three-table (R, abiotic variables; L, species abundances; Q, species traits) ... neutral model requires metrics that can detect disturbance impacts on th
1988). Does the organization of postural responses follow the same rules in ..... amplitude. 1.0. 190 m. E. 160 o ~ g m 130 a. 100. 70. A. 220. 190. E. 160 o ~.
generally considered to be a good indicator of lemming habitat quality .... calculate the numerical responses were obtained from ..... Human deposits. 0.1 (0.6).
was assayed in aortae by RNase protection assay. .... of endothelium removal in ''endothelium-free'' aortae. .... digest unbound label and unprotected mRNA.
Nov 15, 2008 - second-best macroeconomic policy management at a time when monetary and ... foundations of economic growth theory remain fully valid. .... tariffs fell over the period 1990-2006 both in major advanced economies (the ..... http://ec.eur
produced by contraction of the quadriceps was transmit- ted to the bar and ... trations of free fatty acids (FFA), glycerol, glucose, lac- tate, and proteins, and the ...
implications for management strategies. Simon Blanchet,1,2,* ... question is essential for ecological managers as it allows ... response to fragmentation was highly species-specific, with the smallest fish species (P. .... Material and methods.
Sep 24, 2015 - Part I: Mapping cognitive ontologies to brain function. â Part II: Statistical ... tools did you use? I am having memory .... BSD, Python, OSS.
Dec 12, 2005 - A most ancient concept of biology â Biology as the science of ... Organization (idea of organs, relatively automous parts) .... In progress ...
Mar 1, 2011 - affected (e.g. McKinney, 1997), we restricted our ap- proach to contemporary ..... sions are retained and summed for the 10 points of a square as a ..... Rouault G, Candau JN, Lieutier F, Nageleisen L-M, Martin J-C et al., 2006.
comparing subjects' responses to robotic emotional ..... In order to test the presence of a resonance effect, we ..... development of cultural intelligence: why.
preceding P200 to form a sharp peak, whereas this is not the. In contrast to ...... Electroencephalogr Clin Neurophysiol 1991; 79: 488â502. the basal ganglia.
(1989), force was measured in units of 0.613 x 10"2 N (0.625 g). The computer was ... The food delivery mechanism consisted of a 10 mL glass syringe mounted in a ... were corrected for the prevailing temperature and barometric pressure. (STP). .....
... detects frequent and large-scale dispersal in water voles. Molecular Ecology 12:1939-1949. ... Page 11 ... 2. Extinctions occur independently in different patches and local dynamics ... Am Nat 152:530-542. ... Alberts SC, Altmann J, 1995.
Mar 21, 1997 - Paramecium is a unicellular organism that possesses a specialized pathway for regulated secretion that is ... model system for the genetic dissection of this process. ... not a vital function under laboratory conditions, many secretory
May 24, 2005 - interface '92. Kaufmann, San ... rendering on large displays and virtual reality. He is currently ... work to infrared simulation and tunnel lighting.
ABSTRACT. Most systems of Automatic Language Identification are ... closed set identification task, the system reaches 85 % of correct ... corresponding with natural sound categories .... performing an on line detection of changes in the auto ...
Feb 1, 2010 - Dr. Serafin D. Talisayon, Philippines, served as the volume editor. The opinions expressed in ... Question-and-Answer Session. 33. Technical ...
emergent organizations, agent behavior, complex systems, genetic encoding ... 1. individuals representing potential solutions for the problem are built from a ...
