Décision et fonction exécutive préfrontale The Enlightened Brain
Etienne Koechlin Institut National de la Santé et de la Recherche Médicale, Université Pierre et Marie Curie, Ecole Normale Supérieure, Paris, France.
The prefrontal cortex Prefrontal cortex
Human brain
Prefrontal Executive Function
Sensory signals
Actions
Sensory signal
Sensory signal
Action-2 Action-1
Action
Executive Control in the Human Prefrontal cortex ______________________________
Cognitive « I choose an apple »
Affective « I like apples »
Motivational « I want an apple »
Coronal section Summerfield & Koechlin, in press
How the PFC function achieves action selection? « Cognitive control »
How the engagement of the PFC function is driven? « Motivation »
Hypothesis: the LPFC subserves cognitive control l o tr n o » c e on v i i t t i c n le g Co « Se
PFC coronal section
Cognitive Control and Information Theory Total Information for selecting action A
Q(A) = =
Mutual Information between Signal S & action A
I(S,A) P(S,A) log2 P(S)P(A)
Remaining Information for selecting action A unrelated to signal S
+
Q(A|S)
+ [- log2 P(A|S)] Measures Cognitive Control …
Predictions Reaction Times
• RTs ~ Q(A) = I(S,A) + Q(A|S) fMRI Activations
• LPFC ~ Q(A|S) •PM ~ Q(A) =
I(S,A) + Q(A|S)
Experimental Protocol Instruction cues
S’12
ICi
Q(A|S)
S1 S2 S3 S4 S4 S6 S7 S8 S9 S10 S11 S12
ICj
S2’’
2.4 sec
I(S,A) Left Right Response
Times
Behavioral results Reaction Times ms I(S,A) = 1 bit I(S,A) = 0 bit
Q(A|S) (bit)
Koechlin, Ody, Kouneiher, Science, 2003
fMRI signal changes
fMRI data
Ant. LPFC
Pos. LPFC
I(S,A) = 1
Premotor I(S,A) = 0
Q(A|S) (bit)
Koechlin et al., Science, 2003
The model
LPFC
Cognitive control
Q(A|S) I(A,S)
PM
Sensory control Q(A)= I(A,S) + Q(A|S)
(S)timulus
(A)ction
Is Cognitive Control fractionable?
Back to Information Theory
Cognitive control
Q(A|S) = I(C,A|S) Q(A) =
Episodic control
Contextual control
I(S,A)
+
Q(A|S,C)
+
Q(A|S)
The cascade architecture Q(A|S,C)
Episodic (diachronic)
A. LPFC
Cognitive control
Q(A|S,C) I(C,A|S)
P. LPFC Contextual (synchronic) Q(A|S)
I(A,S)
PM
Sensory control Q(A)= I(A,S) + Q(A|S)
Past events
(S)timulus (C)ontext
Action
Predictions Reaction Times
• RTs ~ I(S,A) + I(C,A|S) +Q(A|S,C) fMRI Activations
•A. LPFC ~ Q(A|S,C) •P. LPFC ~ I(C,A|S) +Q(A|S,C) •PM ~ I(S,A) + I(C,A|S) +Q(A|S,C)
Experimental Protocol Context: I(C,A|S)
Instruction cues: Q(A|S,C)
S’12
ICi
S1 S2 S3 S4 S4 S6 S7 S8 S9 S10 S11 S12
ICj
S2’’
2.4 sec
I(S,A) Left Right Response
Times
Behavioral results RTs ms
Reaction Times Reaction Times I(C,A|S) = 1 bit
I(C,A|S) = 0 bit
Q(A|S,C) (bit) Koechlin, Ody, Kouneiher, Science, 2003
fMRI signal changes
Ant. LPFC
I(C,A|S) = 1 I(C,A|S) = 0
Pos. LPFC
Premotor
Q(A|S,C) (bit)
Koechlin et al., Science, 2003
Contextual vs. episodic control in the LPFC PreMotor Caudal ongoing episode
Rostral
Temporal dimensions of cognitive control Synchronic
Diachronic
Experimental Protocol: cognitive factors
Instruction cues ICi
Context S1 S2 S3
x
S4 S5
x
S6 S7 S8
x
S9
Episodic Left resp.
Left resp.
Right resp.
Context ICi
S1 S2 S3
x
S4 S5
x
S6 S7 S8
x
S9
Contextual Left resp.
ICi
S1 S2 S3
x
S4 S5
Left resp. Right resp.
x
S6 S7 S8
x
S9 Times
Left resp. Distractor
Left resp. Right resp. Task
2.4 sec
Baseline
Contextual vs. Episodic control Episodic control
Contextual control
Trials
Kouneiher, Charron, Koechlin, Nature Neurosci., 2009
Is cognitive control further fractionable ?
Yes …
Episodic vs. branching control PreMotor
pending episode
past episode
ongoing episode
Polar
Rostral
Caudal
Temporal dimensions of cognitive control Synchronic Koechlin et al., 1999: Nature; 2000: PNAS. Koechlin & Hyafil, 2007, Science.
Polychronic
Diachronic
Experimental paradigm Delayed Contextual Episodic Branching performance control control control
TABLET Control
... A ...
B
E
T
Delay
... A
B
L
t
E
e
A
a
L
L
T ...
T ...
√
√
"T?"
Dual-task
... A
B
L
t
e
a
L
T ...
e
a
L
T ...
√
√
√
√
"t?"
Branching
... A
B
L
t
√
√
"t?"
...
... Scans 3s
Caudal LPFC
Rostral LPFC
Polar LPFC
Koechlin et al., Nature 1999
Experimental Paradigm Random ... A B L t e a L T b E a b T B t E e l t... t?
t?
t?
t?
t?
TABLET
Predictive ... A l b T e a B a t E t b L e e A t a B ... t?
t?
t?
t?
t?
t?
Scans
Baseline Repetitive Stimulus-Response associations
Koechlin et al., PNAS 2000
Neurocomputational model Active task
Mfc
Fpc
Lpc
Koechlin, Hyafil 2007, Science
Pending task
Expected rewards
ng episode
Contextual control: Hierarchical levels
oing episode
Superordinate chunk Chunk 1
A12
A23 A21
A22
A23 A31
A32
A33
Hierarchical dimensions of cognitive control Synchronic Hierarchical
A11
Chunk 3
Chunk 2
Polychronic
Diachronic
0,4
start
increment
stop
BA 45
0,2
Simple chunk performance
Event-related Signal change fMRIMR signal changes
0 -0,2
Posterior BCA (BA44)
0,6
B
BA 44
0,4 0,2
Chunk
0
BA 6
-0,2
Premotor (BA6)
0,6
C
A11
A12
A23
0,4 0,2 0 0 -0,2
5
10
15 0
5
10
15 0
Time (sec)
5
10
15
Koechlin & Jubault. 2006, Neuron Jubault, Ody & Koechlin, 2007, J.Neurosci.
0,4
start
increment
stop
BA 45
Superordinate chunk performance
0,2
Event-related Signal change fMRIMR signal changes
0 -0,2
Posterior BCA (BA44)
0,6
B
BA 44
0,4
Superordin. chk
0,2 0
BA 6
-0,2
Premotor (BA6)
0,6
C
Chk1 Chk2 Chk3 SA SA SA
SA SA SA
SA SA SA
0,4 0,2 0 0
5
10
15 0
5
10
15 0
Time (sec)
5
10
15
Koechlin & Jubault. 2006, Neuron Jubault, Ody, Koechlin, 2007, J. Neurosci.
Summary: cognitive control • Cognitive control is organized as a cascade of top-down selection processes from posterior to anterior LPFC regions. • Contextual control is implemented in posterior LPFC • Episodic control is implemented in anterior PFC • Both contextual and episodic control are further fractionable into two control levels. • Conditional entropy Q(A/lower level signals) measures selection demands at each control level.
What drives the engagement of cognitive control in the LPFC ? Motivation …?
Psychological theory of motivation (Hull, 1943) Excitatory potential of action i
Ei = Pi x D Frequency of action i
Global incentive factor
Motivation has ambiguous effects on action selection: beneficial or detrimental
Cognitive vs. Motivational Selective information Conditional entropy
Incentive values Free-energy
(Koechlin et al., 2003)
(Friston et al., 2007) 2
2,5
1,8
1,6
2
Ai
low entropy
1,4
large free-energy
1,2
1,5
1
0,8
1
large entropy
0,6
0,4
0,5
low free-energy
0,2
0
0
Distribution of neuronal activity over alternative options
Ai A Entropy = - ∑ log i ∑ Aj ∑ Aj
Free-energy = log ∑Aj
Statistical Physics of executive function Total Energy = Entropy + Free-Energy Lateral prefrontal activations
Cognitive control demands
Motivational control
Hypothesis: The medial PFC subserves motivational control
C
e v i t i n og
l o r t n co
e re F +
gy r e n E
Motivational control
Prefrontal cortex Coronal section
Hypotheses • Medial PFC regulate the engagement (i.e. free-energy) of lateral PFC regions in cognitive control according to rewards/penalty at stake in action, independently of control demands (i.e. conditional entropy).
• The organization of motivation in the medial PFC parallels the architecture of cognitive control in the lateral PFC: To each cognitive control level corresponds a medial region modulating its free-energy
The dual model
Kouneiher, Charron, Koechlin, 2009, Nature Neurosci.
Predictions • Pre-SMA exhibits transient activations varying as the reward/penalty at stake in immediate action (contextual motivation). • dACC exhibits sustained activations representing the reward/penalty at stake in subsequent action (episodic motivation). • Interactions from medial to lateral PFC regions convey contextual and episodic motivation values. • Post-LPFC activations show additive effects of contextual motivation and control, whereas anterior LPFC activations show additive effects of episodic motivation and control. • top-down interactions from anterior to posterior LPFC reflect episodic control and motivation
Experimental Protocol: cognitive factors
Instruction cues ICi
Context S1 S2 S3
x
S4 S5
x
S6 S7 S8
x
S9
Episodic Left resp.
Left resp.
Right resp.
Context ICi
S1 S2 S3
x
S4 S5
x
S6 S7 S8
x
S9
Contextual Left resp.
ICi
S1 S2 S3
x
S4 S5
Left resp. Right resp.
x
S6 S7 S8
x
S9 Times
Left resp. Distractor
Left resp. Right resp. Task
2.4 sec
Baseline
Contextual vs. Episodic control Episodic control
Contextual control
Trials
Kouneiher, Charron, Koechlin, Nature Neurosci., 2009
Experimental Protocol: contextual motivation +200%
ICi
S1 S2 S3
x
Low contextual motivation
+5%
ICi
S1 S2 S3
x
Low contextual motivation
+200%
S4 S5
x
+200% +200%
S6 S7 S8
x
Large extra pay-offs (2 euros)
S9
High contextual motivation +5%
S4 S5
x
+5%
+5%
S6 S7 S8
x
Low contextual motivation
S9
negligible extra pay-offs (5 cents)
Behavioral performances Blocks with large bonus trials Blocks with low bonus trials
Standard Bonus
Standard Bonus
Kouneiher, Charron, Koechlin, Nature Neurosci., 2009
Contextual motivation: fMRI data Medial
Lateral Effective connectivity
Standard Bonus Standard Bonus
Trial type Blocks with large bonus trials Blocks with low bonus trials Kouneiher, Charron & Koechlin, Naturre Neurosci. 2009
Experimental Protocol: episodic motivation +200%
ICi
S1 S2 S3
x
Low contextual motivation
+5%
ICi
S1 S2 S3
x
Low contextual motivation
+200%
S4 S5
x
+200% +200%
S6 S7 S8
x
High episodic motivation
S9
High contextual motivation +5%
S4 S5
x
+5%
+5%
S6 S7 S8
x
Low contextual motivation
Low episodic motivation S9
fMRI data: Episodic motivation Episodic motivation
Contextual motivation
Standard Bonus Standard Bonus
Trial Trialtype type
Kouneiher, Charron, Koechlin, Nature Neurosci., 2009
High incentive blocks Low incentive blocks
Control + Motivation Episodic motivation (sustained effect)
Contextual motivation (transient effect)
Kouneiher, Charron, Koechlin, Nature Neuro 2009
Reaction times (ms)
Control + Motivation
710 700
Episodic blocks
690 680 670 660
710 700
Contextual blocks
690 680 670 660
650
Baseline blocks
640 630 620 610 600
Blocks with large bonus trials Blocks with low bonus trials
Standard Bonus Trials
Motivation and control in the PFC
Standard
Bonus
Standard
Bonus
Standard Bonus
To conclude…
• The prefrontal executive function is organized as two parallel, hierarchical systems from posterior to anterior regions in the medial and lateral PFC .
• Post- and mid- lateral PFC select action sets according to immediate contextual signals (contextual control) and temporally remote events (episodic control) respectively, operating through a cascade of top-down interactions towards premotor cortex.
• Post- and mid-. medial PFC evaluate immediate contextual incentives and temporally remote incentives for weighting through medial-to-lateral interactions the involvment of contextual and episodic control, respectively.
• The dichotomy between control and motivation in prefrontal executive function is captured by the basic distinction between the concept of entropy and free-energy in population of neurons.
• Motivation enhances prefrontal control rather than improving selection!
