Université Lyon 1
Centre de Recherche en Neurosciences de Lyon
Olivier BERTRAND Dynamique Cérébrale et Cognition
DHSS - Ecole Polytechnique Cerveau et Cognition 25 novembre 2010
Interfaces Cerveau-Machine
Inserm U821 Lyon
Les Interfaces Homme-Machine (IHM) : Interactions indirectes entre le cerveau et la machine via une commande motrice. (interrupteur, souris, (interrupteur souris clavier, clavier capteurs de mouvement, mouvement orientation du regard, parole, …)
Les Interfaces Cerveau-Machine (ICM) : Interactions directes entre le cerveau et la machine, sans commande motrice. Mesure la plus utilisée : ll’activité activité électrique cérébrale Æ électrophysiologie en temps-réel
Inserm U821 Lyon
Real-time electrophysiology
To measure in real-time electrophysiological components (spike, LFP, EEG, MEG) specific to a particular mental process or state, state with multiple applications : Brain-Machine Interface (BMI) to control external devices • to restore communication in patients with strong motor disabilities
NeuroFeedback Training (NFT) and Rehabilitation • self-regulation of specific brain activities • domains : attention disorders, motor rehab., depression, epilepsy, pain, …
Basic Neuroscience • to better understand the « neural code » and brain plasticity y • dynamic manipulation of an experimental protocole according to brain state
Video games Video-games • enriched game-play or « serious games »
Inserm U821 Lyon
Publications
Brain-Computer Interface & Neurofeedback papers 150 125 100 75 50
3 25 0 1998
1999
2000
2001
2002
2003
2004
2005
2006
20 years ago : 3-4 groups world-wide years ago g : 6-8 g groups p 10 y 2009 : ~ 100 groups
2007
2008
Brain-Machine Interface – BMI (BCI)
Inserm U821 Lyon
feedback
closed-loop p BCI signal acquisition
translation to commands
real-time signal processing i
(feature extraction, classification)
Inserm U821 Lyon
Une convergence de disciplines Neurosciences fondamentales
Applications cliniques
Progrès technologiques
Inserm U821 Lyon
Electrophysiological signal recordings
Unit
Multi-U
Waldert, 2009
Inserm U821 Lyon
Electrophysiological signal recordings ElectroCorticoGram
Intracranial EEG
Spike, LFP
EEG
MEG
Inserm U821 Lyon
Brain-Machine Interface (BMI) • BMI as a communication aid without movement.
• Clinical goals Ö to restore communication and control to people with severe motor disorders: - amyotrophic lateral sclerosis (ALS) - spinal cord injury - muscular distrophies - brainstem stroke, locked-in syndrome. • Invasive or non-invasive BMIs • to learn how to associate a mental state to a desired action (appropriate mental processes and good markers ?) ¾ endogeneous processes: e.g., motor imagery ¾ exogenous processes: e.g., selective attention
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
Sensorimotor rhythms: mu (~10 Hz) and beta (15-25 Hz) Event-Related Desynchro. Mu rhythm desynchronization C3
100 uV
motion onset Pfurtscheller
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
Sensorimotor rhythms: motor execution mu (~10 Hz) and beta (15-25 Hz) ERD
motor imagination
Pfurtscheller
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
Pfurtscheller et al. (2006) Brain Res. Wolpaw and McFarland (2004) PNAS
Mental imagery of hand motion
EEG
Voluntary modulation of 15 H Hz (m (mu)) sensorimotor rhythm
EEG
Cursor control on a video-display ideo displa
This approach requires extensive training
ICM non-invasive (imagerie motrice)
Inserm U821 Lyon
• Phase de calibration (off-line) Instructions au sujet (imagerie main droite, droite main gauche) choix de la fréquence et des électrodes du rythme mu calcul du gain, ex : déplacement curseur/puissance du mu • Phase d’utilisation (on-line) estimation du mu en temps-réel (fenêtre ~ 0.5 à 1 s) transformation en temps-réel temps réel : puissance mu Æ déplacement du curseur • Problèmes d’adaptation p variabilité interindividuelle importante apprentissage difficile, recalibrations régulières
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
10-15 Hz mu rhythm y analysis y MEG rest
real right hand movement real left hand movement real movement
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
10-15 Hz mu rhythm y analysis y MEG rest imagination of right hand movement imagination of left hand movement imagination of movement
Non-invasive BMIs (motor imagery)
Inserm U821 Lyon
On-line measure of 15Hz mu : execution vs imagery
MEG
Non-invasive BMIs (motor imagery)
Foot vs hand movement (go/stop) in virtual reality environement after extensive BMI training
Inserm U821 Lyon
Galan et al 2008
Pfurtscheller et al 2006
Inserm U821 Lyon
Invasive BMI
Codage de la direction du mouvement
Inserm U821 Lyon
tuning curve broad direction tuning direction du mouvement
un neurone du cortex moteur primaire ((faible a b e sé sélectivité ec é à la ad direction ec o du mouvement) ou e e )
Georgopoulos et al., 1986
Codage de la direction du mouvement
Inserm U821 Lyon
« population vector »
une ligne noire = 1 neurone
direction du mouvement
Georgopoulos et al., 1986
prédiction possible de la direction du mouvement à partir d’une population de neurones
3D movement prediction from electrodes in p premotor,, primary p y motor,, and posterior p parietal cortical areas
Nature, 2000
Equipe Nicolelis, USA – Wessberg et al, Nature, 2000
3D BCI control of a prosthetic arm
Inserm U821 Lyon
Robotic arm control and grasping (5 degrees of freedom) with a micro-electrode array in the motor cortex
Velliste et al., 2008
Inserm U821 Lyon
Invasive BMIs in human
Cortical implant in complete tetraplegia
• Motor cortex implant, p g activity y • Records spiking from 100 neurones, • Intention-driven neuronal activity • Computes a linear model after training session • Cursor control • Short training period
Donoghue’s Team– Hochberg et al., Nature, 2006
Non-invasive BMIs
Inserm U821 Lyon
• Other BMIs for restoring communication stimulus-driven driven activity (exogenous processes): • stimulus Voluntary orientation of attention on a given stimuli S Specific f modulation off some evoked components P300, steady-state responses
Inserm U821 Lyon
P300 Speller BMIs Selective attention on the letter to select Ö P300 Task : to count the selected letter when it flashes (10 to 15) S i l : flashing Stimulus fl hi by b lines li or columns (period ~150 ms)
EEG P300 to target letter
non target t t Mattout et al., 2008 Maby et al., 2010
P300 Speller BMIs
Inserm U821 Lyon
Inserm U821 Lyon
P300 Speller BMIs Selective attention on the letter to select Ö P300 T d ff b Trade-off between t robustness and speed
~ 5 lletters tt /minute / i t
Current improvements : - Optimal spatial-filtering - Adaptive classification to optimize bit-rate - Word selection instead of spelling letter by letter (word prediction algorithm coming from mobile phone world)
Visual Steady-State response
Inserm U821 Lyon
Frequency tagging of visual stimuli and selective attention Un stimulus périodique engendre une réponse cortical périodique (même fréquence)
14 Hz
+ 17 Hz
14 Hz
17 Hz alpha
Kelly et al, 2005
Inserm U821 Lyon
Auditory Steady-State response Frequency tagging of auditory streams and selective selecti e attention
Auditory SSR - MEG
V. Attina, E. Maby, N. Weisz, J. Mattout, O. Bertrand
21 Hz SSR
29 Hz SSR
Low pitch
High pitch
29 Hz AM
21 Hz AM
29 Hz 21 Hz
Time-frequency plot of averaged SSR
Auditory SSR - MEG Orientation of attention Low pitch
High pitch
29 Hz AM
21 Hz AM
21 Hz Steady-State
29 Hz Steady-State 21 Hz
29 Hz
Auditory SSR - MEG Orientation of attention Low pitch
High pitch
29 Hz AM
21 Hz AM
21 Hz Steady-State
29 Hz Steady-State 21 Hz
29 Hz
Auditory SSR - MEG
LDA classification on 2-sec moving time-windows based on spectral power at f and 2f over temporal regions
L ft attention Left tt ti Right attention 30 sec
Auditory SSR as a possible marker f real-time for l ti attention tt ti monitoring/control it i / t l
Remarques sur les différentes BMI
Inserm U821 Lyon
• Imagerie mentale motrice non-invasif (EEG, (EEG MEG) : • difficultés d’apprentissage, concentration nécessaire • variabilité interinter et intra-individuelle intra individuelle • ~60% des sujets sont capables de faire la tâche invasif (micro-electrode (micro electrode array) : • apprentissage plus rapide • moins sensible aux autres activités mentales • Attention sélective (visuelle, auditive) non-invasif (EEG, MEG) : • peu d’apprentissage, plus naturel • débit lent, fatigue
Inserm U821 Lyon
Non-invasive BMIs • Other BMIs for restoring communication • endogeneous processus: other types of imagery
To consider various types of mental tasks (verbal, visual, spatial) t benefit to b fit from f better b tt contrasts t t due d to t hemispheric h i h i specialization i li ti To include source reconstruction and coherence/synchrony measures to improve selection of the most discriminant features.
e.g., visuo-spatial i ti l navigation imagery task
M. Besserve, PhD 2007
Magnetoencephalography and BMI ?
Inserm U821 Lyon
• MEG is obviously not an appropriate device for an operational ti l BMI dedicated d di t d to t communication i ti and d control t l in disabled (size, price, complexity, …) • MEG could certainly help to identify new markers for BMI, during an exploratory phase, that could then be adapted d t d to t EEG recordings. di MEG • MEG could be useful to efficiently train subjects to learn a BMI (good compromise between temporal and spatial resolution, and functional specificity). specificity) • MEG is quick to install, and non-invasive
Neurofeedback Training (NFT)
Inserm U821 Lyon
• Auto-regulation d’une composante de l’activité cérébrale (EEG) non-invasive invasive • Même technologie que les ICM non • Exemple de pathologies potentiellement concernées 9 déficits défi it attentionnels, tt ti l hyperactivité h ti ité 9 stress post-traumatrique 9 douleurs chroniques 9 épilepsie
• études controlées ? • marqueurs ? • mécanismes é i ?
fMRI Neurofeedback Training Control over brain activation and pain learned by using real-time functional MRI deCharms et al. PNAS, 2005 Delay of ~8s for the feedback Anterior cingulate (ACC)
• voluntary control over activation in a specific brain region (ACC), • leads to control over pain perception, • Impact on severe, chronic clinical pain.
Inserm U821 Lyon
Inserm U821 Lyon
MEG Neurofeedback Training Motor deficits after stroke MEG recordings
Task : Self-modulation of sensorimotor rhythm (mental imagery) Feedback by up-down motion of the cursor on the screen +
Feedback byy proportional p p motion of the orthosis on the p paralysed y hand
Æ Sensory input related to motor control Birbaumer, 2007
Inserm U821 Lyon
MEG Neurofeedback Training Motor deficits after stroke
Task : Self-modulation of sensorimotor rhythm (mental imagery) Feedback by up-down motion of the cursor on the screen +
Feedback byy proportional p p motion of the orthosis on the p paralysed y hand
Æ Sensory input related to motor control Æ Speed-up rehabilitation Birbaumer, 2007
Neurofeedback challenges Need to target:
Inserm U821 Lyon
specific brain activities, i specific in ifi brain b i regions, i related to specific brain processes.
Requires to better understand the mechanisms of pathologies g in terms of dysfunctioning y g regions, g , certain p networks and interactions. Requires R i t understand to d t d the th potential t ti l mechanisms h i off neural plasticity and cortical reorganization related to those pathologies. pathologies
Inserm U821 Lyon
Other applications of real-time electrophysiology
Real-Time Oscillatory Brain Mapping (Epilepsy) Real-time quantification of alpha,beta and gamma activity
BRAIN TV
Visual Display
Spectral Analysis
Lachaux et al. PLoS One, 2007
Intracranial EEG Data Acquisition
Collaboration Ph. Kahane, CHU Grenoble
Real-Time Oscillatory Brain Mapping BRAIN TV
Lachaux et al. PLoS One, 2007
« Each E h time you play l music, gamma goes up ! »
« If I sing, almost nothing …. »
No gamma when we speak
gamma for loud noises Little g
Video-Games applications
Inserm U821 Lyon
A growing field : real-time EEG for video-games (Nintendo, Sony, …., + several Silicon Valley companies ….) A opportunity An i for f the h development d l off new technologies. h l i
Video-Games applications
Inserm U821 Lyon
A growing field : real-time EEG for video-games (Nintendo, Sony, …., + several small companies ….) A opportunity An i for f the h development d l off new technologies. h l i
Inserm U821 Lyon
Multidisciplinary challenges new mental tasks new feedback
new signals (EEG, MEG (EEG MEG, iEEG LFP, spikes, others )
(neurostimulation)
BCI exemplesfs
new electrophysiological markers
new signal processing methods (noise reduction, source localization, adaptive methods, co co-adaptation) adaptation)
High-speed P300 speller
J. Mattout, E. Maby
Inserm U821 Lyon
Inserm U821 Lyon
Multidisciplinary challenges new mental tasks new feedback
new signals (EEG, MEG (EEG MEG, iEEG LFP, spikes, others )
(neurostimulation)
BCI exemplesfs
new electrophysiological markers Acceptability & Ethics
The ultimate goal
new signal processing methods (noise reduction, source localization, adaptive methods, co co-adaptation) adaptation)
Inserm U821 Lyon
Brain Dynamics and Cognition J. Mattout
E. Maby
P-E. Aguera
O Bertrand O.
F. Lecaignard
G Sanchez G.
C. Delpuech P. Bouchet
M. Perrin K. Jerbi
J-P. Lachaux
Inserm U821 Lyon
