Brain Control Club progress meeting Project: Introduction and Projects
- 1. First progress meeting Brain Control Club 4-5-2017 CRI http://cri-paris.org/scientific-clubs/brain-control-club/
- 2. Organization & Members • Group leader: Ignacio Rebollo (CRI) • Communication: Hernan Anllo (CRI) • Finance: Claire Baudelet (CRI) • Mentors: Dr. Stephen Whitmarsh (ENS) Dr. Guillaume Dumas (Pasteur) Prof. Robert Oostenveld (DCCN, KI) • Approx. 12 CRI students working on 4/5 projects
- 3. Brain Control Club Goal • Understanding theory and applications of BCI • Through hands-on creation of BCI’s Approach • Group projects on a specific BCI application • Sharing knowledge within and between projects Support • Autodidactic • Educational tools and material • Necessary hardware and software • Hackathons • Mentorship • Invited experts
- 4. Brain Control Club Goal • Understanding theory and applications of BCI • Through hands-on creation of BCI’s Approach • Group projects on a specific BCI application • Sharing knowledge within and between projects Support • Autodidactic • Educational tools and material • Necessary hardware and software • Hackathons • Mentorship • Invited experts Goal • Status of each project • Accomplishments so far • Share knowledge between projects • Identify obstacles Projects • Nirvana • Morpheus • EEG-game • EEGsynth.org • EEG-composition (NEW) Progress meeting
- 6. Background • Bodily signals (heart, stomach, respiration, brain) interact with each other during different mental/bodily states • Brain alpha (10Hz) power fluctuations are associated with vigilance fluctuations, together with changes in respiration and heart rate • Understanding their coupling in real time could allow us to detect certain mental states • Although these processes are normally unconscious, explicit knowledge of them (e.g. sounds), allow us to exert some control over them, a technique known as biofeedback
- 7. Objectives 1. Develop tools for online tracking of mental states such as vigilance, concentration or stress 2. Use biofeedback to help users attain or maintain a desired mental state
- 8. Progress so far 1. Software • MATLAB for processing raw incoming data • EEGsynth.org modules • Openbci2ft (monopolar, bipolar), • EEG playback 2. EEG instrumentation and hardware • EEG recording • OpenBCI architecture • EEG recording by Raspberry Pi 1. Testcase • Play the sound of the power of alpha based on pre- recorded dataset
- 9. Next steps 1. Acquire multimodal 20 minutes dataset (EGG, ECG, EEG) for playback 2. Update Openbci2ft.c to allow recording from 8 to 16 channels 3. Online detection of eye blinks and saccades 4. EGG module (online detection of 0.05Hz waveform) 5. Heartbeat module 6. Physiological connectivity modules (e.g. Heart – Brain coupling)
- 10. Obstacles so far Signal quality and preprocessing • Amplifier saturates sometimes • OpenBCI GUI shows good data, but data received in MATLAB seems very noisy • Optimal acquisition parameters for the EEGsynth? • Optimal filter settings (using MATLAB)?
- 11. Project Morpheus
- 12. Background
- 13. Targeted memory reactivation (TMR) Background
- 14. What happens in stage 2?
- 15. Hypothesis
- 16. Hypothesis
- 19. • do task with many associated sounds • separate sounds into 4 categories and play them during sleep: • sounds after deltas • sounds during spindles • no sounds • non-specific (random) sounds • test performance on task Hypothesis
- 20. Plan Task design • obtain object-sound pairs • code task with GUI EEG analysis • get signal in real time • make sounds respond to detected events • record sleep data • make online detection work using pre-recorded data (replayed) • record exact timing of sounds to future analysis • test sleep session: 4 sounds only (no task) – analyse and debug Experiment • recruit ~5-10 subjects • test • analyse data :))
- 21. Plan Task design • obtain object-sound pairs • code task with GUI EEG analysis • get signal in real time • make sounds respond to detected events • record sleep data • make online detection work using pre-recorded data (replayed) • record exact timing of sounds to future analysis • test sleep session: 4 sounds only (no task) – analyse and debug Experiment • recruit ~5-10 subjects • test • analyse data :))
- 22. Brain-based music composition Liburn Jupolli & Stephen Whitmarsh
- 23. Accent marks in music composition staccato staccatissimo marcato accent tenuto
- 24. McFadden KL, Steinmetz SE, Carroll AM, Simon ST, Wallace A, et al. (2014) Test-Retest Reliability of the 40 Hz EEG Auditory Steady-State Response. PLOS ONE B. Ross et al. J (2005) Stimulus Induced Desynchronization of Human Auditory 40-Hz Steady-State Responses. Neurophysiol. Kaongoen, Jo (2017) The effect of selective attention on multiple ASSRs for future BCI application. 5th International Winter Conference on Brain-Computer Interface (BCI) Auditory Steady-State response
- 25. Treble (guitar) Bass (guitar) EEGsynth 45Hz AM 35Hz AM Amplitude Amplitude 35Hz EEG 45Hz EEG Schematic design
- 26. Treble (guitar) Bass (guitar) EEGsynth 45Hz AM 35Hz AM Amplitude Amplitude 35Hz EEG 45Hz EEG attention Schematic design
- 27. Treble (guitar) Bass (guitar) EEGsynth 45Hz AM 35Hz AM Amplitude Amplitude 35Hz EEG 45Hz EEG attention Schematic design
- 28. Plan: • Use outputCVgate device to create 0-10v modulation at 35/45 Hz. Use VCA to create amplitude modulation • Analyze relative difference in EEG amplitude at steady-state freq. Modulate VCA amplitude with outputCVgate device So far: • Creating outputCVgate device
- 29. EEG-synth project Per Huttner Jean-Louis Huhta Robert Oostenveld Stephen Whitmarsh and many more…
- 31. Some highlights in recent progress • Thoroughly tested EEGsynth installation instructions for Windows, OS-X, Linux (Raspian, Mint, Redhat) • Pipeline for creating EEG music composition scores • Plug-and-play(back) of EEGsynth on Raspberry Pi • Live light control by EEGsynth on (Neopixel and DMX)
- 33. Offline playback of control signals based on EEG
- 34. Composition & MIDI score based on EEG using LilyPad
- 36. EEG-controlled light in visual art installation March 16th – April 16th
- 37. EEG-controlled light in visual art installation March 16th – April 16th
- 38. Portable EEGsynth costume by Emilia Rota, Stockholm