Challenges in the analysis of EEG – How Open Source and Open Data can help

Robert OOSTENVELD – 8 Sept 2024 – Chengdu
Challenges in the analysis of EEG –
how Open Source and Open Data can help
Robert OOSTENVELD
robert.oostenveld@donders.ru.nl
8 Sept 2024 - Chengdu, China

Challenges in the analysis of EEG
EEG experiments are very flexible
EEG has very high temporal resolution
EEG is relatively affordable
EEG data is noisy
EEG data is highly multivariate
EEG datasets are large
Acquiring EEG is a lot of work
Many EEG analysis methods by different groups that aim for similar results
Not so many standardized pipelines

Open Science = better science
Open access publications
Open peer review
Pre-registration
Open educational resources
Open methodology
Open software
Open hardware
Open data

Open Science - improves quality, efficiency, inclusivity, reprodicibility, …

About me
Since about 20 years heading the Open Source FieldTrip toolbox.
From 2015 onward heading the Open Source EEGsynth artistic BCI toolbox.
From 2012 onward responsible for MEG in the Human Connectome Project
(HCP).
From 2014 onward involved in setting up the Donders and Radboud
institutional repositories, which now hold >450 public datasets.
From 2019-2023 member of the Brain Imaging Data Structure (BIDS)
steering group.

Why share your code as Open Source
Allows others to …
• re-use and replicate your findings.
• improve on your work and share their improvements back to you.
• to find bugs and errors in your work, so that you don’t continue in the wrong
direction.
Increases the reproducibility and impact of your work.
Reprodicibility is a major principle underpinning the scientific method.
Impact is what drives innovation and what you need to fund future research.

Sharing code can be simple … or complex
It can be as simple as adding your analysis code as appendix, supplementary
material to your paper, or sharing it online.
It can be as complex as setting up a long lasting project like FieldTrip or EEGLAB.

If you have not shared code before…
You can look at others’ code that you are using.
What makes it useful for you, what is it that you are missing?
Version control
Documentation on the goals -> the aim
A simple example to get started
Documentation on the details -> the implementation
A way to get in touch (email, forum, issue tracking system)

Learning from others - FieldTrip
FieldTrip is software that we wrote in the lab to work together
with a research team of about 5 people.
Not the aim to make a big project out of it, but just to get our own research done.
Others starting using it internally, guests researchers started using it, and they took it
home with them. Then others (whom we did not know) started using it …

Learning from others - FieldTrip
We initially started with version control using CVS, then SVN, then git.
Using the latest/greatest tool in general is smart.
In 2011 we published a paper that describes the toolbox.
This is cited a lot, which counts as impact.
We already used it in in-house teaching, hence we initially shared the tutorials and example data.
Knowing how to make and maintain a website is an important skill!
Each piece of code that was meant to be used by others requires documentation. Write and discuss
the function “help” and reference documentation before writing the actual code.
Get others involved. Email discussion list, GitHub issues.

FieldTrip also learned from others
Early collaboration with EEGLAB (Arno Delorme and Scott Makeig):
They knew how to deal with “users” to give training and provide documentation.
Early collaboration with SPM team (FIL, UCL):
They knew how to make code maintanable using version control, consistency,
dependencies.

Learning from others
Technical aspects of the code:
- Computer science and software engineering
Aspects of the documentation:
- Not very different from teaching material
- A “wiki” like system is easy to set up
- MarkDown documents are nowadays common
Aspects of community management and collaboration:
- Hard to learn, but you can look at others and copy
https://collections.plos.org/collection/ten-simple-rules/

Managing an Open Source project
Find a convenient place to host your code, documentation and community.
GitHub is designed for “social coding” and working together, not only between
developers, but also with users.
There are alternatives for GitHub, like GitLab or the Chinese counterpart Gitee. Make
sure that people can find it!

Open Data
Findable
Make your data available in a catalog or repository
with a persistent identifier (DOI, handle) and metadata
Accessible
Be explicit about data usage terms (agreement with downloader)
Interoperable
Make your data human and machine readable, e.g. BIDS
Reusable
Make sure you document enough details, e.g. “data descriptor” paper
that can be cited, along with citing our data -> measurable impact!
ilkinson et al. (2016) The FAIR Guiding Principles. Sci Data.

Making EEG data FAIR
The Brain Imaging Data Structure (BIDS) is a community initiative
to make data more FAIR and to improve data sharing
http://www.bids-standard.org/

What is BIDS?
BIDS is a way to organize your existing raw data
To improve consistent and complete documentation
To facilitate re-use by your future self and others
BIDS is not
A new file format
A search engine
A data sharing platform

Brain Imaging Data Structure (BIDS)
External reuse of data: publishing, sharing
Internal reuse of data: archiving, curation, collaborating
Fundamental for repositories like OpenNeuro, PublicNeuro, but also for our own
Donders Repository and as the basis for new analysis pipelines and workflow
development
https://openneuro.org
https://publicneuro.eu
https://data.donders.ru.nl
PublicNeuro.EU data.donders.ru.nl

Brain Imaging Data Structure

BIDS for EEG, MEG, iEEG, MRI, NIRS, PET, motion capture, …
Just a bunch of directories and files on disk.
No special software required (although tools are available).

BIDS for EEG, MEG, iEEG, MRI, NIRS, PET, motion capture, …
data/README
CHANGES
dataset_description.json
participants.tsv
/sub-01/anat/…
/sub-01/meg/…
/sub-01/eeg/sub-01_task-auditory_eeg.edf
/sub-01/eeg/sub-01_task-auditory_eeg.json
/sub-01/eeg/sub-01_task-auditory_channels.tsv
/sub-01/eeg/sub-01_task-auditory_events.tsv
/sub-01/eeg/sub-01_electrodes.tsv
/sub-01/eeg/sub-01_coordsystem.json
Actual EEG data
Directory structure
Metadata

BIDS “sidecar” files for metadata
see also https://github.com/bids-standard/bids-examples

BIDS “sidecar” files for metadata
see also https://github.com/bids-standard/bids-examples
1) represent otherwise missing data
2) make it easier to query/search
As example for EEG:
_participants.tsv and json
_sessions.tsv and json
_scans.tsv and json
_eeg.json
_channels.tsv and json
_electrodes.tsv and json
_coordsystem.json
_photos.jpg

+

Putting code and data together
Research group A: dataset A + analysis pipeline A = result A
Research group B: dataset B + analysis pipeline B = result B
The published results are somewhat comparable, but neither group knows whether
their findings generalize or whether their data or methods are the best.

Putting code and data together – sharing data
Research group A: dataset A + analysis pipeline A = result AA
dataset B + analysis pipeline A = result BA
Research group B: dataset B + analysis pipeline B = result BB
dataset A + analysis pipeline B = result AB

Putting code and data together – sharing data and code
Research group A: dataset A + analysis pipeline A = result AA
Research group B: dataset B + analysis pipeline B = result BB
Research group C: dataset A + analysis pipeline A = result AA
dataset B + analysis pipeline B = result BB

Putting code and data together – BIDS apps
Large data analyses require well-structured data and efficient tools.
An “BIDS app” is a way of organizing your code to make it easy for others
to use it on their data.
MyBidsApp <inputdir> <outputdir> <participant|group> ...
The input directory must be formatted as a BIDS data structure.
The output directory can be formatted as a BIDS derivative data structure.
The application runs either over all participants, or results in agregate group results.
You can combine the two levels by calling it twice.
Gorgolewski et al. (2017) PLoS Comput Biol. https://doi.org/10.1371/journal.pcbi.1005209

Putting code and data together – BIDS apps
BIDS apps are implemented as docker or singularity containers and can run on large
computational infrastructure, like HPC clusters, WeBrain, CBRAIN, NeuroDesk,
BrainLife, etc.
Since EEG datasets can be large, transferring and maintaning a copy of the data
(and results) is not always feasible.
BIDS apps allow you to bring the (standardized) computation
towards the (standardized) data.
Dong et al. (2021) WeBrain. NeuroImage
Hayashi et al. (2024) Brainlife. Nature Methods
Renton et al. (2024) NeuroDesk. Nature Methods

Challenges in the analysis of EEG
Sophisticated algorithms are very detailed and hard to (re)implement.
Sharing code makes reuse efficient and possible.
Open source makes it possible to collaborate, investigate and improve details.
Data is getting more complex, good quality data is hard and expensive to acquire.
Well-curated existing data is invaluable for reuse!
The BIDS standard ensures data to be FAIR, organized and documented.
Being able to write your own algorithms and acquiring your own data
are still important skills.
Open Source allows early career students to get a head-start.
Open Data allows early career students to start with their research straight away.

Challenges in the analysis of EEG –
how Open Source and Open Data can help
Robert Oostenveld
robert.oostenveld@donders.ru.nl
8 Sept 2024 - Chengdu, China

Challenges in the analysis of EEG – How Open Source and Open Data can help

More Related Content

Similar to Challenges in the analysis of EEG – How Open Source and Open Data can help (20)

More from Robert Oostenveld (20)

Recently uploaded (20)

Challenges in the analysis of EEG – How Open Source and Open Data can help

Editor's Notes