Developing and sharing tools for bioelectromagnetic research
0 likes31 views
The document outlines the development and sharing of tools for bioelectromagnetic research, focusing on the history and collaboration within the field. It highlights challenges like the reproducibility crisis and publication bias, advocating for open science practices to improve research quality and efficiency. Key tools and methods, including fieldtrip, are discussed in the context of ethical research data management and collaboration among various stakeholders.
Developing and sharing tools for bioelectromagnetic research
- 1. Developing and sharing tools for bioelectromagnetic research Robert Oostenveld Donders Institute, Radboud University, Nijmegen, NL NatMEG, Karolinska Institute, Stockholm, SE
- 2. Short history of live, and everything… 2002 completed PhD thesis 2002 start Donders Centre for Cognitive Neuroimaging installation of MEG and MRI scanners 2003 internal sharing and coordination of code with Pascal Fries, Ole Jensen, Jan-Mathijs Schoffelen, Markus Bauer, others 2004 sharing of code with external co-workers, FieldTrip was born 2003 collaborating and code sharing with EEGLAB 2005 collaborating and code sharing with SPM 2011 FieldTrip reference paper
- 3. Short history of live, and everything… 2004 first scientific results appearing from Donders 2002 yearly “toolkit” training events 2012 Stapel affaire (scientific misconduct) 2016 Donders Repository for research data 2016 first version of BIDS, followed by MEG, EEG, iEEG, PET, NIRS, … 2021 Open Brain Consent project and paper
- 4. My tools for Open Science Code for data analysis FieldTrip Methods for data sharing Donders Repository Ethics/Legal consent BIDS to organize the data They allow me to do my research and have others replicate it and build on it
- 5. Issues that we are jointly facing Improving quality, efficiency and impact Reproducibility crisis Increasing complexity of research Gap between academic training and professional career Open Science can contribute some solutions
- 6. Lack of trust - in society http://harrieverbon.blogspot.nl/2012/11/diederik-stapel-werd-ook-betaald-door.html
- 7. Lack of trust – among scientists Open Science Collaboration, Science (2015). DOI: 10.1126/science.aac4716
- 8. Replication crisis Incentive structure results in focus on sexy results Publication bias results in more papers with positive than negative results being published and read Trouble in the lab, The Economist (2013)
- 9. Incentives in academia Your career will benefit from Many publications High-impact publications Spectacular results This may result in undesired behavior P-hacking Harking Survival of the fittest promotes bad science (Smaldino & McElreath, 2016)
- 10. The EU's open science policy “Open science is a policy priority for the European Commission and the standard method of working under its research and innovation funding programmes as it improves the quality, efficiency and responsiveness of research. When researchers share knowledge and data as early as possible in the research process with all relevant actors it helps diffuse the latest knowledge. And when partners from across academia, industry, public authorities and citizen groups are invited to participate in the research and innovation process, creativity and trust in science increases.”
- 11. Open Science Open educational resources Open access publications Open peer review Open methodology Pre-registration Open source Open hardware Open data Markus Neuschäfer; https://www.slideshare.net/mneuschaefer/1504-open-knowledgefolien
- 12. Why is the transition to Open Science so hard? Doing science is already hard as it is. Academia is conservative, and slow to change. We don’t learn from each other. We read papers and see presentations: We review and comment during the preparation of presentations We review and comment during the preparation of manuscripts We do not review or comment on code, data and procedures.
- 13. Improving scientific procedures Doing an EEG/MEG study requires a good research question Theoretical underpinning Knowledge of relevant literature Preregistration Acquisition of data Lab setup Ethics Data Management Plan Design and implement your analysis how to start with new (pilot) analysis pipelines how to scale these to publication-quality group analysis Handling of work-in-progress scripts, data, and results Planning for and publicly sharing results, data and analysis details Open Data, Open Source code, and Open Access manuscript
- 14. Transferrable skills for future career Students are initially trained to use small computers but expected to do computations on big data Note: “big data” is complex data, “large data” is large in size but not per see complex
- 15. Sharing of analysis details (code) Manage versions of your analysis scripts Github, Gitlab, Bitbucket Backup and share between computers Collaborate and review Also used for FieldTrip development Toolbox code improvements Website
- 16. Sharing of project plan and details Many considerations and decisions are made during the course of a project. What to do, what not to do? What data to include, what data not to include? Electronic lab notebooks or platforms like the Open Science Framework (osf.io) can be used for documentation.
- 17. Research data management Publicly funded data should be as open as possible, but as closed as necessary Some interesting research data models Allen Institute for Brain Science publishes all data Astrophysics, shared infrastructure of large telescopes You need a plan to manage research data describe the complexity describe the infrastructure describe the procedures
- 18. Academic skills to acquire Critical thinking and analytical skills Good research practices MATLAB or Python coding and code management GitHub or other code versioning/collaborating Data management, e.g. BIDS, ethics, legal Where to share secondary results, e.g. Zenodo and OSF Being able to learn from interaction with others and from other disciplines
- 19. Summary Improving quality, efficiency and impact Reproducibility crisis Increasing complexity of research Gap between academic training and professional career Open Science can contribute some solutions
Editor's Notes
- #2: I am going to interpret “tools” in the wider sense, not only software, but also other tools that we develop during and for our research
- #9: This also applies to methods research
- #10: Institutional incentives are needed to curb bad science
- #16: Having your code visible to others makes you aware of its quality and limitations, collaborating/reviewing allows you to learn
- #17: Documenting complex projects requires thinking about it from a meta-viewpoint
- #18: Writing, discussing and reviewing this plan provides opportunities for learning and improvement