SlideShare a Scribd company logo
The State of science,
journals, peer-review,
thoughts on Open
Science, reproducibility,
and Science 2.0?
Dr. Benjamin Laken
University of Oslo
@benlaken
Science in the context of journals, Open, and the future
Science in the context of journals, Open, and the future
Science in the context of journals, Open, and the future
We are now on the brink of an achievable
aim: for all science literature to be online,
for all of the data to be online and for the
two to be interoperable.
Source XKCD
Essentially, the volume of published work is constantly
increasing.
A rough estimate of the rate of increase is …
Unsurprisingly, the number of PhDs
is also increasing
Nature 472, 276-279 (2011) | doi:10.1038/472276a
Trends in PhDs granted per year
for all disciplines,
values x103
The number of science doctorates
earned each year grew by nearly
40% between 1998 and 2008, to
some 34,000, in countries that are
members of the Organisation for
Economic Co-operation and
Development (OECD). The growth
shows no sign of slowing
Systems Research and Behavioral Science Syst. Res (2014)
Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324
(StatshereforUSAbiomedicalsciences)However, there is no matching rise in permanent positions
This has created ‘The Postdoc Pile-Up’
Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a
In essence a saturated Postdoc market. This means that postdocs who want to
stay in academia are usually destined to travel from one place to another with no
long-term prospects. (One of the reasons you may hear postdocs referring to
themselves as ‘hobos with a PhD’.)
Science in the context of journals, Open, and the future
In the context of the increasing pressures on individual
scientists to compete—largely through publications—enters
the problem of Academic publishers as for-profit entities
The first journal, Proceedings of the Royal Society, was
created by Henry Oldenburg (1619--1677), in 1665.
The idea was to drive science forward through rapid
communication and sharing to a broad audience.
However, things have gone slightly astray over the years…
The role of journals: have they changed from a force of
innovation to a hinderance?
The academic publishing business model explained by
Nature’s Publishing Director
Apple:
Exxon:
Profits (% of revenue) in 2011
Elsevier:
Springer:
John Wiley & Sons:
~25%
~8.5%
36% (£724m)
33.9% (£866m)
42% ($106m)
Are journals overcharging?
Some opinions on academic journals…
Cushing, 2014, https://www.techdirt.com/articles/20140605/05175627467/academic-publisher-fights-publication-paper-
criticizing-publishers-price-increases-profits.shtml
Elsveier a particularly bad example
Is a good solution to science communication simply
traditional journals but as open-access?
Open access does not mean free, often the authors still pay high prices
to publish their research.
The growth of Open Access (OA) journals in the last 10 years opened the
door for the creation of hundreds of new Publishing company and new
journals, who engage in behaviour often best described as ‘predatory’…
They claim their articles are peer-reviewed, however independent
research and many examples have repeatedly shown that the real barrier
to publication in their journals is simply the publication fee…
The Dark-side of the OA journal explosion
In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with
two figures) to International Journal of Advanced Computer Technology…
…they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal
requested a fee of $150.
Tom Spears submitted a completely incoherent paper featuring a made-
up study area (Nepean Desert)
Several famous examples of OA journal ‘stings’
Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.
More famous sting: John Bohannon
J Bohannon Science 2013;342:60-65
Abstract
Dozens of open-access journals targeted in an
elaborate Science sting accepted a spoof research
article, raising questions about peer-review
practices in much of the open-access world.
Peer review reviewed.Few journals did substantial review that identified the paper's flaws.
Another problem: the valuable reviews are usually wasted
It is not a question of if an article will be published, but where.
In this context, the expert reviews, community feedback, and
reproducibility are the hallmarks of quality, and a real factor in
the value of the research.
We need to develop systems which focus on these aspects of
the peer system, rather than see peer-review as the ‘behind
the scenes’ work.
Source: PhD comic
Explosion in OA journals over last decade
(and rapid growth of predatory OA journals)
Published science doubles every ~9 yrs
New PhD’s awarded annually increasing
(simultaneous decrease in TT and Permanent positions)
‘Publish or perish’, saturated PhD-level job market, and resulting
competitiveness has combined with journal profiteering
…and if a paper gets a hard review, rejection usually means a paper will be
submitted (unchanged) to new journal, until accepted
Many Journals = opaque review process, expensive, & behind the
scenes politics/motives
End result (*at least for me) = scepticism, cynicism, frustration, time loss,
high-blood pressure, and madness
Publication ≠ quality guarantee
Favour systems totally open (inc. review) and communal
New paradigm: publication not the end of a funding cycle but the
start of a collaborative communal conversation
Summary
My fantasy model of Science 2.0
Data, code, paper: all in one open repository (like ArXiV)
Version control system (like Github)
All researchers have unique ID profile (like RG/ORCID)
ID-tagged comments. Can be general, or specific:
specific comments tagged to sections, as a thread, including up-
vote system (like StackOverflow) on paper and comments.
Copying, and contributing (pull-requests) allowed
to create rapid-iterations and community refinement (like Github)
Vote-based score system, with scores contributing to different levels of
community privileges (like StackOverflow).
Score a measure of a researchers overall contributions (Like RG), > than
citation metrics [e.g. as bad papers often cited on purpose]
Needs shared super-computer-scale resources to re-run
experiments (Like Google Earth Engine)
(This model already exists for the open software we use every day. The model can be used for
research, and for lecture/course/textbook material too.)

More Related Content

PDF
Who Says Big Companiers Can't Innovate?
PDF
Data Science and Culture
PDF
Booz Allen Hamilton's Data Science Infographic
PDF
Data Science 101
PDF
Data Science For Social Scientists Workshop
PDF
The Field Guide to Data Science
PDF
Booz Allen Field Guide to Data Science
PDF
The New Future | 2014 Global Leadership Summit
Who Says Big Companiers Can't Innovate?
Data Science and Culture
Booz Allen Hamilton's Data Science Infographic
Data Science 101
Data Science For Social Scientists Workshop
The Field Guide to Data Science
Booz Allen Field Guide to Data Science
The New Future | 2014 Global Leadership Summit

What's hot (18)

PPTX
8 steps to diy pd surreydinner
PDF
The Digital Enterprise
PPTX
Enterprise and Acumen:Real World Information Skills and Employability for Bus...
PPTX
The Puzzles Librarians Need to Solve - Vala 2016
PPTX
Digital Business
PDF
The Future of Work
PPT
TREC 2016: Looking Forward Panel
PPTX
'Drinking from the fire hose? The pitfalls and potential of Big Data'.
PPTX
Measuring and Capturing Value of Government Communication
PDF
Gov 2.0 for Texas Certified Public Manager (CPM ) Program
PPTX
Learning From the COViD-19 Global Pandemic
PDF
State Dep't Open Gov Plan
PDF
CIL Stats Workshop April1 2022 Abram Silk.pdf
PPTX
Why Be Open?
PDF
Evolution of assessment 2017 final
PDF
Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World
PPT
H.R Congress Apr '08 in Warsaw
PDF
CIPR 'The Effects of AI on the Professions; A literature repository'
8 steps to diy pd surreydinner
The Digital Enterprise
Enterprise and Acumen:Real World Information Skills and Employability for Bus...
The Puzzles Librarians Need to Solve - Vala 2016
Digital Business
The Future of Work
TREC 2016: Looking Forward Panel
'Drinking from the fire hose? The pitfalls and potential of Big Data'.
Measuring and Capturing Value of Government Communication
Gov 2.0 for Texas Certified Public Manager (CPM ) Program
Learning From the COViD-19 Global Pandemic
State Dep't Open Gov Plan
CIL Stats Workshop April1 2022 Abram Silk.pdf
Why Be Open?
Evolution of assessment 2017 final
Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World
H.R Congress Apr '08 in Warsaw
CIPR 'The Effects of AI on the Professions; A literature repository'
Ad

Viewers also liked (20)

PDF
10 myths about psychology
PDF
Building capacity for open science - COASP Meeting
PDF
Helsinki book bookmarks_v1
PPT
Columbia Talk on Open Notebook Science
PDF
Building Capacity for Open Science
PPTX
The Future of Open Science
PPTX
Open Science and European Access Policies in H2020
PPTX
Open science, open data - FOSTER training, Potsdam
PDF
Relationships between Open Science, Science 2.0, and Social Media
PPSX
Design + Art + Science, and Demoscene
PPTX
Open data and Open Science
PPTX
Introduction to open science
PPTX
Open science
PPTX
Directions in Open Science
PDF
Presentation on Open Science and its 'Impacts';
PPTX
What is Open Science and what role does it play in Development?
PPTX
Open Science: What, why, how?
PDF
Winning research proposals with open science
PPTX
Unit 1, Lesson 1.8 - The Scientific Method (Part Two)
PPTX
Principles and practice of Open Science
10 myths about psychology
Building capacity for open science - COASP Meeting
Helsinki book bookmarks_v1
Columbia Talk on Open Notebook Science
Building Capacity for Open Science
The Future of Open Science
Open Science and European Access Policies in H2020
Open science, open data - FOSTER training, Potsdam
Relationships between Open Science, Science 2.0, and Social Media
Design + Art + Science, and Demoscene
Open data and Open Science
Introduction to open science
Open science
Directions in Open Science
Presentation on Open Science and its 'Impacts';
What is Open Science and what role does it play in Development?
Open Science: What, why, how?
Winning research proposals with open science
Unit 1, Lesson 1.8 - The Scientific Method (Part Two)
Principles and practice of Open Science
Ad

Similar to Science in the context of journals, Open, and the future (20)

PPT
PLoS - Why It is a Model to be Emulated
PPT
What's wrong with scholarly publishing today?
PPT
Author workshop TU Delft 20111122
PPT
PPT
Scott Edmunds, HKU Open Access Week: Experiences from the front-line of Open ...
PDF
Ulrich Pöschl (MPIC/ ACP) -- Multi-stage open peer review integrating the str...
PPT
Legal scholarship and OA publishing: developing radical pathways to free, op...
PDF
HERS SA Academy 8 September 2014: Workshop on Scholarly Journals
PPT
PDF
The new alchemy: Online networking, data sharing and research activity distri...
PPTX
Of CUNY, By CUNY, For CUNY: How We All Benefit from Open Access and Why We Al...
PDF
Perspectives in Scientific Communication: Publishing in Transition
PPTX
Library as publisher
PPT
May 17 editors ag_mexico city
PDF
STRETCHING THE BOUNDARIES OF PUBLISHING: ALTERNATIVES
PPTX
Discussing research with the public in the blogosphere
PPTX
Remapping the Global and Local in Knowledge Production: Roles of Open Access
PPTX
I conf2016
PLoS - Why It is a Model to be Emulated
What's wrong with scholarly publishing today?
Author workshop TU Delft 20111122
Scott Edmunds, HKU Open Access Week: Experiences from the front-line of Open ...
Ulrich Pöschl (MPIC/ ACP) -- Multi-stage open peer review integrating the str...
Legal scholarship and OA publishing: developing radical pathways to free, op...
HERS SA Academy 8 September 2014: Workshop on Scholarly Journals
The new alchemy: Online networking, data sharing and research activity distri...
Of CUNY, By CUNY, For CUNY: How We All Benefit from Open Access and Why We Al...
Perspectives in Scientific Communication: Publishing in Transition
Library as publisher
May 17 editors ag_mexico city
STRETCHING THE BOUNDARIES OF PUBLISHING: ALTERNATIVES
Discussing research with the public in the blogosphere
Remapping the Global and Local in Knowledge Production: Roles of Open Access
I conf2016

Recently uploaded (20)

PPT
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
PDF
HPLC-PPT.docx high performance liquid chromatography
PDF
IFIT3 RNA-binding activity primores influenza A viruz infection and translati...
PPT
Chemical bonding and molecular structure
PPTX
2. Earth - The Living Planet earth and life
PPTX
ognitive-behavioral therapy, mindfulness-based approaches, coping skills trai...
PDF
diccionario toefl examen de ingles para principiante
PDF
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
PPTX
Cell Membrane: Structure, Composition & Functions
PDF
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
PPTX
ECG_Course_Presentation د.محمد صقران ppt
PDF
bbec55_b34400a7914c42429908233dbd381773.pdf
PDF
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
PPTX
Comparative Structure of Integument in Vertebrates.pptx
PDF
MIRIDeepImagingSurvey(MIDIS)oftheHubbleUltraDeepField
PDF
Phytochemical Investigation of Miliusa longipes.pdf
PDF
An interstellar mission to test astrophysical black holes
PPTX
Classification Systems_TAXONOMY_SCIENCE8.pptx
PPTX
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
PPTX
ANEMIA WITH LEUKOPENIA MDS 07_25.pptx htggtftgt fredrctvg
The World of Physical Science, • Labs: Safety Simulation, Measurement Practice
HPLC-PPT.docx high performance liquid chromatography
IFIT3 RNA-binding activity primores influenza A viruz infection and translati...
Chemical bonding and molecular structure
2. Earth - The Living Planet earth and life
ognitive-behavioral therapy, mindfulness-based approaches, coping skills trai...
diccionario toefl examen de ingles para principiante
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
Cell Membrane: Structure, Composition & Functions
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
ECG_Course_Presentation د.محمد صقران ppt
bbec55_b34400a7914c42429908233dbd381773.pdf
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
Comparative Structure of Integument in Vertebrates.pptx
MIRIDeepImagingSurvey(MIDIS)oftheHubbleUltraDeepField
Phytochemical Investigation of Miliusa longipes.pdf
An interstellar mission to test astrophysical black holes
Classification Systems_TAXONOMY_SCIENCE8.pptx
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
ANEMIA WITH LEUKOPENIA MDS 07_25.pptx htggtftgt fredrctvg

Science in the context of journals, Open, and the future

  • 1. The State of science, journals, peer-review, thoughts on Open Science, reproducibility, and Science 2.0? Dr. Benjamin Laken University of Oslo @benlaken
  • 5. We are now on the brink of an achievable aim: for all science literature to be online, for all of the data to be online and for the two to be interoperable.
  • 7. Essentially, the volume of published work is constantly increasing. A rough estimate of the rate of increase is …
  • 8. Unsurprisingly, the number of PhDs is also increasing Nature 472, 276-279 (2011) | doi:10.1038/472276a Trends in PhDs granted per year for all disciplines, values x103 The number of science doctorates earned each year grew by nearly 40% between 1998 and 2008, to some 34,000, in countries that are members of the Organisation for Economic Co-operation and Development (OECD). The growth shows no sign of slowing
  • 9. Systems Research and Behavioral Science Syst. Res (2014) Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/sres.2324 (StatshereforUSAbiomedicalsciences)However, there is no matching rise in permanent positions
  • 10. This has created ‘The Postdoc Pile-Up’ Nature 520, 144–147 (09 April 2015) doi:10.1038/520144a In essence a saturated Postdoc market. This means that postdocs who want to stay in academia are usually destined to travel from one place to another with no long-term prospects. (One of the reasons you may hear postdocs referring to themselves as ‘hobos with a PhD’.)
  • 12. In the context of the increasing pressures on individual scientists to compete—largely through publications—enters the problem of Academic publishers as for-profit entities The first journal, Proceedings of the Royal Society, was created by Henry Oldenburg (1619--1677), in 1665. The idea was to drive science forward through rapid communication and sharing to a broad audience. However, things have gone slightly astray over the years… The role of journals: have they changed from a force of innovation to a hinderance?
  • 13. The academic publishing business model explained by Nature’s Publishing Director
  • 14. Apple: Exxon: Profits (% of revenue) in 2011 Elsevier: Springer: John Wiley & Sons: ~25% ~8.5% 36% (£724m) 33.9% (£866m) 42% ($106m) Are journals overcharging?
  • 15. Some opinions on academic journals…
  • 17. Elsveier a particularly bad example
  • 18. Is a good solution to science communication simply traditional journals but as open-access? Open access does not mean free, often the authors still pay high prices to publish their research. The growth of Open Access (OA) journals in the last 10 years opened the door for the creation of hundreds of new Publishing company and new journals, who engage in behaviour often best described as ‘predatory’… They claim their articles are peer-reviewed, however independent research and many examples have repeatedly shown that the real barrier to publication in their journals is simply the publication fee…
  • 19. The Dark-side of the OA journal explosion In 2005, following journal submission request emails, Peter Vamplew sent a 10 pg paper (with two figures) to International Journal of Advanced Computer Technology… …they accepted the paper. An anonymous reviewer rated it as “excellent”, and the journal requested a fee of $150.
  • 20. Tom Spears submitted a completely incoherent paper featuring a made- up study area (Nepean Desert) Several famous examples of OA journal ‘stings’ Sent to 18 journals. 8 responded quickly, accepting, asking a $1k - 5$ fee.
  • 21. More famous sting: John Bohannon J Bohannon Science 2013;342:60-65 Abstract Dozens of open-access journals targeted in an elaborate Science sting accepted a spoof research article, raising questions about peer-review practices in much of the open-access world. Peer review reviewed.Few journals did substantial review that identified the paper's flaws.
  • 22. Another problem: the valuable reviews are usually wasted It is not a question of if an article will be published, but where. In this context, the expert reviews, community feedback, and reproducibility are the hallmarks of quality, and a real factor in the value of the research. We need to develop systems which focus on these aspects of the peer system, rather than see peer-review as the ‘behind the scenes’ work.
  • 24. Explosion in OA journals over last decade (and rapid growth of predatory OA journals) Published science doubles every ~9 yrs New PhD’s awarded annually increasing (simultaneous decrease in TT and Permanent positions) ‘Publish or perish’, saturated PhD-level job market, and resulting competitiveness has combined with journal profiteering …and if a paper gets a hard review, rejection usually means a paper will be submitted (unchanged) to new journal, until accepted Many Journals = opaque review process, expensive, & behind the scenes politics/motives End result (*at least for me) = scepticism, cynicism, frustration, time loss, high-blood pressure, and madness Publication ≠ quality guarantee Favour systems totally open (inc. review) and communal New paradigm: publication not the end of a funding cycle but the start of a collaborative communal conversation Summary
  • 25. My fantasy model of Science 2.0 Data, code, paper: all in one open repository (like ArXiV) Version control system (like Github) All researchers have unique ID profile (like RG/ORCID) ID-tagged comments. Can be general, or specific: specific comments tagged to sections, as a thread, including up- vote system (like StackOverflow) on paper and comments. Copying, and contributing (pull-requests) allowed to create rapid-iterations and community refinement (like Github) Vote-based score system, with scores contributing to different levels of community privileges (like StackOverflow). Score a measure of a researchers overall contributions (Like RG), > than citation metrics [e.g. as bad papers often cited on purpose] Needs shared super-computer-scale resources to re-run experiments (Like Google Earth Engine) (This model already exists for the open software we use every day. The model can be used for research, and for lecture/course/textbook material too.)