Research evaluation systems and the percolation of bad science
Download as PPTX, PDF1 like1,685 views
The document discusses the negative impact of performance-based research evaluation systems, highlighting how they encourage poor practices and reduce pluralism in science. It argues that current systems prioritize metrics over genuine scientific inquiry, leading to a distortion of research goals and values. To foster better science and policy, the document advocates for a shift away from these evaluation systems in favor of approaches that value sound science and diverse ideas.
Research evaluation systems and the percolation of bad science
- 1. Performance-based incentives, research evaluations systems and the percolation of bad science Alberto Baccini (Dep. of Economics and Statistics, University of Siena) INET ID # INO17-00015 is gratefully acknowledged
- 4. Research evaluation systems reinforce the incentives of the publish or perish environment
- 6. Research evaluation systems reduce pluralism in science
- 7. Hindering interdisciplinarity Suffocation of wider social engagement Following the herd (Heckman) Avoiding risk in selecting research topics Known and documented effect of research evaluation systems
- 8. Growing perverse incentives in academia “National assessment procedures, such as the Research Excellence Framework, incentivize bad practices”
- 10. Adjusting contents of papers for: 1. Boosting productivity indicators 2. Pumping-up citation indicators
- 11. Gaming the rules: salami slicing
- 12. A new documented record in salami slicing 39 papers/10 retracted
- 13. Gaming the rules: manipulating authorship For being an author is sufficient to be more senior than the people who did the experiment
- 14. Gaming the rules in Italy? 2010 MATHEMATICS
- 15. Do senior professors game the rules in Italy? 2010 2010 2010 2010 2010 Mathematics Logic Accounting Management Professor of geometry Econometrics 2010 Political economy 2010
- 16. Pump-up the citations in Italy?
- 17. A new Italian miracle?
- 18. The introduction of incentives rewarding metrics of production and citations, corrupt the social norms and values that should regulate interactions in science Priority in discovery ceases to be the main goal of a scientist The new goal is gaining appropriate labels and performing better on metrics Summary 1
- 19. Provisional scientific truths emerge from a battlefield of conflicting ideas as result of the “public use of reason”. In the current system rankings of scientists or journals guarantee definitive scientific truths In research evaluation systems, a governmental ranking guarentees the definitive scientific truth of science Summary 2
- 20. Research evaluation systems put freedom of science at risk Governments may drift science in the desired direction by an appropriate design of research evaluation The hegemony of a mainstream economic policy can be reinforced by stiffening the research evaluation rules (assign top ranks only to mainstream journals) Summary 3
- 21. For having good policy advices we need pluralism and good science. Both are harmed by research evaluation systems. Minimize impact of research evaluation systems on science. Or in other words: reverse the direction of the last thirty years of science and research policies. Dismiss the research evaluation systems. (At least decouple them from authomatic funding mechanism) A plea for a new research policy
- 22. Good policy advices arise from the battle of ideas, not from the battle for achieving labels or conquering a publication spot in a top journal - Governments should consider seriously: • the growing bottom-up pressure in science for replacing “excellence” with “sound science” • innovative proposals (e.g. random funding) for funding research in view of fostering pluralism • to reward “sound” scientists that provides data, methods, protocols, software openly for replication and reuse How to change the PoP environment?
- 23. Performance-based incentives, research evaluations systems and the percolation of bad science Alberto Baccini (Dep. of Economics and Statistics, University of Siena) INET ID # INO17-00015 is gratefully acknowledged