![“Manuscripts that have been published in Leukemia are too
frequently ignored in the reference list of newly submitted
manuscripts, even though they may be extremely relevant. As
we all know, the scientific community can suffer from
selective memory when giving credit to colleagues. While we
have little power over other journals, we can at least start
by giving you and others proper credit in Leukemia. We have
noticed that you cite Leukemia [once in 42 references].
Consequently, we kindly ask you to add references of articles
published in Leukemia to your present article.”](https://image.slidesharecdn.com/metrics-whattheyareandhowtousethem-october2016-171213154322/85/Metrics-what-they-are-and-how-to-use-them-61-320.jpg)
![The Metric Tide.
[We] propose the notion of
responsible metrics as a way of
framing appropriate uses of
quantitative indicators in the
governance, management and
assessment of research.](https://image.slidesharecdn.com/metrics-whattheyareandhowtousethem-october2016-171213154322/85/Metrics-what-they-are-and-how-to-use-them-83-320.jpg)
Metrics: what they are and how to use them
- 1. Metrics What they are and how to use them Chris Biggs and David Jenkins
- 2. Bibliometrics are the quantitative analysis of publication and citation data
- 3. Examples of bibliometric measures include citation counts, Journal Impact Factor and H-index
- 4. You can get bibliometrics from databases such as Scopus, Web of Science and Google Scholar
- 5. Altmetrics look at the mentions and uses of research that are not captured by traditional bibliometrics
- 6. Altmetrics include mentions, uses, downloads, views, shares from a wide variety of sources
- 7. Metrics are seen as important because they acknowledge that a paper is relevant and known
- 8. Bibliometrics are seen as objective, inexpensive to produce/use, economical in terms of time and scalable
- 9. What do you think about metrics?
- 10. How are you using metrics?
- 11. What do you want to get out of this session?
- 12. Metrics can be used on CVs, personal websites and on bid applications
- 13. Fawcett, Tom. "An introduction to ROC analysis." Pattern recognition letters 27.8 (2006): 861-874.
- 15. 0 1000 2000 3000 4000 5000 6000 7000 8000 No.ofcitations Data source Fawcett, Tom. "An introduction to ROC analysis." Pattern recognition letters 27.8 (2006): 861-874. Web of Science Scopus Google Scholar
- 17. Citations measure what is cited not what is high quality
- 18. Journal Impact Factor is designed to allow you to see how highly cited the average article in a journal is
- 19. Journal Impact Factor is used as a proxy for importance and quality
- 20. No. of citations received in 2015 by articles published in the journal during 2013 and 2014 No. of articles published in the journal during 2013 and 2014 JIF 2015 =
- 21. You can get Journal Impact Factor from Journal Citation Reports
- 25. The H-index is designed to measure productivity and impact
- 26. “The definition of the index is that a scholar with an index of h has published h papers each of which has been cited in other papers at least h times”
- 27. Exercise Calculate the h-indexes for the researchers on the sheet. What does their score tell you about them?
- 28. H-index is criticised because it can’t be used to compare researchers in different disciplines, at different career stages or who have different outputs
- 29. The H-index measures consistency rather than outright impact
- 32. Altmetrics aim to cover “the evaluation gap”
- 33. You can get altmetrics from dedicated services or embedded in databases
- 35. Research can get high metrics for different reasons
- 40. The Benefits To make your research known To increase chance of citation To correct attributions To ensure research is counted in research assessment To increase chance of new collaboration To increase chance of funding
- 41. Different functions have created different systems
- 42. Different systems have different modus operandi
- 43. Some profiles already exist, some need to be created, all need to be curated.
- 44. Scopus
- 45. 1 2 3
- 46. Google Scholar
- 47. ResearchGate
- 48. The Uses and Abuses of Metrics
- 49. A shortcut to evaluation. Metrics appear to be objective, cost effective and useful where the evaluator isn't a domain expert.
- 50. Old boys network. Better than relying on old boys network to make decisions.
- 51. Funders might use metrics to evaluate bids and allocate monies.
- 52. Universities might use metrics to recruit and promote.
- 53. Publishers might use metrics to demonstrate reputation.
- 54. Individuals might use metrics to support career progression and standing in their field.
- 55. This environment may encourage unwanted behaviours.
- 56. Inappropriate application of metrics. For instance, where JIF factor is used as an indicator of the quality of an article or author.
- 58. Gaming or chasing metrics.
- 59. Coercive citations. Editors ask authors to cite papers from the journal they are submitting to.
- 61. “Manuscripts that have been published in Leukemia are too frequently ignored in the reference list of newly submitted manuscripts, even though they may be extremely relevant. As we all know, the scientific community can suffer from selective memory when giving credit to colleagues. While we have little power over other journals, we can at least start by giving you and others proper credit in Leukemia. We have noticed that you cite Leukemia [once in 42 references]. Consequently, we kindly ask you to add references of articles published in Leukemia to your present article.”
- 62. Editorials. Editorials may preferentially cite papers from their own journal.
- 64. Publish more review articles. Under the assumption that review papers are more likely to be cited than research papers.
- 66. Free citations. Non-citable items such as news articles, editorials, book reviews, abstracts of meetings, comments, correspondence, viewpoints.
- 68. Citation cartels. Where groups of editors and journals cite each other.
- 69. Cited 490 papers in total • 445 papers from Cell Transportation within 2 year JIF window • 44 papers from Medical Science Monitor 2 year JIF window
- 70. Cited 124 papers in total • 96 papers from Cell Transportation within 2 year JIF window • 26 papers from The Scientific World Journal within 2 year JIF window
- 71. • 87 papers from Cell Transportation within 2 year JIF window • 32 papers from Medical Science Monitor 2 year JIF window
- 72. • 109 papers from Cell Transportation within 2 year JIF window • 29 papers from The Scientific World Journal within 2 year JIF window
- 73. Padded citations. The addition of superfluous citations to manuscripts prior to submission.
- 74. Honorary authorship. The inclusion of individuals as authors on manuscripts, or grant proposals, even though they did not contribute to the research effort.
- 76. Fig. 1 Percentage of respondents who report that honorary authors have been added to their research projects, they have been coerced by editor to add citations, or who have padded their citations, sorted by field of study and type of manipulation.
- 77. Fig 2. Reasons for adding honorary authors to grants and manuscripts.
- 78. Salami publishing. Publication of two or more papers from a single study.
- 79. Self-citation. How much is too much?
- 80. Individuals will seek to publish in Journals with high impact factors. Does this affect research?
- 82. The Metric Tide. "Metrics hold real power: they are constitutive of values, identities and livelihoods."
- 83. The Metric Tide. [We] propose the notion of responsible metrics as a way of framing appropriate uses of quantitative indicators in the governance, management and assessment of research.
- 84. Responsible Metrics. The Leiden Manifesto.
- 85. Responsible Metrics. San Francisco Declaration on Research Assessment.
- 86. Any questions?
- 87. Thank you!