What is your h-index and other measures of impact
- 1. WHAT IS YOUR H-INDEX AND OTHER MEASURES OF ACADEMIC IMPACT Berenika M. Webster, PhD ULS/ISchool workshop 11 November 216 1
- 3. They are used to measure/demonstrate impact of… • National science systems • Research-producing institutions • Research groups • Individual researchers In order to… • Understand impact of investment • Award funding • Employment and promotion decisions • Apply for funding, job, promotion • Identify experts, collaborators, etc. 3
- 4. Learning outcomes • At the end of sessions participants will • be able to identify paper-level and author-level indicators of impact • be able to extract these indicators from Scopus, WoS and SciVal • Understand how individual researchers may use these indicators to demonstrate impact 4
- 5. “Citizen” bibliometrics… • Productivity (publications counts) • leads to “salami slicing”, maybe • quantity vs quality • Impact (citation counts) • but what impact? • Impact factor • Speaks to prestige of outlet, not quality of individual paper • 20% of papers in Nature get 80% of all citations (averages do not work when distributions are skewed) • h-index • arbitrary, simplistic and lacks consistency • it’s always highest in Google Scholar 5
- 6. (BIBLIOMETRIC) STORY OF ONE PUBLICATION aka article-level metrics 6
- 7. Story of one publication 7 Kato, M., Han, TW., Xie, S., Shi, K., Du, X., Wu, LC., Mirzaei, H., Goldsmith, EJ., Longgood, J., Pei,J., and Grishin, N.V. and Frantz, DE., Schneider, JW., Chen, S., Li, L., Sawaya, MR., Eisenberg, D, Tycko, R. and S. McKnight. (2012) “Cell-free Formation of RNA Granules: Low Complexity Sequence Domains Form Dynamic Fibers within Hydrogels.”, CELL, 149 (4): 753-67. (citations 325, IF 34.242)
- 8. Story of one publication • Journal characteristics • JIF or SNIP or another indicator of impact in context • Acceptance rates (e.g. from Cabell's Directory of Publishing Opportunities) • Co-author characteristics • Names, institutions, countries • Citations • Number of citations in context (normalised baselines) • Ratio • Percentile distribution • Characteristics of citing papers • authors, institutions, countries, subject fields, journals • Altmetrics • Predictor of citation impact (???) • Indicator of attention • Indicator of impact outside academic realm (e.g. citations in policy documents or clinical guidelines) 8
- 9. Story of one publication: WoS 9
- 10. Story of one publication: Scopus 10
- 11. Story of one publication: altmetrics 11
- 12. Story of one publication: more Altmetrics 12
- 13. Baselines • Web of Science (WoS) (at Pitt publication and citation data going back to 1980) • Highly Cited?; Hot?, Impact Factor value • Essential Science Indicators (ESI) (publication and citation data going back 10 years + current year) • across 22 broad disciplines • Baselines (Averages and Percentiles) • Journal Citation Reports (JCR) • IF values of thousands of journals and much more (citation networks, rates of obsolescence, etc.) • Scopus (complete citation data going back to 1996, project to extend to 1970 under way to be completed by end of 2016) • discipline adjusted citation rates (more granular disciplinary and sub-disciplinary divisions), also adjusted for all doc. types • SciVal (complete publication and citation data going back to 1996. Based on SCOPUS-indexed publications) • Over 20 indicators of productivity and impact 13
- 14. Story of one publication 14 My paper has been published in Cell, 2nd top ranked journal in the field of biochemistry and mol. biology and 3nd top title in the filed of cell biology (JCR, 2015 ed.) To date it was cited 325 times, which places it in the top 1% of all world’s 2012 biochem. and mol. biology papers. The paper has an international and multidisciplinary reach. Citations come from authors from 34 different countries and from across 95 different journals in 35 different fields (including 7 citations from Nature and 6 from Science). (WoS, 20 Sept. 2016). My paper also attracted attention from non-scholarly audiences, … Kato, M., Han, TW., Xie, S., Shi, K., Du, X., Wu, LC., Mirzaei, H., Goldsmith, EJ., Longgood, J., Pei,J., and Grishin, N.V. and Frantz, DE., Schneider, JW., Chen, S., Li, L., Sawaya, MR., Eisenberg, D, Tycko, R. and S. McKnight. (2012) “Cell-free Formation of RNA Granules: Low Complexity Sequence Domains Form Dynamic Fibers within Hydrogels.”, CELL, 149 (4): 753-67. (citations 325, IF 34.242)
- 15. Exercise • Murphy, S.V. and A. Atala (2014) “3D bioprinting of tissues and organs” NATURE BIOTECHNOLOGY; 32(8):773-785. DOI: 10.1038/nbt.2958 • Describe it using bibliometric indicators discussed today • Citation count • Normalised citation count • Percentile position • Impact of publishing journal • Characteristics of citing publications • Use WoS or SCOPUS and Altmetric.com 15
- 16. Web of Science and Essential Science Indicators • Murphy, S.V. and A. Atala (2014) “3D bioprinting of tissues and organs” NATURE BIOTECHNOLOGY; 32(8):773-785. • 337 citations • 337/5.31 = 63.45 (normalised citation impact) • In top 1% of all 2014 Biology and Biochemistry publications (HICI paper) 16
- 17. 17
- 18. 18
- 19. (BIBLIOMETRIC) STORY OF ONE AUTHOR aka aruthor-level metrics 19
- 20. Story of one author • Source of information about publications • CV • Pitt Faculty Info System (Elements) • ORCID • Google Scholar profile • How much output is captured in databases • Scopus Author Identifier • ResearcherID (WoS) 20
- 21. Scholarly communication practices 21 Subject area Books and book chapters Conference papers Journal articles History 45.6 3.8 50.6 Politics and Policy 43.1 10.8 46.1 Language 40.5 7.6 51.8 Human Society 31.3 5.6 63 Philosophy 29.8 5.4 64.8 Economics 27.4 8 64.5 Law 26.2 1.9 71.9 The Arts 25.2 20.3 54.5 Education 21.8 23.6 54.5 Architecture 20.8 43.6 35.6 Psychology 18.9 4.9 76.2 Journalism, library 18.6 24.2 57.2 Management 13 34 52.9 Earth Sciences 8.6 9.2 82.2 Medical & Health Sci 6.6 2.9 90.5 Biological Sciences 6.6 2.7 90.7 Agriculture 6.3 14.7 79 Computing 5 62.3 32.8 Mathematical Sciences 5 11.2 83.8 Engineering 2.9 45.1 52 Physical Sciences 2.7 7.3 90 Chemical Sciences 2.3 1.9 95.7 L. Butler, 2006 And what about all other research outputs?
- 22. Author metrics • Overview of productivity and impact • Should use size-dependent variables as these take into consideration volume of outputs • Publications (other outputs) counts and characteristics • Total citation counts in context (discipline, age, output type) • Excellence measures (top 1, 5, 10% of distributions) • ESI’s Highly Cited author? • h-index (if you must) – though it is inconsistent (example later) 22
- 23. Story of one author 23
- 24. SCOPUS view 24 Citation per publication rate for my 200 publications indexed in Scopus is 91.2, only 5 of my publications (2.5%) were not cited o date. (Scopus, 8 June 2016) My research is multidisciplinary spanning across biochemistry, genetics and molecular biology, physics chemistry, materials science and engineering. I have been publishing consistently since 1982, averaging 7 publications per year in high quality, peer reviewed outlets. These publications, collectively receive upwards of 1,000 citations per year. (Scopus, 8 June 2016)
- 25. Story of one author 25 Citations to my publications in the last 10 years placed me in the group of top 1% of biochemistry researchers in the world and 5 of these publications placed in the top 1% of publications in their field. (ESI, 20 Sept 2016)
- 26. SciVal view - overview 26 In the last 5 years, 68% of my publications placed in the top 10% of all similar publications (by field and age), and 32% of them were published in the top 10% of scientific journals. (SciVal, 8 June 2016)
- 27. My publications in biochemistry consistency outperform these of NIH and USA as a whole in relation of impact and percentage of publications in top 1% in the world. (SciVal, 8 June 2016) 27
- 28. Whose h-index? One author – 3 different values 28 137 vs. 143 vs. 160
- 29. Author 1 Author 2 Author 3 15 150 15 10 100 10 10 50 10 5 25 5 5 5 5 4 1 0 4 0 0 3 0 0 3 0 0 1 0 0 Does not account for: •insensitive to highly-cited publications •citation characteristics of publication outlets •citation characteristics of fields of science •age of publications •type of publications •co-authorship •self-citations •scientific age of author 29
- 30. More problems with h-index 30 Ludo Waltman and Nees Jan van Eck, JASIST 2012 If two scientists achieve the same absolute performance improvement, their ranking relative to each other should remain unchanged. Scientist X Scientist Y Publication Citations 1 5 6 2 5 6 3 5 6 4 5 6 5 5 3 6 2 3 7 2 3 Scientist X Scientist Y Publication Citations 1 8 8 2 8 8 3 5 6 4 5 6 5 5 6 6 5 6 7 5 3 8 2 3 9 2 3 Scientist X h-index 5 Scientist Y 4 Scientist X – 5 Scientist Y - 6
- 31. Exercise: create a bibliometric profile of a researcher • Ludo Waltman, Leiden U • Krzysztof Matyjaszewski, CMU • Ervin Sejdic, Pitt • Anne-Wil Harzing, U Melbourne • Try SCOUPS, WOS and SciVal to find out • Number of publications • Total number of citations • Normalised citations (e.g. within a discipline) • Subject distributions • Characteristics/impact of citing publications • For an entire career span/for last 5 years 31
Editor's Notes
- #8: Researchers usually provide Raw citation count (lacks context) JIF(says nothing about impact of individual publication
- #10: View Jln Metrics Co-authors (institutions and countries) Citation count HI CI? If not: check ESI baselines and percentiles HI CI author? Citing pubs – authors, institutions,