Bibliometrics in practice: how to generate reports for institutions.
Download as PPTX, PDF4 likes2,621 views
The document outlines the principles and practices of generating bibliometric reports for institutions, focusing on the annual memory report from the University of Granada. It details the various types of bibliometric reports, data sources, and indicators used in evaluating research performance, as well as the importance of benchmarking and contextualizing results. The document emphasizes the necessity for transparency and reproducibility in reporting, providing tools for institutions to effectively communicate their research impact.
Bibliometrics in practice: how to generate reports for institutions.
- 1. Daniel Torres-Salinas Bibliometrics in practice: how to generate reports for institutions.
- 2. Introduction
- 3. Objectives and index ★ Learn how to design bibliometric reports at the institutional level (e.g., universities, hospitals, research centers). We are going to concentrate in a specific report: the annual memory (University of Granada - UGR). ★ Mission of the bibliometrics units ★ Talk Index ○ What information sources can we use ○ Content and selection of bibliometrics indicators ○ Making data available online ○ Examples of different reports
- 4. Type of bibliometric Reports If you are working for a bibliometric unit or research evaluation unit What kind of bibliometric reports can you offer to your institution? Annual memory Reports on a specific topics relevant to the institution Reports on demand by policy managers
- 5. Type of bibliometric Reports - Specific topics, for example gender, collaboration - Specific areas, for example humanities - Specific information sources, for example altmetrics - Specific document type, for example books - Reports on demand, examining a program - Publication … Open Access - Reports on new social media
- 6. Case Study: University of Granada For this course we are going to take as a case study the annual memory from the University of Granada “Indicators and statistics at the University of Granada” Download at: https://tinyurl.com/y8ywsdu3
- 7. Structure of a bibliometric report For an annual memory we have to consider at least the following sections: Main sections 1) Summary 2) Sources and indicators 3) General view 4) Fields and disciplines 5) Benchmarking and comparisons 6) Funding information
- 9. Types of sources ★ External database ○ Bibliographic databases and Citation Indexes: Scopus or Web of Science Core Collection, …. ○ Bibliometric suites: Incites or Scival ★ Internal databases ○ Current Research Information Systems (CRIS) ○ Institutional administrative databases (Projects, Staff, etc…) ★ Other complementary sources ○ World University Rankings ○ Online academic profiles ○ ….
- 10. Types of sources ★ External database ○ In-Cites for bibliometric indicators (Around 60% of the report) ★ Internal databases ○ e-proyecta: internal management database for projects ★ Other complementary sources ○ ARWU (Shanghai) - Benchmarking and verification of scientific policies ○ Google Scholar Profiles to identify outstanding researchers and promote internet visibility
- 11. Incites for reports ★ At the University of Granada we use InCites from Clarivate Analytics. We recommend this bibliometric suite if: ○ you are working in large and multidisciplinary institutions ○ you don't have time for normalization and data cleaning ★ Advantages: indicators are already calculated, you can download raw data and use them for your report ★ Disadvantages: non-normalization for authors, inaccurate information for institutions due to errors in the organization enhanced field... Bibliometric suites (Incites or Scival) are expensive (between 35.000- 60.000 Euros) depending institution´s size.
- 12. Incites for reports The main advantage of InCites is provided useful datasets!!
- 14. What indicators - Which bibliometric indicators should you use? - Easy interpretation. Complex indicators are difficult to comprehend and should be avoided. - Standard indicators approved by the international community. - They have to reflect the different dimensions of research performance
- 15. What indicators Useful indicators for reports. Four types of dimensions seven indicators Lorem ipsum dolor sit amet at nec at adipiscing 03 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Lorem ipsum dolor sit amet at nec at adipiscing 02 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Lorem ipsum dolor sit amet at nec at adipiscing 01 ● Donec risus dolor porta venenatis ● Pharetra luctus felis ● Proin in tellus felis volutpat Excellence04 ● % Highly Cited Papers ● Papers in top journals (Science & Nature) Impact03 ● % of papers in first quartile journals ● Category Normalized Citation Impact Collaboration02 ● Number of documents with international collaboration Production01 ● Number of documents ● Number of citable documents
- 16. Contextualization of performance ★ Bibliometric indicators make sense when comparing with different aggregation levels, i.e. compare university with the national average. How is our institution performing in comparison with the national average?. ★ Some indicators for benchmarking are the Category Normalized Citation Impact (InCites), relative indicators like percentage of papers in first quartile journals or percentage of papers in international collaboration ★ In Granada, we compare different indicators with three geographical regions: Spain, European Union and USA ★ Without contextualization there is no meaning!
- 17. Contextualization of performance ★ Example for the publication profile in high impact factor journals ★ Indicator > Percentage of papers in first quartile journals ★ The publication profile of the University of Granada is quite similar to the national and international standards
- 18. ★ It is important to compare also our institution with other similar institutions. We have to select a coherent and homogeneous benchmarking group taking into account at least this variables: ○ Size of research output How many papers? ○ Size of staff How many faculty? ○ Same institutional objectives Focus on teaching or research? ○ Similar disciplinary profile Humanities, Life Sciences…? ★ At the University of Granada we compare our results with Spanish historical universities with a multidisciplinary profile. ★ Data from other institutions is retrieved from the Shanghai Ranking Benchmarking & Comparisons
- 19. Benchmarking & Comparisons ★ Example: benchmarking of the university of Granada with Spanish universities. ★ UGR performs well according to the number of citable papers, impact and collaboration, but has to improve the share of papers in first Quartile journals All universities perform better than Granada according based on the percentage of papers in First Quartile Journals
- 20. ★ If we are working at universities it is important also to represent in a bivariate graph the position of our institution within our national university system Benchmarking & Comparisons Nr of Citable Papers Cat.NormalizedCitationImpact UGR Impact World Impact
- 21. ★ Another question that we have to adress in our report is the thematic profile of our institutions; detecting best fields fields and disciplines ★ For this we must consider the use of different aggregation levels. Select at least one general level for an overview of the fields (for example Essential Science Indicators, 22 fields) and a more specific discipline level (for example Web of Science Categories, more than 200 categories). ★ Differentiate always between Science and Social Social Sciences / Humanities Fields and disciplines
- 22. Fields and disciplines Cat.NormalizedCitationImpact Nr of Citable Papers 1 Clinical Medicine, 2 Physics, 3 Social Sciences, 4 Chemistry, 5 Geosciences, 6 Engineering, 7 Mathematics, 8 Psychology, 9 Agricultural Science, 10 Computer Sciences, 11 Ecology, 12 Biology, 13 Plant & Animal Science, 14 Neuroscience, 15 Space Science, ... We can see the general scientific profile of the UGR according ESI classification. In this case we can identify very productive but low impact areas (1, 3, 4); very productive and high impact areas (2); relatively productive but with high impact areas (15) and not productive and low impact areas (14, 17, ….) Take care when choosing a classification, for example ESI does not have an specific field for the Humanities Impact < World Impact < World
- 23. Fields and disciplines ★ In this example we have the second level of presentation of the data at the disciplinary level, the WoS categories applied to the social sciences and the humanities.
- 24. Combining fields and institutions ★ Another interesting option to complete the report is to compare the performance of our institution with other universities in different scientific fields. ★ At University of Granada we check its position in five different fields (Natural Science, Engineering, Health Sciences, Social Sciences and Humanities) and compare its results with the Spanish university system.
- 25. Combining fields and institutions Nr of citable papers Category Normalized Citation Impact In this example we can see the position of the University of Granada in the field of Natural Sciences according to two different bibliometric indicators.
- 26. ★ We use the number of projects and total funding in competitive programmes to contextualize our results. We consider two calls a) Spanish R&D National programmes and b) European funding from the European Commission. ★ Nr. of project and total funding are important indicators but also success rate (Nr of applications / Proposals selected for funding). It is important also to compare these indicators with national or UE average and do benchmarking with other institutions Non bibliometric Indicators: funding In order to complement the information provided by bibliometric indicators we can include information on other inputs
- 27. Non bibliometric Indicators: funding ★ In the University of Granada we focus on national research programmes. In this example we can see the results of four Spanish universities in the last year. We can see that the University of Granada is the second university in Spain with a larger number of projects conceded
- 28. Indicators at author level ★ For authors we have included the researchers with the highest H-index in Web of Science for different scientific categories
- 29. Indicators at author level ★ Finally, we include the most outstanding researchers in "Google Scholar Profiles". We review all profiles and only include those with correct information. We have analyzed a total of 2000 University of Granada profiles and approved 1,700. These profiles have been classified into five different scientific areas. In the report we present a summary of the 200 most cited researchers Link to google scholar profile
- 30. Making data available online
- 31. Transparency and availability of data ★ To promote institutional transparency, reproducibility, verification of results and data reuse by other departments, we share our data in an standardized way. ★ Livemetrics Portal. We have developed a portal that includes a dynamic version of the report with the main indicators
- 34. Granada and Modena - Collaboration
- 35. Videoreports
- 36. Daniel Torres-Salinas Bibliometrics in practice: how to generate reports for institutions.