Interdisciplinary Research Collaborations: Evaluation of a Funding Program

17/05/2006 1Herhaling titel van presentatie
Interdisciplinary
Research
Collaborations:
Evaluation of a
Funding Program
Nadine Rons
Vrije Universiteit Brussel (VUB)
Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) &
Eleventh COLLNET Meeting, 19-22 October 2010, Mysore, India
Version 13.10.2010

October 2010 - Slide 2
6th WIS & 11th COLLNET
Structure
o  Place in the broader context of INTERDISCIPLINARITY
o  Origins & scope
o  Interdisciplinarity in research, studied bibliometrically
o  Focus: synergies between researchers from different disciplines
o  Research policy: search for best practices
o  Experiences from a FUNDING PROGRAM
o  Program presentation: procedures & criteria
o  Program evaluation: network composition, output & impact, feedback
o  Conclusions

Origins of interdisciplinarity
o  Human KNOWLEDGE divided in disciplines
o  categorization of scientific knowledge, subject to change
o  division in more specialized communities, focusing efforts in a
growing context of information produced and communicated
o  if institutionalized: shaping knowledge production & distribution
o  in their present form: fairly recent (±200 years), relatively stable,
instrument for science studies
o  Need for CROSSING THE BOUNDARIES
o  the ‘interdisciplinary’ nature of reality
o  political & socio-economic priority areas (nanotechnology, climate
research), ‘transdisciplinary’ step towards application
o  opportunities for progress through cross fertilization

Kinds of interdisciplinarity

Bibliometric studies of interdisciplinarity
o  Research publication DATABASES
o  Web of Science: created to “bring together material that would never
be collated by the usual subject indexing” in “an association-of-ideas
index” (Garfield, Science 122, 1955)
o  Scopus
o  Examples of empirical STUDIES
o  Cross-disciplinary citations (Porter and Chubin, Scientometrics 8, 1985)
o  Larger time lag (Rinia et al., Scientometrics 51, 2001)
o  ‘Natural’ shares of cross-disciplinary activity (Rinia et al., Scientometrics 54,
2002)
o  Evolution over time (Porter and Rafols, Scientometrics 81, 2009)
o  Measures for integration and specialization (Porter et al., Scientometrics 72,
2007)
Bordons et al. Analysis of cross-disciplinary research through bibliometric tools. In: Moed et al.
(Eds.), Handbook of quantitative science and technology research, pp. 437-456, 2004

Focus in current research policy
o  FUNDING programs / policies for interdisciplinary research
o  ‘Top down’, thematic
o  Open call, reserved fraction
o  ASSESSMENT of interdisciplinary research
o  Dedicated issue of Research Evaluation (Research Evaluation 15, 2006)
o  Heterogeneity
o  No generally accepted methodology yet
o  Search for best practices
o  Crucial: composition and functioning of the evaluation committee
o  VUB: HORIZONTAL RESEARCH ACTIONS (HOA) program
o  Set up in 2002, evaluated in 2009

HOA program focus
o  INTERDISCIPLINARY RESEARCH
o  Integrating expertise from different disciplines
o  Offering an all-encompassing expertise in a theme proposed
‘bottom up’
o  COLLABORATIONS
o  Between researchers from different departments / faculties
o  In the framework of a research project
o  Stimulated / supported by a dedicated funding program

HOA call & selection phases
o  CALL for applications
o  Applications incl. theme & its relevance, applicants & teams, project
description & budget, external collaborations
o  Selection COMMITTEE
o  Board of the Research Council
o  Broad scope, open attitude to different standards, coherence, good
overview of expertise and performances, views expected to surpass
disciplinary perspectives
o  PRE-SELECTION and SELECTION
o  Pre-selection: advice from two experienced reviewers per
application (of which 1 member of the Board)
o  Final selection: after oral presentation before the Board

HOA criteria
o  TOPIC
o  Interdisciplinary and innovative character, added value, importance for
science and society
o  PARTNERS
o  Separate strengths in disciplinary performance
o  All-encompassing expertise in the proposed theme
o  PROJECT and PLANNING
o  Planned function of the researcher(s) financed, embodying the
integration of expertise
o  Planned international network participations
≈ How panel members define a good proposal (Lamont et al., Research Evaluation 15, 2006)
≈ Grounds for evaluation suggested by researchers (Feller, Research Evaluation 15, 2006)

Program evaluation
o  Analysis of APPLICATION DATA
o  7 generations: (Co)applicants, affiliations to teams & faculties
o  -> Degree of interdisciplinarity, faculty participation
o  BIBLIOMETRIC study
o  First 4 generations: Co-publications & co-citations in Web of Science
o  -> Measures for output & impact and evolution over time *
o  SURVEY
o  Funded applications followed for 4 full years (full project length)
o  Validity of data & indicators used, opinions
* Not used in this context:
interdisciplinarity of references, breath of journal subject categories

Faculty participation
o  ALL FACULTIES are represented among the applicants
Faculties: MEdicine
and pharmacy,
SCience and bio-
engineering sciences,
ENgineering sciences,
ARts and philosophy,
EConomics, social and
political sciences, LAw
and criminology,
PHysical education
and physiotherapy,
PSychology and
educational science;
CA: Central
academic; linear
correlation coefficient
r=0,910 for N=9 (i.e.
8 faculties + central
academic staff).

Degree of interdisciplinarity
o  A majority of BROAD INTERDISCIPLINARITY
o  The total sample of applications links each of the 8 faculties to at
least 5 others.
o  Degree of interdisciplinarity: joining applicants from different faculty
clusters* (broad), from different faculties within one cluster
(medium), and from different departments within one faculty
(narrow). [* ‘Social Sciences and Humanities’, ‘Basic, Natural and
Applied Sciences’ or ‘Biomedical Sciences’]

Output & impact
o  BIBLIOMETRIC interdisciplinary OUTPUT & IMPACT
o  Co-publications: joint publications by applicants from different
departments
o  Co-citations: publications citing applicants from different departments
A: "Newly activated", B: Previously active", C: "Output only", D: "Citation impact only", E: "Not visible".
See also: Added value of funding visible after ±3 years (Rons, Proceedings 11th Conference on STI, 2010)

Feedback & opinions
o  Confirmed interdisciplinary SYNERGIES
o  The large majority (82%) of co-publications concern a synergy (as
opposed to an application of expertise from one field in another)
o  Confirmed OUTPUT & IMPACT representation
o  A majority of co-publications (89%) and co-citations (64%) is related to
the network topic and contain the majority of related output and
impact of various forms (next to 30% other output items and 11%
other impact items reported: contributions in proceedings, Dutch
language journal articles and books for professional audience, PhD’s
obtained and in preparation, obtained research funding)
o  Confirmed SPECIFIC NEEDS
o  Stimulating and necessary funding channel
o  Longer startup periods due to multiple differences in traditions

Program Evaluation: conclusions
o  DEGREE OF INTERDISCIPLINARITY and FACULTY
PARTICIPATION
o  The program attracts a potential present in all faculties for -
mainly ‘broad’ - interdisciplinary collaborations
o  OUTPUT & IMPACT
o  The program successfully selects & supports interdisciplinary
research synergies, and activates funded as well as unfunded networks
o  FEEDBACK & OPINIONS
o  The program stimulates interdisciplinary collaborations and responds
to a need for a dedicated funding channel
Presentation of the REPORT to the Research Council -> Program continued

Possibilities for future studies
Bibliometric characteristics of interdisciplinary research synergies
Feasibility of normalization in interdisciplinary domains
Indicators for ex-post evaluation of interdisciplinary research

THANK YOU FOR YOUR ATTENTION!
Nadine Rons
  Vrije Universiteit Brussel (VUB)
  VUB R&D Dept, Research Coordination Unit
  Pleinlaan 2, B-1050 Brussels, Belgium
  Nadine.Rons@vub.ac.be
  http://rd-ir.vub.ac.be/en_GB/people/show/id/554
  http://www.vub.ac.be/studyinginbrussels/index.html

Interdisciplinary Research Collaborations: Evaluation of a Funding Program

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Interdisciplinary Research Collaborations: Evaluation of a Funding Program (20)

More from Nadine Rons (10)

Recently uploaded (20)

Interdisciplinary Research Collaborations: Evaluation of a Funding Program