Applying ‘best fit’ frameworks to systematic review data extraction
1 like1,760 views
This document discusses applying 'best fit' framework methodology to systematic review data extraction, specifically to determine the skills and knowledge librarians need to assist researchers with data management, mining, and visualization. The authors conducted a systematic review identifying relevant studies and employing a framework synthesis approach to analyze qualitative data and extract competencies required by librarians. Challenges included identifying suitable frameworks and ensuring objectivity, while the next steps involve refining identified themes and developing an evidence-based model.
Applying ‘best fit’ frameworks to systematic review data extraction
- 1. Applying ‘best fit’ frameworks to systematic review data extraction Andrea Miller-Nesbitt, Catherine Boden, Andrew Booth, et al. 7t h International Conference on Qualitative and Quantitative Methods in Libraries, Paris France May 2015
- 2. Overview • Background • Systema2c review process • ‘Best fit’ framework methodology • Applica2on of methodology to our project
- 3. Background Conduct a systema2c review to address one of the 15 ques2ons iden2fied in the MLA Research Agenda: Appraising the Best Available Evidence
- 4. Research question What skills and knowledge must librarians possess in order to be able to design tools to help researchers visualize, mine, and otherwise manage large and complex data gathered during both quan2ta2ve and qualita2ve research?
- 5. Research question What skills and knowledge must librarians possess in order to be able to design tools to help researchers visualize, mine, and otherwise manage large and complex data gathered during both quan2ta2ve and qualita2ve research?
- 6. *Catherine Boden Brooke Billman Lorely Ambriz Andrea Miller-‐Nesbitt Martin Morris Andrew Booth Abby Adamczyk Anne Woznica Keith Engwall Rienne Johnson Betsy Clark
- 7. Search Databases: PubMed, Embase, ACM, LISA, LISTA, ERIC, Web of Science, WorldCat Date limits: Ar2cles • 2000 to May 2014 Books • 2005 to May 2014
- 8. Preliminary results Records aTer duplicates removed (n=3910) Records screened (n=3910) Full-‐text ar2cles assessed for eligibility (n=165) Ar2cles included in qualita2ve analysis (n=26*) Data visualiza2on (n=7) Data mining (n=9) Data management (n=24) Records excluded in 2tle abstract screen (n=3745) Full-‐text ar2cles excluded (n=70*)
- 9. Data extraction “Best fit” framework synthesis Large result set Time constraints Large research team MulEple facets within research quesEon
- 10. Framework synthesis • Deduc2ve process used for systema2c reviews • Highly structured approach to analyzing qualita2ve data • A priori framework is iden2fied or developed from a range of sources • Clearly defined themes in order to ensure transparency, consistency and speed of data coding
- 11. ‘Best fit’ framework synthesis “The ‘best fit’ framework synthesis method offered a means to test, reinforce and build on an exis2ng published model, conceived for a poten2ally different but relevant popula2on…this approach produces a rela2vely rapid, transparent and pragma2c process.” (Carroll et al., 2013, p1)
- 12. Research ques2on Iden2fy ‘best fit’ frameworks, conceptual models or theories Iden2fy relevant studies for analysis Generate a priori framework using thema2c analysis Extract data from included studies Code evidence from included studies against a priori framework Create new themes by doing thema2c analysis on evidence that cannot be coded against the framework Incorporate new themes into a priori framework to produce new conceptual model (Carroll et al., 2013, p.3)
- 13. ‘Best fit’ framework(s) synthesis • DigCCurr Data management • TBD Data mining Data visualiza2on • TBD
- 15. Data extraction form “what are the competencies described in the ar2cle for designing tools that support archival storage?” “what are the competencies in the ar2cle for designing tools for valida2on and quality control of digital objects/packages?” … “descrip2on of competencies relevant to the design of tools for data management that did not fit in the categories above” “there is insufficient data to be extracted with regard to research data management” “record any obvious issues about study quality”
- 16. Challenges • Iden2fying appropriate frameworks • Lack of granularity • Missing various concepts (especially tools) • Did not adequately address ‘competencies’ • Maintaining objec2vity • Ensuring we do not force data into the a priori framework
- 17. Next steps • New themes iden2fied • Relevance of data management, mining or visualiza2on frameworks to librarians’ roles • Applica2on of ‘best fit’ framework methodology to LIS research
- 18. Projected outcome Generate an evidence-‐based model, that iden2fies the competencies required of librarians involved in the design of tools used for data management, mining or visualiza2on.
- 19. Selected references Barnek-‐Page, E., & Thomas, J. (2009). Methods for the synthesis of qualita2ve research: a cri2cal review. BMC Medical Research Methodology, 9(1), 1-‐11. Carroll, C., Booth, A., & Cooper, K. (2011). A worked example of "best fit" framework synthesis: a systema2c review of views concerning the taking of some poten2al chemopreven2ve agents. BMC medical research methodology, 11(29). Carroll, C., Booth, A., Leaviss, J., & Rick, J. (2013). "Best fit" framework synthesis: refining the method. BMC medical research methodology, 13(1), 37. Dixon-‐Woods, M. (2011). Using framework-‐based synthesis for conduc2ng reviews of qualita2ve studies. BMC medicine, 9(1), 39. Eldredge, J. D., Ascher, M. T., Holmes, H. N., & Harris, M. R. (2012). The new Medical Library Associa2on research agenda: final results from a three-‐phase Delphi study. J Med Libr Assoc, 100(3), 214-‐218. doi: 10.3163/1536-‐5050.100.3.012 Eldredge, J. D., Harris, M. R., & Ascher, M. T. (2009). Defining the Medical Library Associa2on research agenda: methodology and final results from a consensus process. J Med Libr Assoc, 97(3), 178-‐185. Ritchie, J., & Spencer, L. (1994). Qualita2ve data analysis for applied policy research. In A. Bryman & R. G. Burgess (Eds.), Analyzing qualitaNve data (pp. 173-‐194). London; New York: Routledge.
- 20. Questions? Andrea Miller-‐Nesbi McGill University, Montreal, QC andrea.miller-‐nesbik@mcgill.ca