Clements - Visualizing the impact of mission driven organisations
1 like326 views
This document summarizes research on measuring the impact of mission-driven organizations like NASA. It discusses challenges in quantifying new knowledge generated by research and how it benefits stakeholders. The document outlines NASA's goals for the International Space Station regarding scientific, technological and educational purposes. It also provides statistics on the breadth of scientific disciplines studied on the ISS and growth in citations to this research, demonstrating diffusion of knowledge within space journals and more broadly. In conclusion, it acknowledges key contributors and states that a "science of science" approach can objectively describe ISS science outputs and impacts.
Clements - Visualizing the impact of mission driven organisations
- 1. Visualizing the impact of mission driven organizations: Multiple perspectives on knowledge flows and impact from research ©2016,CenterforKnowledgeDiffusion,Allrights reserved Margaret M. Clements, Center for Knowledge Diffusion Molly Macauley, Resources for the Future mclements6@yahoo.com, macauley@rff.org, Please do not cite or quote without permission of the authors. The financial support of the US National Aeronautics and Space Administration under Contract NNJ12GA17C is gratefully acknowledged. © 2016, Center for Knowledge Diffusion, All rights reserved
- 2. Science as New Knowledge: ( “delta K over K”) (NASA Chief Scientist 2011) Common challenges in corporate R&D labs, at universities, and in government • How to measure and count new knowledge • How to characterize its significance for the agency, or corporate shareholders, or promotion and tenure review, or funders, and … for society • How best to use this information to encourage new science and innovation (and manage internal resources as science managers and leaders) and • How, what, by whom, and why to communicate credibly to diverse stakeholders and in turn, their constituencies • Revision of knowledge (major as well as minor revision – delta k / k is a ratio – revise textbook) © 2016, Center for Knowledge Diffusion, All rights reserved
- 3. White House, June 28, 2010, The National Space Policy of the United States of America • “(The ISS is to be utilized) for scientific, technological, commercial, diplomatic, and educational purposes; • Support activities requiring the unique attributes of humans in space; • Serve as a continuous human presence in Earth orbit; and • Support future objectives in human space exploration” US GAO, March 28, 2012, NASA: Significant Challenges Remain for Access, Use, and Sustainment of the International Space Station • “NASA must ensure that the management of the ISS national laboratory results in effective utilization of the station for its primary purpose – scientific research.” • “The ISS was constructed to support three activities: scientific research, technology development, and development of industrial applications” ISS Science Mission © 2016, Center for Knowledge Diffusion, All rights reserved
- 4. US House of Representatives, 113th Congress, 1st Session, “National Aeronautics and Space Administration Authorization Act of 2013, June 12, 2013 -- Discussion Draft • “The (NASA) Administrator shall provide…criteria for defining the ISS as a research success” and a strategic plan for conducting competitive, peer-reviewed research in physical and life sciences and related technologies on the ISS.” White House 8 January 2014, ISS Extension to at least 2024 • The ISS is a unique facility that offers enormous scientific and societal benefits. The Obama Administration’s decision to extend its life until at least 2024 will allow us to maximize its potential, deliver critical benefits to our Nation and the world, and maintain American leadership in space. ISS Science © 2016, Center for Knowledge Diffusion, All rights reserved
- 5. White House, Winter 2015, FY2016 Budget Request Executive Summary, p. SUM-5 This budget request strongly supports ISS research. Scientists and engineers will develop and execute experiments and technology demonstrations in diverse disciplines including physics, biology, materials science, robotics, communications, and human physiology. Insights gained from these studies will be essential for planning future deep space missions, including to Mars. The Center for the Advancement of Science in Space (CASIS), the research management organization for the ISS National Laboratory, will continue to enable federal, academic, and commercial research activities. Exciting research will include potential medicines and interventions that will improve human health both in space and here on Earth. ISS Science Mission, continued ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 6. Research is motivated by: Consideration for Applied Use QuestforFundamentalUnderstanding Quadrant Model of Scientific Research (Stokes, 1997) ISS Science in Theories of R&D ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 7. US National Aeronautics and Space Administration http://www.nasa.gov/mission_pages/station/main/onthestation/facts_and_figures.html About the size of a football field, with the cubic capacity of a five-bedroom house, orbiting about 220 miles above the Earth The ISS: Physical Capacity and Operating Characteristics
- 8. Credit:http://www.bing.com/images/search?q=diagram+of+space+station&id=8CBCF3B31624B0AE910A1E565DD2AA9C60F0928B&FORM=IQFRBA#view=detail&id=8CBCF3B31624 B0AE910A1E565DD2AA9C60F0928B&selectedIndex=0 The ISS and International Collaboration
- 9. OECD Handbook on Measuring the Space Economy UCSD Map of Science US Science of Science Policy Historical Studies in the Impact of Spaceflight ©2016,CenterforKnowledgeDiffusion,Allrights reserved Approach Background
- 10. Approach Rationale Grant agencies, Administrators, R&D managers: to monitor grant flows, research developments, funding strategies and patterns; to identify scientific frontiers, the dynamics of scientific fields, and emerging complementary fields. NSF, NIH, USDA, NOAA, etc. Researchers and students: to detect social networks, “invisible colleges,” potential and cross-disciplinary collaborators, the interconnectedness of science Industry, national security agencies: to identify major science results, ascertain how knowledge is diffusing, see gaps and promise in research directions that industry might want to pursue Agencies and Librarians: unique visual interface to digital resources Society: to dramatically improve the access to, communication of, and utilization of the public’s investment in science ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 11. Table 1. ISS PSD Summary of Outputs (from ISS Office) Research Program Journal Conference Patent Technical Paper Other Biology and Biotechnology 235 8 1 Earth and Space Science 52 4 Human Research 171 11 3 5 Physical Science 94 33 2 Educational Activity 6 6 4 Technology 27 69 14 Exploration 5 24 5 2 Summary/Review 13 23 Data: a subset of ISS research program areas Note: entries not updated; cohort totals 888 records and over 4300 first-order citations to them – as of November 2013 (bulk of main data, new entries can be added) ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 12. Disciplines Linked by Shared Investigations • Each segment of the arc represents a discipline • Link shows that two disciplines share at least one investigation. • Wider links indicate more shared investigations. • Disciplines are linked based on the investigations cited in published paper. • If a paper in one discipline cites an investigation and a paper in another discipline also cites that investigation, the two disciplines are linked by that investigation. • Hover over disciplines to show how closely linked it is to other disciplines. Click on a link to see which investigations are shared between them. ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 13. © 2016, Center for Knowledge Diffusion, All rights reserved
- 14. Disciplines and Shared Investigations ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 15. • Science of science – a type of science analysis and assessment -- is a key federal initiative, led by OSTP/OMB/White House and NSF & with the participation of the science-oriented agencies (NIH, USDA, DoI, DoE, EPA, NASA) • Science of science approaches seek to go beyond mere “counts” of publications (outputs) to ascertain and demonstrate “delta K over K” (outcomes) – that is, the generation and diffusion of new knowledge through networks of people and their formal and informal exchanges of ideas • Salient attributes of our approach are that it is objective and external to syntheses prepared by the agency Key Findings ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 16. • ISS science – in particular, as inventoried and curated by the ISS science office -- can be described by the science of science approach (that is, methods, analytical studies, and visual displays of results) – for use by decisionmakers & to help inform the science community as well as the interested public • Using science of science assessment, the ISS science portfolio is extraordinarily impressive in breadth (the range of science disciplines) and impact (growth and rates of growth in citations to ISS research) both within specialized space journals – evidence of the diffusion of specialized knowledge to benefit space activities • In mainstream (non-space) journals – evidence of the diffusion of knowledge for broader societal benefit • Salient attributes of our approach are that it is objective and external to syntheses prepared by the agency ©2016,CenterforKnowledgeDiffusion,Allrights reserved
- 17. Acknowledgements • Michael J. Stamper • Thomas Grenfell Smith • Joseph Cottam • David Ernst • Project Steering Committee © 2016, Center for Knowledge Diffusion, All rights reserved