The Trials and Tribulations of Software Development in Astronomy: Where are we now and where can we go from here?
- 1. The Trials and Tribulations of Software Development in Astronomy: Where are we now and where can we go from here? Kelle Cruz CUNY Hunter College
- 2. Astro at the City University of New York (CUNY) • Hunter College is a 4-year college in the CUNY system • PhD program based at the CUNY Graduate Center • CUNY Astronomers joined together to form CUNY Astro, based at the American Museum of Natural History (AMNH)
- 3. • ~15 Astro Faculty at various CUNY campuses • Galactic, Extragalactic, Cosmology, Theory, Computational, Observers. • American Museum of Natural History is our central location cunyastro.org
- 4. Comparative Brown Dwarf-Exoplanet Studies .org
- 5. More Astro in New York City
- 6. New Masters in Astro Program • Collaboration between www.gc.cuny.edu/astrophysics • Two year program combining coursework and a Masters’ research thesis. • Prepare students for success in a future PhD program via research experiences, coursework, and community. • Provide students with immediately marketable advanced degree for STEM employment. • Joint program between CUNY Astro and Center for Computational Astronomy. • Full funding thanks to the Simons Foundation. Masters in Astrophysics
- 7. Science was…
- 9. Science was… Photo credit: Harvard-Smithsonian Center for Astrophysics
- 10. Science is Software Photo credit: Jeyhan Kartaltepe
- 11. Software permeates our field • “Little” scripts • Small packages • Big packages • “Enterprise” • Simulations • Models • And much more… Kelle Cruz, March 2023 Credit: NOIRLab
- 12. And yet… We are not taught how to use computers to do astronomy. Kelle Cruz, March 2023
- 13. Astro2020 Decadal Survey: Better than Astro 2010! Finding: Software development has become an essential part of every sub-field of astronomy. However, software developers and large software development efforts are not adequately funded or supported by existing structures. But no “Recommendations”. And nothing about software education.Kelle Cruz, March 2023
- 14. I care about maximizing efficiency of science The goal of this blog, AstroBetter, is to provide information and tips about streamlining all the ways we use our Macs everyday.
- 18. We are not taught how to use computers to do science. Kelle Cruz, March 2023
- 19. The Looming Diversity Disaster Software development and computational training remains a privilege AND Software dev skills are now necessary to succeed in astro THUS the current landscape exacerbates the already un-level playing field. Kelle Cruz, March 2023
- 20. Inclusive practices, education, & broadening participation in software development needs to be a huge priority for all programs. Kelle Cruz, March 2023 The Looming Diversity Disaster
- 21. Science should be reproducible/replicable same data + same methods = same results Modified from https://xkcd.com/242/ https://arxiv.org/abs/1802.03311 - Barba 2018.
- 22. Describing code is hard “To calculate distances and areas on the solar-disc, firstly helioprojective coordinates are converted to heliographic coordinates using the world coordinate system available in the standard solar data analysis software…” doi: 10.1007/s11207-015-0808-7 Courtesy Danny Ryan Kelle Cruz, March 2023
- 23. We need to share our scripts • Normalize making code visible even if it’s embarrassing. • Looking at code should be a normal and expected part of the peer review process. Kelle Cruz, March 2023 Copyright: © bakhtiarzein - stock.adobe.com
- 24. We need to test our code Integration tests are just as important as uncertainty propagation. Kelle Cruz, March 2023
- 26. Software Syllabus • The shell • Basic programming • Environment management • Software engineering: Github, testing, documentation • Advanced programming: parallelization, optimization • (Anything is better than nothing.) Kelle Cruz, March 2023
- 27. Current Learning Methods • Self-taught: StackExchange, etc • Research: Advisors, group- mates. • Workshops: Software carpentry, Astropy, etc • Online tutorials • Topical Workshops: Gaia, LSST, etc. • LSSTC Data Science Fellowship • Hackdays & hackweeks • Programming hours and study groups • Some Coursework Kelle Cruz, March 2023
- 28. What we need • Recognition of the problem. • Willingness to do the work of changing curriculum. • Instructor training and resources. • Intro textbooks with readings and assignments. • More tutorials and better tutorial discoverability & indexing. • Degree program flexibility to take courses in other departments and/or via other institutions. Kelle Cruz, March 2023
- 30. Open Source Definition Software whose source code is made available under an open source license. An open source license grants anyone the rights to inspect, modify and distribute the code (with some possible caveats). https://opensource.org/osd Kelle Cruz, March 2023
- 31. Open Source Misconceptions Putting code on my webpage makes it open. No license = copyright, all rights reserved. Nobody else can use, copy, distribute, or modify your work without being at risk of take-downs or litigation. True False True False https://choosealicense.com/no-permission/ Kelle Cruz, March 2023
- 32. Open Source Misconceptions If I make my code open, I must support and maintain it. Making code open means other people can use and modify it, even if you abandon it. True False True False Kelle Cruz, March 2023
- 33. Open Source Misconceptions Anyone can participate in the development of open code Code can be distributed with an open source license but developed behind closed doors True False True False Kelle Cruz, March 2023
- 34. NASA Open-Source Science Initiative SMD’s new policy on Scientific Information As open as possible, as restricted as necessary, always secure ● Publications are made openly available with no embargo period. ● Research data and software are shared at the time of publication. ● Mission data are released as soon as possible and is freely available. ● Unrestricted mission software is developed openly. ● Recognizes software as a scientific product. ● Data should be released with Creative Commons Zero and software with permissive, commonly used licenses ● Encourages using and contributing to Open Source Software. Scientific Information Policy Website & FAQ NASA Open-Source Science Initiative SMD-41a Slide credit NASA and Steve Crawford
- 35. NASA Open-Source Science Initiative Sustaining Open Source Software NASA selected 16 proposals in 2021 supporting 22 different open source projects to provide sustainable support. NASA Open-Source Science Initiative NASA is putting money where its mouth is Slide credit NASA and Steve Crawford
- 36. NASA is putting money where its mouth is 5 NASA OSSI Funding F.15 High Priority Open-Source Science Supporting innovative open source tools, software, frameworks, data formats, and libraries. F.16 Supplement for Software Platforms Supplemental support to existing awards for usage of scientific platforms. Budget TBD. F.14 Transform to Open Science Training Tutorials showcasing open science in action and NASA cloud data, summer schools, virtual cohorts. F.2 Topical Workshops, Symposia, and Conferences Events, Hackathons, un-conferences, and challenges that build open science skills, Training in open science. Rolling deadline. F.7 Support for Open Source Tools, Frameworks, and Libraries Support and maintain open source tools, frameworks, and libraries that are significantly used by the SMD community. F.8 Supplemental Open Source Software Awards Supplemental award to encourage the modernization of legacy software and release as open source. NASA OPEN-SOURCE SCIENCE INITIATIVE Slide credit NASA and Steve Crawford
- 37. NASA Open-Source Science Initiative NASA Transform to Open Science (TOPS) A $40 million, 5-year mission to accelerate adoption of open science Strategic Goals: ● Support 20K researchers to earn NASA's open science badge ● Double the participation of historically excluded groups across NASA science ● Enable five major scientific discoveries through open science principles Engagement Incentives Coordination Capacity Sharing Join us as we embark on the 2023 Year of Open Science with NASA TOPS! 6 NASA is putting money where its mouth is Slide credit NASA and Steve Crawford
- 38. Increase the value of software: Be part of the solution • Prioritize your own learning. It’s a rapidly changing landscape. It’s ok to not be an expert, no one is. • Share all types of code, including scripts. Ask code to be made available during peer review. • Encourage open science & software engineering practices, especially in your research groups. • Bring open science & software engineering skills into the classroom & academic programs. Kelle Cruz, March 2023