2015 FOSS4G Track: Visualization and Analysis of Spatiotemporal Data using Free and Open Source Software at the National Renewable Energy Laboratory by Dan Getman
1 like428 views
The document discusses the National Renewable Energy Laboratory's (NREL) transition from proprietary to open source software for spatial analysis and application development. It describes NREL's geospatial data science team and their work analyzing renewable energy and energy efficiency data. The transition to open source provided benefits like flexibility, cost savings, and promoting integrated architecture, but also challenges like difficulty staffing roles that require expertise across multiple open source technologies. Examples of projects using open source include a renewable electricity futures study visualization and various spatial applications.
2015 FOSS4G Track: Visualization and Analysis of Spatiotemporal Data using Free and Open Source Software at the National Renewable Energy Laboratory by Dan Getman
- 1. Visualization and Analysis of Spatiotemporal Data using Free and Open Source at the National Renewable Energy Laboratory GIS In The Rockies Dan Getman | September 24, 2015
- 2. DOE National Labs: Academia Industry National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City
- 3. DOE National Labs: Academia Unicorn http://sovaeart.deviantart.com/art/Aura-183473554 http://en.wikipedia.org/wiki/Harvard_Law_School http://en.wikipedia.org/wiki/New_York_City Industry National Renewable Energy Laboratory
- 4. NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 16 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory
- 5. NREL : Department of Energy National Laboratory o ~ 2400 People Focused on o Energy Efficiency o Renewable Energy Geospatial Data Science Team o ~ 16 People Focused on o Spatial Analysis – Traditional GIS focused on EE and RE o “Big Data” and Spatial Statistical Analysis o Spatiotemporal Data Visualization o Spatial Application Development What’s The Point? o Making sure the decisions we make with RE/EE are good ones National Renewable Energy Laboratory
- 6. What are we talking about here? o Looking at a progression from proprietary to open source software for analysis and application development o What were the drivers for this? o What has it meant for our team? o What are the Good and Bad outcomes of the decision? o What are we able to do with all of this anyway? National Renewable Energy Laboratory
- 7. Starting Point: Analysis (5 years back) Traditional Spatial Analysis Big Data Analysis (Solar/Wind Data) C/C++ TB of Data National Renewable Energy Laboratory
- 8. Starting Point: App Development Featur e Server National Renewable Energy Laboratory
- 9. Starting Point: App Development Featur e Server National Renewable Energy Laboratory
- 10. Current : Spatial Analysis Pathway National Renewable Energy Laboratory
- 11. Current : Spatial Analysis Pathway National Renewable Energy Laboratory
- 12. Current : Spatial Analysis Pathway National Renewable Energy Laboratory
- 13. Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory
- 14. Current : Spatial Analysis Pathway Developer.NREL.Gov o Query raw datasets o Run NREL models o Download subsets of very large datasets o Build applications off of our services o API Umbrella is Open Sourced and is the source for API.DATA.GOV National Renewable Energy Laboratory
- 15. Current : Spatial Analysis Pathway National Renewable Energy Laboratory
- 16. Analysis o R and libs o Python and libs o Postgres/PostGIS o Quantum Open Source Hit List: Application Development o Ruby on Rails o Python and libs o Node JS o Geoserver o Postgres/PostGIS o Tons of JS libs o Backbone and Ember o D3 o jQuery o Leaflet o OpenLayers General o Github o Linux o HDF National Renewable Energy Laboratory
- 17. Benefits: o Flexibility and Scalability o Laptop -to- Server -to- Servers -to- HPC o Ability to process multiple billions of data values o Costs and Ease of Use o Experimentation without having to purchase software o So we can use the right tools for the problem at hand o Promotes Modular Design o Design to support substitution of tools as needed o Promotes Integrated Architecture o Analysis uses the same data and design as applications Open Source Reflections National Renewable Energy Laboratory
- 18. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP Open Source Reflections National Renewable Energy Laboratory
- 19. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory
- 20. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554
- 21. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554
- 22. Complexities: o Staffing can be difficult o Geospatial + Analysis + ESRI= NP o Geospatial + Analysis + Some SQL = NP o Geospatial + Visualization + ESRI = NP o Geospatial + Analysis + Really knows Postgres = o Geospatial + Visualization + Advanced JS = Open Source Reflections National Renewable Energy Laboratory http://sovaeart.deviantart.com/art/Aura-183473554
- 23. Examples of Our Work National Renewable Energy Laboratory o Spatiotemporal Data Visualization o Renewable Electricity Futures Study Visualization o 40k possible maps in one visualization o Spatial Application Development o Open Carto – Web Based Mapping Platform o HyDRA – Hydrogen Vehicle Technology o Geothermal Prospector o Wind Prospector for Philippines o Spatial Analysis o Creation of large spatiotemporal resource datasets
- 24. Let’s take a look at a few examples… National Renewable Energy Laboratory
- 25. Your Questions and Feedback are Welcome National Renewable Energy Laboratory Dan Getman – dan.getman@nrel.gov