Cloud processing close to the Earth data
1 like1,445 views
The document discusses the NextGEOSS initiative, aimed at improving access to Earth observation data through cloud processing and open data policies to support research and societal applications. It highlights the importance of long-term earth observation data in addressing challenges such as climate change, biodiversity loss, and resource management, while emphasizing collaboration among public and private sectors. The initiative focuses on creating a federated data hub, empowering users with advanced tools for data access, analysis, and application development.
Cloud processing close to the Earth data
- 1. Cloud Processing close to the Earth data Herve Caumont Terradue Srl. NextGEOSS KOM - Lisbon, January 18th 2017
- 2. Agenda Foreword 1. A new workplace 2. When Earth data matters 3. Hubs and federations 4. For knowledge producers Final remarks
- 3. Foreword Applied research serves society, its funding is focused to solve current or future societal challenges To measure and understand the earth conditions, new and improved numerical models are leading the way, along with improved data assimilation pipelines. How will the Earth Observation community better support this goal ? “major technical challenge is the complexity of the data and analytics procedures across disciplines (rather than the size of the data per se)” Realising the European Open Science Cloud EC Directorate-General for Research and Innovation
- 4. A New Workplace Our road to bring EO data mainstream 1
- 5. NextGEOSS is open NextGEOSS builds heavily on openness, especially Open Data policies, to deliver the European GEO data hub. Taking heritage of the GEO 2005-2015 achievements, promoting the use of Earth Observations in decision making, and innovating for GEOSS, a set of coordinated, independent EO, information and processing systems. Looking at the next decade, NextGEOSS will engage with communities by promoting innovative GEOSS powered applications from Europe. Cloud processing close to the data shall be a fast, cost effective and user friendly capability so that application developers can focus on creating added value products. Terradue Cloud Platform supports NextGEOSS approach to Cloud
- 6. Terradue Cloud Platform Integration layers: ▪ Connecting scientists to the Web and Cloud resources empowering their research ▪ Tailored for Earth Sciences data integration & analysis Partnership mindset: ▪ Dedicated entry points for different types of value adders Since 2013, we’ve supported more than 40 companies and research institutes in accessing earth observation data & deploying processing services
- 7. Terradue Cloud Platform 2009-2014 2015-Forward ESA G-POD G-POD Cloud Experiment GEO Supersites Virtual Archive EO Thematic Platforms GENESI-DR/DEC GEOWOW MELODIES H2020 Projects 2006-2008 ▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform, with network effects carrying business opportunities
- 8. Terradue Cloud Platform 2009-2014 2015-Forward ESA G-POD G-POD Cloud Experiment GEO Supersites Virtual Archive EO Thematic Platforms GENESI-DR/DEC GEOWOW MELODIES H2020 Projects 2006-2008 Technological innovations, domain standards takeup ▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform, with network effects carrying business opportunities
- 9. Terradue Cloud Platform 2009-2014 ESA G-POD G-POD Cloud Experiment GEO Supersites Virtual Archive EO Thematic Platforms GENESI-DR/DEC GEOWOW MELODIES H2020 Projects 2006-2008 Technological innovations, domain standards takeup 2015-Forward Learning from experiments, capacity building ▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform, with network effects carrying business opportunities
- 10. Terradue Cloud Platform ▪ Co-funded ESA and EC projects, in a common vision ▪ Validation of evolutions via user engagement on the platform, with network effects carrying business opportunities 2009-2014 2015-Forward ESA G-POD G-POD Cloud Experiment GEO Supersites Virtual Archive EO Thematic Platforms GENESI-DR/DEC GEOWOW MELODIES H2020 Projects 2006-2008 Operations level agreements, user engagement Technological innovations, domain standards takeup Learning from experiments, capacity building
- 11. Enabling ‘network effects’ What does a Platform mean nowadays ? ✓ A lever of Web and Cloud technologies ✓ A business model for value co-creation ✓ A framework to bring innovation to new or larger communities
- 12. Terradue Cloud Platform A workplace & APIs to connect Data Providers, Developers, Integrators and information producers Providers Consumers ▶ Mission: ▶ Rapid prototyping and benchmarking of algorithms ▶ Seamless data access whatever stage of the process ▶ Automated data processing ▶ Scalable resource management ▶ Information conveyance
- 13. When Earth data matters The ‘decades of observations’ challenge 2
- 14. Long time series for earth science Earth observation data is a unique source of global measurements over decades Source: H. Laur, 2nd Workshop European Cloud Computing Strategic Plan Heidelberg, 26 Oct 2011 Source: C.Minet et al., 2012 When calibrated and combined with other sources, it empowers validation & interpolation models
- 15. Earth observation categories Atmosphere Cryosphere Source: GMES Example of societal benefits Support the enactment of policies such as the monitoring of greenhouse gas emissions Example of societal benefits Document both climate change trends and new economic opportunities in the Arctic based on sea ice monitoring Connect existing activities to NextGEOSS
- 16. Earth observation categories Geosphere Hydrosphere Source: GMES Example of societal benefits Estimate erosion rates sending sediments and nutrients into waterways Mitigate geohazards risks impacting human settlements Examples of societal benefits Deliver advice to farmers based on weather conditions & soil moisture Sustain ship routing efficiency based on ocean conditions Connect existing activities to NextGEOSS
- 17. Earth observation categories Biosphere Source: GMES Examples of societal benefits Document Essential Biodiversity Variables to improve ecosystem benefits in face of increasing anthropogenic pressures Monitor the state and trends of ecosystems in biogeographical regions (e.g. mountain, arid, semi- arid, coastal and marine) for internationally recognized protected aras and to define the needs of future protected areas Connect existing activities to NextGEOSS
- 18. Illustration from the GEO Geohazard Supersites
- 19. NextGEOSS challenge: Private sector ❏ overcome the fragmentation of the European landscape of existing public & private Earth Observation data infrastructures ❏ maximise their combined exploitation
- 20. Socio-economic open datasets govdata.de data.gov.uk data.gouv.fr catalog.data.gov PublicData.eu Private sector ▪ Address requirements for the development of the European digital economy ▪ Bring together multi-source assets having an open data policies ▪ Help more researchers & developers to compose data-driven applications NextGEOSS foreseen impact:
- 21. Hubs and federations Empowering the commons 3
- 22. A federated access for Earth Observations data
- 30. Data & processors swarming Data Staging from distributed repositories Data packages capturing input needs Deliver new products and analysis results towards community hubs Appliance deployment to distributed hosts in the cloud NextGEOSS Service Suite
- 31. Architectures & Innovation ▪ Web and Cloud services ▪ Standard App containers ▪ Mobile technologies ▪ Global Network operations ▪ Open Data Policies ▪ System of systems governance "the architecture of a complex technical system fundamentally influences, and is influenced by, the economic system that drives the development, production, and use of the system" Internet Architecture and Innovation MIT Press, Barbara van Schewick
- 32. For knowledge producers Addressing value creation chains 4
- 34. Data access Research and operational data infrastructures Copernicus service infrastructures Within the NextGEOSS consortium: GEOSS registered resources Copernicus scientific & public access NextGEOSS Data Hub Faceted search User feedback Data mining QoS
- 35. Data staging OGC OpenSearch API Data Buckets API Enablers for Community hubs NextGEOSS Data management Data I/O workflow automation in a distributed environment
- 36. Cloud bursting NextGEOSS support to community hubs in requesting, managing and accessing resources for cloud bursting Covering both the open science and commercial applications opportunities
- 37. Experience with community hubs 5 Climate Change Carbon market Climatology Oceanography Ship logistics Land management Water quality Desertification Biodiversity Urban accounting Volcanology Seismic analysis Disaster Risk Reduction EC FP7 GEOWOW (2011-2013) EC FP7 SenSyF (2013-2015) EC FP7 MELODIES (2014-2016) ESA Exploitation Platforms (2014 onwards) 8 14 35+ H2020 EcoPotential (2015-2018) 30+ Campaigns (2013 onwards) NextGEOSS intended outreach through innovative research Pilots & business opportunity Pilots Atmosphere Marine Land Cold regions Biodiversity Air pollution & Health Water management Agriculture / Agronomy Food security Teritorial planning Solar energy Urban growth Smart cities Secure societies Citizen observatories
- 38. Final Remarks With NextGEOSS, state of the art technology will match a proven governance model, driven by the public sector with possible contributions from private entities, and deliver to society a new capacity to learn faster from earth observations, and make better informed decisions
- 39. References WMO Global Atmoshere Watch Implementation Plan 2016-2023 http://www.wmo.int/pages/prog/arep/gaw/documents/IPFinalDraftMay11.pdf GSNL 2017-2019 GSNL Implementation plan http://www.earthobservations.org/documents/gsnl/201607_GSNL_Implementation_plan_2017_2019.pdf Realising the European Open Science Cloud, EC Directorate-General for Research and Innovation https://ec.europa.eu/research/openscience/pdf/realising_the_european_open_science_cloud_2016.pdf Internet Architecture and Innovation, MIT Press, Barbara van Schewick https://mitpress.mit.edu/books/internet-architecture-and-innovation