2019 02-12 eosc-hub for eo

EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536.
eosc-hub.eu
@EOSC_eu
Tiziana Ferrari, Project Coordinator
Dissemination level: Public

12/02/2019 EuroGEOSS Meeting 2
• Introduction to the European Open Science Cloud and EOSC-hub
• EOSC technical approach EO data analysis challenge
• Today’s EOSC-hub / EGI support to NextGEOSS pilots, GEOSS and EO
exploitation platforms
Outline

12/02/2019 3
The federated infrastructure and supporting initiative providing
all researchers, innovators, companies and citizens
with seamless access to an open-by-default, efficient and cross-
disciplinary environment
for storing, accessing, reusing data, tools, publications and other
scientific outputs for research, innovation and educational purposes
About the European Open Science Cloud
Credits: EOSCpilot (https://eoscpilot.eu/)
EuroGEOSS Meeting

20 digital research infrastructures,
EGI, EUDAT CDI and INDIGO-DataCloud
jointly offering services, software and data
for advanced, data-driven research & innovation
A project overview

Mission
EGI
Federation
EUDAT
INDIGO-
DataCloud
Research
Infrastruct
ures
The EOSC-hub project mobilises providers of pan-
European relevance offering services, software and
data for advanced data-driven research and
innovation.
These resources are offered via the Hub – the
integration and management system of the
European Open Science Cloud, acting as
a European-level entry point for all stakeholders.

6
EOSC-hub Service catalogue
Arts and Humanities
Medical and Health Sciences
Physical Sciences Earth Sciences
Biological Sciences
Data Lifecycle Management, Compute,
Scientific Applications
e.g. Marketplace, EOSC AAI
Accounting and Monitoring
Helpdesk, Collaboration Platforms
Research
Enabling
Services The
Hub
Access
Enabling
Services
https://www.eosc-hub.eu/catalogue/
12/02/2019 EuroGEOSS Meeting

7
Support to the Research Data Lifecycle
Processing & Analysis
Data Management, Curation &
Preservation
Access, Deposition & Sharing
Access enabling Services
● B2FIND (data)
● Marketplace (Services)
● Applications on Demand
● Federated HTC & Cloud Compute IaaS &
PaaS
● Processing of sensitive data
● Jupyter Notebook
● Application DB (software & VM)
● B2DROP (data)
● B2Note (data)
● B2SHARE (data)
● DataHub
● Federated AAI. monitoring,
accounting
● SLA and order Management
● Security incident response and
policies
● Technical support & Training
● B2HANDLE
● B2SAFE
● European Certified Trusted
Repository
● Thematic data analytics
● Scientific Workflow Management,
Orchestration (DIRAC, PaaS
Orchestrator)
1
2
3
4
Discover & Reuse

8
Services by area
e-Infra
EGI
Federation
EUDAT CDI
INDIGO-
DataCloud
Humanities
Language
and
literature
(CLARIN)
Arts
(DARIAH)
Engineering
Environmen
tal
engineering
(sea vessels,
LNEC)
Civil
Engineering
(Disaster
Mitigation)
Medical
and Health
Sciences
Biological
Sciences
(ELIXIR)
Structural
biology
(WeNMR)
Natural
sciences
PHYSICAL SCIENCES
Astronomy (LOFAR)
Fusion (ITER)
High Energy Physics
(CMS and VIRGO)
Space Science
(EISCAT-3D)
EARTH SCIENCE
EO Pillar
GEO
Climate Research
(ENES)
Seismology (ORFEUS,
EPOS)
BIOLOGICAL
SCIENCES
Marine and
freshwater biology
(IFREMER)
Biodiversity
conservation
(LifeWatch)
Ecology (ICOS)

12/02/2019 9
• Federation of distributed data repositories supporting single sign on
• Transparent data access service
- Smart caching of data remotely stored
- Local access to data regardless of the cloud provider of choice
• Provisioning of data products
• Analytics of distributed data with Jupyter notebooks and publishing of
output data
EOSC scalable distributed data processing
challenge
EuroGEOSS Meeting

12/02/2019
EuroGEOSS Meeting
10
EOSC compute platform/Jan 2019
> 380 Meuro / year opex

Scientific computing demand 2010-2018
4.4 Billion CPU core wall time
delivered in 2018
> 1 Million computing cores
for the first time in the EGI
history
356 PB disk & 380 PB tape
storage
1170 open access
publications / year
31 large scale ESFRI
projects/landmarks
supported
+20%
utilization
of
computing
in 2018

12/02/2019 13
• Heterogeneous backend
storage
• Common interfaces
(Web, REST, POSIX,
CDMI)
• Common AAI with
Check-in
• Discovery of Datasets in
the EGI DataHub
Federation of data repositories
EuroGEOSS Meeting

12/02/2019 14
• Clients uses one ore more
providers to access data
• Data can be accessed over
multiple protocols
Transparent data access
EuroGEOSS Meeting

12/02/2019 15
• Site A hosts data & computing
resources
• Site B only hosts data
Site X can use data from A and B
• Without pre-staging
• Via pre-staging using APIs
• Local data access “à la” POSIX with
FUSE
Data caching
EuroGEOSS Meeting

12/02/2019 17
e.g. Multi-cloud analytics with Jupyter
Notebooks
EuroGEOSS Meeting

12/02/2019 EuroGEOSS Meeting 18
EO applications and data in the EOSC Marketplace
https://marketplace.eosc-portal.eu/

12/02/2019 19
EO exploitation services
The EO-Pillar service provides access to different services in the
field of Earth Observation (EO).
Data Access and
Computing services
• EODC Data Catalogue
Service
• EODC JupyterHub for
global Copernicus data
• OSX-Sentinel
• CloudFerro Data
Collections Catalog
• CloudFerro Infrastructure
• CloudFerro EO Finder
• CloudFerro EO Browser
EO Data Exploitation
services
• Geohazards Exploitation
Platform (GEP)
• MEA Platform
• Rasdaman EO Datacube
EO general user
services
• Sentinel Hub
• EPOSAR Service
Service Launch
Integration with
Hub
All services except MEA Platform
and EPOSAR
01/07/2018
EGI Check-in
EGI Cloud Compute
EGI DataHub
B2SHARE Monitoring
AccountingMEA and EPOSAR 01/06/2019
EO Pillar - VA Metrics
EODC EO DATA available: > 3 PB
OSS-X Sentinel: Number of published products
CloudFerro: EO data available: > 9 PB
CloudFerro: Number of users: 200
CloudFerro: EOBrowser Collections: 9

12/02/2019 20
ECOPotential VLab: managing protected
environments
• Europe’s protected areas and surrounds are
vulnerable to human-induced events and
processes including those associated with
climate change.
• As an example, the biodiversity of Doñana
National Park in Spain is affected by
agricultural expansion, and urban
development and climatic variability with all
impacting on water supply.
• Whilst large amounts of Earth Observation
data are available, these are vastly
underexploited for monitoring changes and
implementing management actions aimed
at nature conservation and sustainable and
wise use.
Map showing
combination of
daily, between
image observations
and annual change
in hydroperiod
superimposed on
the land cover
classification
EuroGEOSS Meeting

12/02/2019
21
Architecture Overview
VLab APIs
VLab
Portal and apps
Long-term
preservation
archives
Knowledge bases
Remote processing
services
Source code
Workflows
• Discover the service through the EOSC
Portal
• Identify available related workflows
related to the issue of interest
• Explore available data through GEOSS
• Execute the workflow taking advantage
of the EOSC infrastructure
EuroGEOSS Meeting

Platform based on virtualization & federation of
satellite EO data
▪ Provide services & support to the geohazards community
On-demand & systematic processing services
▪ Cloud Compute power, managing multi-tenant resources
Access to Copernicus Sentinels repositories
▪ Plus access to hundred TBs of EO data archives (ERS and
ENVISAT), and other EO missions (ALOS-2, Cosmo-Skymed
and TerraSAR-X) under CEOS WG Disaster and the GSNL
agreements
Geohazards Exploitation Platform | GEP

GEP | Cloud APIs, Hybrid Cloud
Openstack API -
powered by libcloud
CloudFerro IaaS
EODC Cloud
EC2 - powered by jclouds
Amazon Web Services
Terradue
Openstack API -
powered by libcloud
CreoDIAS
AWS
DIAS
T2
Openstack API -
powered by libcloud
ONDA
> 18 K Data Products generated/month

It’s a data driven systematic processing. The
service has followed a ramp-up process starting
from Dec 2016 until Aug 2017:
■ EU Tectonic area
■ World tectonic area (25%)
■ World tectonic area (40%)
It currently processes 150+ Sentinel-1 SLC pairs
per day.
DLR InSAR Browse Medium
Resolution Service
Supported by
BELNET-BEGRID (Belgium)
and
ReCaS Bari (Italy)
GEP | Data Driven Scheduled Processing
https://geohazards-tep.eu

P-SBAS stands for Parallel Small BAseline
Subset and it is a DInSAR processing chain for
the generation of Earth deformation time series
and mean velocity maps. Input: SLC (Level-1)
Sentinel-1 data.
CNR-IREA P-SBAS Sentinel-1
processing on-demand
https://geohazards-tep.eu
Will be
supported by
BELNET-BEGRID (Belgium)
GEP | User Driven Processing

Users
- Discover services through a
secure online catalogue with a
central access point.
- Access services via harmonised
access policies and Service Level
Agreements. Depending on the
customer’s needs, access will be
provided through a centrally
managed credit-based allocation
system and/or through long-term
resource allocation.
- Leverage a large portfolio of
generic and thematic services via
standard interfaces
- Pilot
Providers
- Contribute to EOSC with services
based on a harmonised corpus of
access, security and provisioning
policies.
- Federate services by relying on
harmonised processes and tools
for service integration and
management.
- Scale-up in-house capacity by
procuring services via a centrally
run procurement and purchase
framework.
- Participate in service enabling with
users
12/02/2019 27
EOSC Marketplace:
Value proposition of Users and Providers
EuroGEOSS Meeting

Service/Product
providers
• Contribute to the portfolio
& Integrate
• Promote, Train, Exploit
• Manage the Hub
Early adopters
• Pilot
• Use
• Co-develop
• Integrate with
federated AAI,
computing and
storage
Consumers
• Credit allocation &
exploit
• Support
• Improve with feedback
• Use
28
EuroGEOSS pilots roles in EOSC

Research Project Enabling Cycle
1. Evolve enabling
technologies
2. Integrate services
3. Provide → support →
access → consume
4. Engage with external
user communities and
providers, collect
requirements
30/01/19 29

12/02/2019 30
• Federated facilities 100% funded by member
states and national research performing
organizations
• Recovery of marginal costs
- Sponsored access
▪ National funding agencies
▪ EC contributions through the virtual access
instrument
▪ EC contribution via INFRAEOSC-01 procurement
action “OCRE”
- Pay for use
(Current) Business model
EuroGEOSS Meeting

EOSC-hub Week 2019
10-12 April, Prague
Launch of Call for Research Projects
for integrated service provisioning
In EOSC
Come and become a provider!
https://eosc-portal.eu/for-providers

12/02/2019 32
• The European Open Science Cloud will progressively federate research
data, applications, software and the computing facilities across Europe and
beyond, requiring data and service interoperability
- Co-provisioning of computing to data, with single sign own access
- Advanced data management and high-performance access technologies
• EOSC-hub interested in supporting the EuroGEOSS programme with DIAS
- Adoption of common standards of integrated DIAS and EOSC will allow
coordinated support to EuroGEOSS pilots from testing to go-to-market phase
Conclusions
EuroGEOSS Meeting

eosc-hub.eu @EOSC_eu
Thank you
for your attention!
Questions?

2019 02-12 eosc-hub for eo

More Related Content

What's hot (20)

Similar to 2019 02-12 eosc-hub for eo (20)

More from EGI Federation (13)

Recently uploaded (20)

2019 02-12 eosc-hub for eo