SlideShare a Scribd company logo
Pilot Project for HDF5 Metadata
Structures for SWOT
Ed Armstrong and Steve Hughes
NASA Jet Propulsion Laboratory
2016 Summer ESIP Meeting
July 19, 2016
National Aeronautics and
Space Administration
Jet Propulsion Laboratory
California Institute of Technology
Pasadena, California
© 2016 California Institute of Technology.
Government sponsorship acknowledged.
2016 ESIP Summer Meeting
 First NASA mission to incorporate granule level ISO 19115-2
metadata attributes
 HDF5 Metadata Groups
 Produced via two methods
 Level 2: XML serialization (XML bindings in software)
 Level 3 and 4: XML style sheets conversions
 Essential Metadata Attributes (Required and Recommended)
 Both Series (Collection) and Dataset (Granule) level
 Examples: Space/Time. Production and distribution attribution.
Processing lineage. Versioning history. Abstract.
SMAP Metadata Review
2016 ESIP Summer Meeting
 HDF5 structure and metadata layout and the Essential Attributes in
the ISO Metadata Model
 Plan
 From the SWOT Information Archtecture (IA) and Use Cases, start
capturing the Essential Attributes in the ISO Metadata Model into a
knowledge representation framework.
 Stage the Essential Attribute information in the framework for
design analysis, write specification documents for users, and write
metadata snippets for the HDF5 structure. This work will focus on
identifying the thin “top-level ontology” in the SWOT IA.
 Extraction of the metadata to the framework.
 Generate a specification document (e.g. Data Dictionary) for the
metadata.
 Generate example metadata snippets appropriate for the HDF5 structure
SWOT Metadata Pilot Project
2016 ESIP Summer Meeting
 Exercise HDF Product Designer
 Create HDF5/JSON representations of metadata
contents
 Import into Product Designer
 Export snippets to ISO 19115
 Product Designer leverages the THREDDS ISO service
 Review and iterate
 TBD: Writing ISO metadata to HDF5
 Timeline: Next two months
HDF5 Metadata snippets

More Related Content

PPTX
ICESat-2 Metadata and Status
PPTX
SPD and KEA: HDF5 based file formats for Earth Observation
PPTX
Matlab, Big Data, and HDF Server
PPTX
Data Analytics using MATLAB and HDF5
PPT
Summary of HDF-EOS5 Files, Data Model and File Format
PPTX
Bridging ICESat and ICESat-2 Standard Data Products
PPTX
Utilizing HDF4 File Content Maps for the Cloud Computing
PPT
HDF5 Performance Enhancements with the Elimination of Unlimited Dimension
ICESat-2 Metadata and Status
SPD and KEA: HDF5 based file formats for Earth Observation
Matlab, Big Data, and HDF Server
Data Analytics using MATLAB and HDF5
Summary of HDF-EOS5 Files, Data Model and File Format
Bridging ICESat and ICESat-2 Standard Data Products
Utilizing HDF4 File Content Maps for the Cloud Computing
HDF5 Performance Enhancements with the Elimination of Unlimited Dimension

What's hot (20)

PPTX
Working with Scientific Data in MATLAB
PPTX
Incorporating ISO Metadata Using HDF Product Designer
PPTX
PPTX
MATLAB and Scientific Data: New Features and Capabilities
PPTX
HDF-EOS Data Product Developer's Guide
PPT
GES DISC Eexperiences with HDF Formats for MEaSUREs Projects
PDF
HDFCloud Workshop: HDF5 in the Cloud
PPTX
Moving form HDF4 to HDF5/netCDF-4
PDF
Working with HDF and netCDF Data in ArcGIS: Tools and Case Studies
PPTX
HDF and netCDF Data Support in ArcGIS
PPTX
HDF Group Support for NPP/NPOESS/JPSS
PPT
HDF-EOS to GeoTIFF Conversion Tool & HDF-EOS Plug-in for HDFView
PPTX
Efficiently serving HDF5 via OPeNDAP
PPTX
MATLAB, netCDF, and OPeNDAP
PPSX
Data Are from Mars, Tools Are from Venus
PPTX
HDF OPeNDAP Project Update and Demo
PDF
Using IDL with Suomi NPP VIIRS Data
Working with Scientific Data in MATLAB
Incorporating ISO Metadata Using HDF Product Designer
MATLAB and Scientific Data: New Features and Capabilities
HDF-EOS Data Product Developer's Guide
GES DISC Eexperiences with HDF Formats for MEaSUREs Projects
HDFCloud Workshop: HDF5 in the Cloud
Moving form HDF4 to HDF5/netCDF-4
Working with HDF and netCDF Data in ArcGIS: Tools and Case Studies
HDF and netCDF Data Support in ArcGIS
HDF Group Support for NPP/NPOESS/JPSS
HDF-EOS to GeoTIFF Conversion Tool & HDF-EOS Plug-in for HDFView
Efficiently serving HDF5 via OPeNDAP
MATLAB, netCDF, and OPeNDAP
Data Are from Mars, Tools Are from Venus
HDF OPeNDAP Project Update and Demo
Using IDL with Suomi NPP VIIRS Data
Ad

Viewers also liked (9)

PPT
Using visualization tools to access HDF data via OPeNDAP
PPTX
Scientific Computing and Visualization using HDF
PPT
PPTX
Hdf5 current future
PDF
Unidata's Approach to Community Broadening through Data and Technology Sharing
Using visualization tools to access HDF data via OPeNDAP
Scientific Computing and Visualization using HDF
Hdf5 current future
Unidata's Approach to Community Broadening through Data and Technology Sharing
Ad

More from The HDF-EOS Tools and Information Center (20)

PDF
HDF5 2.0: Cloud Optimized from the Start
PDF
Using a Hierarchical Data Format v5 file as Zarr v3 Shard
PDF
Cloud-Optimized HDF5 Files - Current Status
PDF
Cloud Optimized HDF5 for the ICESat-2 mission
PPTX
Access HDF Data in the Cloud via OPeNDAP Web Service
PPTX
Upcoming New HDF5 Features: Multi-threading, sparse data storage, and encrypt...
PPTX
The State of HDF5 / Dana Robinson / The HDF Group
PDF
Cloud-Optimized HDF5 Files
PDF
Accessing HDF5 data in the cloud with HSDS
PPTX
Highly Scalable Data Service (HSDS) Performance Features
PDF
Creating Cloud-Optimized HDF5 Files
PPTX
HDF5 OPeNDAP Handler Updates, and Performance Discussion
PPTX
Hyrax: Serving Data from S3
PPSX
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
PDF
HDF - Current status and Future Directions
PPSX
HDFEOS.org User Analsys, Updates, and Future
PPTX
HDF - Current status and Future Directions
PDF
H5Coro: The Cloud-Optimized Read-Only Library
PPTX
MATLAB Modernization on HDF5 1.10
HDF5 2.0: Cloud Optimized from the Start
Using a Hierarchical Data Format v5 file as Zarr v3 Shard
Cloud-Optimized HDF5 Files - Current Status
Cloud Optimized HDF5 for the ICESat-2 mission
Access HDF Data in the Cloud via OPeNDAP Web Service
Upcoming New HDF5 Features: Multi-threading, sparse data storage, and encrypt...
The State of HDF5 / Dana Robinson / The HDF Group
Cloud-Optimized HDF5 Files
Accessing HDF5 data in the cloud with HSDS
Highly Scalable Data Service (HSDS) Performance Features
Creating Cloud-Optimized HDF5 Files
HDF5 OPeNDAP Handler Updates, and Performance Discussion
Hyrax: Serving Data from S3
Accessing Cloud Data and Services Using EDL, Pydap, MATLAB
HDF - Current status and Future Directions
HDFEOS.org User Analsys, Updates, and Future
HDF - Current status and Future Directions
H5Coro: The Cloud-Optimized Read-Only Library
MATLAB Modernization on HDF5 1.10

Recently uploaded (20)

PPTX
Spectroscopy.pptx food analysis technology
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Approach and Philosophy of On baking technology
PDF
Machine learning based COVID-19 study performance prediction
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Electronic commerce courselecture one. Pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Network Security Unit 5.pdf for BCA BBA.
PPTX
sap open course for s4hana steps from ECC to s4
PPT
Teaching material agriculture food technology
Spectroscopy.pptx food analysis technology
“AI and Expert System Decision Support & Business Intelligence Systems”
Agricultural_Statistics_at_a_Glance_2022_0.pdf
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Approach and Philosophy of On baking technology
Machine learning based COVID-19 study performance prediction
20250228 LYD VKU AI Blended-Learning.pptx
Understanding_Digital_Forensics_Presentation.pptx
Electronic commerce courselecture one. Pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Encapsulation_ Review paper, used for researhc scholars
Mobile App Security Testing_ A Comprehensive Guide.pdf
Diabetes mellitus diagnosis method based random forest with bat algorithm
Review of recent advances in non-invasive hemoglobin estimation
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Network Security Unit 5.pdf for BCA BBA.
sap open course for s4hana steps from ECC to s4
Teaching material agriculture food technology

Pilot Project for HDF5 Metadata Structures for SWOT

  • 1. Pilot Project for HDF5 Metadata Structures for SWOT Ed Armstrong and Steve Hughes NASA Jet Propulsion Laboratory 2016 Summer ESIP Meeting July 19, 2016 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Pasadena, California © 2016 California Institute of Technology. Government sponsorship acknowledged.
  • 2. 2016 ESIP Summer Meeting  First NASA mission to incorporate granule level ISO 19115-2 metadata attributes  HDF5 Metadata Groups  Produced via two methods  Level 2: XML serialization (XML bindings in software)  Level 3 and 4: XML style sheets conversions  Essential Metadata Attributes (Required and Recommended)  Both Series (Collection) and Dataset (Granule) level  Examples: Space/Time. Production and distribution attribution. Processing lineage. Versioning history. Abstract. SMAP Metadata Review
  • 3. 2016 ESIP Summer Meeting  HDF5 structure and metadata layout and the Essential Attributes in the ISO Metadata Model  Plan  From the SWOT Information Archtecture (IA) and Use Cases, start capturing the Essential Attributes in the ISO Metadata Model into a knowledge representation framework.  Stage the Essential Attribute information in the framework for design analysis, write specification documents for users, and write metadata snippets for the HDF5 structure. This work will focus on identifying the thin “top-level ontology” in the SWOT IA.  Extraction of the metadata to the framework.  Generate a specification document (e.g. Data Dictionary) for the metadata.  Generate example metadata snippets appropriate for the HDF5 structure SWOT Metadata Pilot Project
  • 4. 2016 ESIP Summer Meeting  Exercise HDF Product Designer  Create HDF5/JSON representations of metadata contents  Import into Product Designer  Export snippets to ISO 19115  Product Designer leverages the THREDDS ISO service  Review and iterate  TBD: Writing ISO metadata to HDF5  Timeline: Next two months HDF5 Metadata snippets