Status of HDF-EOS, Related Software and Tools

Status of HDF-EOS,
Related Software and Tools
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XIV
Champaign, IL
Sep. 29, 2010

1

TOOLKIT / HDF-EOS Support
•
•

Raytheon was awarded EOSDIS Evolution &
Development (EED) contract in May 2010
Maintenance support for TOOLKIT/HDF-EOS is
funded on EED through March 2015

• Current support







HDF-EOS 2 & 5
SDP & MTD Toolkits
HDF-EOS2 to GeoTIFF Converter (HEG)
HDF-EOS plug-in for HDFView
Other Toolkit/HDF-EOS related software
Toolkit/HDF-EOS/HEG helpdesk
2

ECS Support of HDF-EOS
• Both HDF4 and HDF5 based flavors of HDF-EOS are
part of ECS baseline.
• OS’s:

Solaris (10)
Linux (32 & 64-bit, including IA64)
Mac OS X (Power PC and Mac Intel)
Cygwin
Windows XP (MTD Toolkit, HDF-EOS2 & 5,
HEG, HDFView)

• Compilers: Fortran 77/90 , g77/pgf90
C, C++, gcc, g++
Intel Fortran (SGI Altix), gfortran (Mac Intel)
3

Downloadable Software
• DAACS are supported with
– TOOLKIT/HDF-EOS & HEG deliveries during
yearly DAAC releases or patch deliveries
– On demand

• Other users can download our software from
– our tools’ web sites
– directly from our ftp site:
ftp://edhs1.gsfc.nasa.gov/edhs
4

Downloadable Software
• SDP Toolkit (Science Data Processing Tools)
• MTD Toolkit (Metadata, Time/ Date Tools)
• HDF-EOS2 (HDF4 based)
• HDF-EOS5 (HDF5 based)
• heconvert (converts HDF4 - based Grid/Point/Swath
structures to HDF-EOS 5 equivalents)
• DumpMetadata (Dump HDF-EOS metadata)
5

Downloadable Applications
• HEG (HDF-EOS2 to GeoTIFF conversion)
• HDFView Plug-in, Java-based browser
(HDF4, HDF5, HDF-EOS 2 and 5 access)
• HE5View* (HDF5 - based browser)
• EOSView* (HDF4 - based browser)

* Dropped support for OS updates on July 2006
6

Software/Tools

Brief description of
TOOLKIT, HDF-EOS,
And
HDFView Plug-in

7

Toolkit
•

Science Data Production (SDP) Toolkit :
collection of tools used by
- Science software developers who produce
code to process instrument data.
- HDF-EOS users who need to
•
•
•
•

Access metadata in HDF-EOS files
Perform Time/Date conversion
Access Digital Elevation Model Data
Access Ancillary Data, such as Digital Chart
of the World database (DCW), Olson World
Ecosystem files, DEM data files, etc.
8

Toolkit (cont.)
•

Tools provided in SDP Toolkit:
- AA
-

CBP
CSC
CUC
DEM
EPH
GCT
IO
MEM
MET
PC
SMF
TD

(Ancillary Data Access)
(CELESTIAL Body Position)
(Coordinate System Conversion)
(Constant and Unit Conversions)
(Digital Elevation Model Access)
(Ephemeris Data Access)
(Geo Coordinate Transformation)
(Input Output (File I/O))
(Memory Management)
(Metadata Access)
(Process Control)
(Status Message File (Error/Status))
(Time and Date Conversion)
9

Toolkit (cont.)
•

MTD Toolkit:
– a short version of SDP Toolkit
– Tools provided:
• MET
(Metadata Access)
• TD
(Time and Date Conversion)
– Also supported in windows

10

HDF-EOS
• HDF-EOS:
– an extension to HDF which standardizes storage

of and access to common Earth Sciences data
structures

• By providing a single interface to data structures
common to earth science
• By providing a container for EOS inventory, archive
and product specific metadata
• By establishing a consistent relationship between
geolocation and science data.
11

HDF-EOS 2
• HDF4 - based, storage format for EOS standard
products.
• Used operationally by:
– MODIS, MISR, ASTER, Landsat, AIRS
– other EOS instruments

• Support for Grid/Point/Swath structures

12

HDF-EOS 5
• HDF5 - based, a complete rewrite of HDF-EOS2
with a different user interface.
• Used operationally by EOS Aura instrument teams
• Designed to ‘resemble’ HDF-EOS 2 to the
maximum extent possible.
• Supports same data structures (and more: Zonal
average)
• Added prefix ‘HE5_’ to HDF-EOS 2 functions
to distinguish two libraries
13

HDFView Plug-in
• HDFView:

A Java-based browser from The HDF Group
for HDF4 and HDF5 based data.

• HDF-EOS plug-in
– Extends HDFView to browse HDF-EOS data in both
versions.
– Is a plug-in module and does not change The HDFView
functionality
– Have identical look and feel when displaying HDF and
HDF-EOS objects
– Users can access a complete breakdown of HDF-EOS
objects. e.g. grid info, projection info, dims, datafields,
and attributes
14

HEG
• HEG
•
•
•
•
•
•

GeoTIFF conversion
Multi-band GeoTIFFs
Swath to Grid conversion
Spatial subsetting, Field Subsetting
Reprojection
Stitching, sub-sampling and more ….
15

Software Releases

SDP Toolkit/HDF-EOS:
• Last release – August 2010
– TOOLKIT 5.2.17
– HDF-EOS 2.17, HDF- EOS5.1.13
• with HDF4.2.5, HDF5-1.8.5, Szip2.1, zlib1.2.5

• Next release – (late 2010 – early 2011)
– TOOLKIT 5.2.18 (with new 500m DEM data)
– HDF-EOS 2.18, HDF-EOS5.1.14
• with new releases of HDF4 (2.6), 5 (1.8.6)

17

Software Releases
MTD Toolkit:
• Last release – Dec 2006
– Version 5.2.13
in-sync with SDP TOOLKIT 5.2.13
• with HDF4.2r0, HDF5-1.6.3, Szip1.2
•

• Next release – Oct 2010
– Version 5.2.17
• in-sync with SDP TOOLKIT 5.2.17
• with new releases of HDF4 (2.5), 5 (1.8.5)
18

Software Releases
HEG (HDF-EOS2 to GeoTIFF Converter):
• Last release – Dec 2009
– Version 2.10
– Patch release Jan 2010

• Next release – Dec 2010
– Version 2.11

19

Software Releases
HDF-EOS Plug-in for HDFView:
• Last release – Apr 2010
– For HDFView Version 2.6

• Next release – Early 2011
– For HDFView Version 2.7 to be released
– Version 2.6 plug-in works with HDFView 2.7
20

Software Releases
HECONVERT (HDF-EOS2 to HDF-EOS5
•
•

Converter):

Last releases
– 2001, 2004 No Versioning , Oct. 2008 Version 2.0
Next release
– Nov. 2010 Version 3.0
» Fix for some missing attributes in the converted file

DumpMetadata (ASCII Dump of HDF-EOS metadata)
•

Last release
– Dec. 2008 Version 1.0
21

Recent Major Development/Fixes
TOOLKIT:
– XML metadata writing capability for inventory metadata in
SDP toolkit (5.2.16 release).
Added XML style sheet to new TOOLKIT distribution for
metadata display with Internet Explorer and other browsers
– Fix for writing metadata into an HDF file when multiple HDF
files are open
– libiconv delivery in tar files for Mac and Cygwin
– Fix for setting correct endianness flag in linux for eph tools
(Fix is for MOPITT eph files reading)
22

HDF-EOS2:
– Added Dimension scales for Swath and Grid

Fix for:
– Subsetting with GDdefboxregion results in an
extra row and/or column in linux (round-off
problem)
– Inconsistency in dimensions between real dims
and the values in the structure metadata
– Problem on setting other compressions for swath
and grid when disabling szip compression
23

– Segfault in creating swath when HDF files
opened in read-only mode
– Segfault in EHinquire function because of a
VGroup with class "CDF0.0”
– Segfault in GDapi (sincos redefinition problem)
– Segfault in sample test grid subsetter
– compinfo functions must return -1 only for
errors (not for COMP_NONE)
– GDrs2ll should fail for any projection other than
BCEA
24

HDF-EOS5:
– Added Dimension scales to Swath, Grid, and
Zonal Average fields
– Fix for problem in setting fillvalue;
Fillvalue is only for the field defined right after
setting fillvalue

25

–

Support for all features in HDFView2.6

–

Correction for displayed lat/lon in a swath around
the international date line

–

Bug fix for HDF5 fillvalue and chunking info display

–

Bug fixes for saving HDF-EOS5 files after
modification
26

HEG:
Added Product Support:
– AMSR-E Daily 25 Km Sea Ice AE_SI25
– AMSR-E Daily land AE_Land3
– MOD29E1D: MODIS/Terra Sea Ice Extent & IST
Daily L3 Global 4km EASE-Grid Day, Version 4
– Tested many new products and updated the
list of Hegable MODIS, ASTER, AIRS, MISR, and
AMSR-E products in HEG Users Guide and HEG
web site
27

Enhancement:
– Added a new projection, Cylindrical Equa Area
(CEA), to support AMSR-E Daily Land product.
– Pixel resampling support for gridded data with NoReprojection
– Outputting all fields in a file to a single HDF-EOS
output when HDF-EOS output is requested.
– Setting of projection parameters for multiband
geotiff only once in the GUI
28

Bug fixes:
– Pixel size unit correction for consistency between
conversion and stitch GUIs.
– Problem resolution with several products MOD43C3,
MOD10CM, MOD29P1N, AE_RnGd, AE_Rain
– Correcting sphere radius written to outputs for
MODIS SIN grids
– Resolving problems
• with band selection in stitch GUI
• with occasional core dump (runs that take long
time to complete) in Datapool HEGserver
29

Future Plans
TOOLKIT/HDF-EOS:
• XML Metadata reading in TOOLKIT
• Support for new versions of HDF4 - HDF5
• Update of DEM Tools for land-sea mask and new 500m
DEM data
• Fix for getting correct compression info for a field in
HDF-EOS5, when GDcompinfo/SWcompinfo fails to get
the info from Structure metadata
• Support (or improvements) for 64-bit platforms
• Fix for Wrong values for some indices returned from
SWindexinfo() in both hdfeos2 and hdfeos5
30

Future Plans
• Store info in structural metadata somewhere
else
• Auto Configuration for TOOLKIT
• RPMs for HDF-EOS and Toolkit
• Validation tool for HDF-EOS5 files
• Fix for user/DAAC reported bugs

31

Future Plans (cont.)
• More functionalities :
–
–
–
–

Cut/paste objects, modify/delete attributes.
Create new grid/swath
Support Zonal Average object for HDF-EOS5
Add World Coast Line map to HDFView Plug-in

HEG:
– Performance improvement for Lambert Azimuthal
Projection conversion
– Support HEG in MAC Intel
– Resolve reported bugs or requests from DAACs/Users
32

Availability
• Access to libraries, applications, Documentation:
TOOLKIT:

http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html

HDFView:

http://newsroom.gsfc.nasa.gov/sdptoolkit/HDFView/
HDFView_hdfeos_plugin.html

HEG:

http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html
HDF-EOS:

http://www.hdfeos.org/

( new look, sample codes + HDF-EOS Forum )

• Email

– Abe_Taaheri@raytheon.com
– Landover_PGSTLKIT@raytheon.com

33

Status of HDF-EOS, Related Software and Tools

More Related Content

What's hot (20)

Similar to Status of HDF-EOS, Related Software and Tools (20)

More from The HDF-EOS Tools and Information Center (20)

Recently uploaded (20)

Status of HDF-EOS, Related Software and Tools