Dimension Scales in HDF-EOS2 and HDF-EOS5

Dimension Scales
in
HDF-EOS2 & HDF-EOS5
Abe Taaheri, Raytheon IIS
HDF & HDF-EOS Workshop XIV
Champaign, IL
Sep. 29, 2010

Page 1

•
•
•

What is a Dimension Scale ?
Dimension Scales & Metadata
Dimension Scales APIs
–
–

•

Code example
–
–

•

he2
he5

Writing (he2, he5)
Reading (he2, he5)

Sample he2 and he5 files with Dimension Scales
Page 2

What is a Dimension Scale?
It is a sequence of numbers
placed along a dimension to
demarcate intervals along it

• HDF4
- It is an array with size and name similar to
its assigned dimension

- Stored using a structure similar to the SDS
array

- One scale is assigned per dimension
Page 3

* What is a Dimension Scale?

• HDF5
– A HDF5 dataset.
– With additional metadata that identifies
the dataset as a Dimension Scale

– Typically Dimension Scales are logically
associated with the dimensions of HDF5
Datasets

– The meaning of the association is left to
applications.
* Pedro Vicente talk, HDF/ HDF-EOS Workshop IX
Page 4

Example: 3D dataset
3 Dimension Scales Datasets

5
7

Dataset: 3D Array with
5 x 7 x 10
dimensions

10
Page 5

More on Dimension Scale
in HDF5
• A dimension scale is not required to be a 1-D
array, or to have a specific datatype

• A dataset dimension can have more than 1
associated dimension scale

• A Dimension Scale can be shared by two or
more Dataset dimensions

Page 6

Example: 3D dataset in HDF5
Several Dimension Scales
Datasets

Dataset: 3D Array with
5 x 7 x 10
dimensions

Page 7

HDF5 Dimension Scale Metadata
• When the Dimension Scale is associated with
a dimension of a Dataset, the association is
represented by attributes of the two datasets.

• The following dataset attributes are used to
describe dimension scale datasets:

– Attribute named “CLASS” with the value
“DIMENSION_SCALE”
– Optional attribute named “NAME”
– Attribute references to any associated
Dataset
Page 8

• HDF4
– Since Version 4.0 ?
– SDsetdimscale()
– SDgetdimscale()
– SDsetdimstrs(): label, unit, format
– SDgetdimstrs()
• HDF5
– Since Version 1.8
– H5DSset_scale()
– H5DSattach_scale(), H5DSdetach_scale()
– H5DSset_label(), H5DSget_label()
– A few more APIs
Page 9

HDF Augmentation Tool

Augmentation Tool adds
HDF5 Dimension Scales
inside the file so that
netCDF-4 APIs can access
the data

Page 10

•

HDF-EOS

–

Added a few routines to HDF-EOS2 to
create dimension scales like HDF4
 Version 2.17

–

Added a few routines to HDF-EOS5 to
create dimension scales like those added
by the HDF Augmentation Tool


Version 1.13
Page 11

• Name:

SWsetdimscale

• Signature:
– intn SWsetdimscale( int32 swathID,
char *fieldname, char *dimname, int32 dimsize,
int32 numbertype, VOIDP data)

• Purpose:
– Sets dimension scale for a given dimension

Page 13

• Name:

SWgetdimscale

• Signature:
– intn SWgetdimscale( int32 swathID,
char *fieldname, char *dimname,
int32 *dimsize, int32 *numbertype,
VOIDP data)

• Purpose:
– Gets dimension scale for a given dimension
Page 14

• Name:

SWsetdimstrs

• Signature:
– intn SWsetdimstrs( int32 swathID,
char *fieldname, char *dimname, char *label,
char *unit, char *format)

• Purpose:
– Sets the label, unit, and format strings for a
given dimension
Page 15

• Name:

SWgetdimstrs

• Signature:
– intn SWgetdimstrs( int32 swathID, char
*fieldname, char *dimname, char *label,
char *unit, char *format, intn len)

• Purpose:
– Gets the label, unit, and format strings for a
given dimension
Page 16

• Similar APIs for Dimension Scales in
– Grid Fields

Page 17

• Name:

HE5_SWsetdimscale

• Signature:
– herr_t HE5_SWsetdimscale( hid_t swathID,
const hsize_t dimsize, hid_t numbertype,
void * data)

• Purpose:
– Sets dimension scale for a field dimension
within the swath
Page 19

• Name:

HE5_SWgetdimscale

• Signature:
– long HE5_SWgetdimscale( hid_t swathID,
hsize_t *dimsize, hid_t *numbertype,
void * data)

• Purpose:
– Gets dimension scale for a field dimension
within the swath

Page 20

• Name:

HE5_SWwritedscaleattr

• Signature:
– herr_t HE5_SWwritedscaleattr( hid_t swathID,
const char *dimname, const char *attrname,
hid_t ntype, hsize_t count[], void *datbuf)

• Purpose:
– Writes/Updates a dimension scale attribute in a
specific swath
Page 21

• Name:

HE5_SWreaddscaleattr

• Signature:
– herr_t HE5_SWreaddscaleattr( hid_t swathID,
void *datbuf)

• Purpose:
– Reads a dimension scale attribute from a
specific dimension
Page 22

• Name:

HE5_SWinqdscaleattrs

• Signature:
– Long HE5_SWinqdscaleattrs( hid_t swathID,
const char *dimname, char *attrnames,
long *strbufsize)

• Purpose:
– Retrieve information about the attributes
defined
Page 23

• Name:

HE5_SWdscaleattrinfo

• Signature:
– herr_t HE5_SWdscaleattrinfo( hid_t swathID,
hid_t *ntype, hsize_t *count)
count : Number of attribute elements

• Purpose:
– Returns information about attribute(s) in a
specific dimension scale
Page 24

• Similar APIs for Dimension Scales in
– Grid

– Zonal Average

Page 25

Writing Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
int main()
{
intn
status, i;
int32
swfid, SWid;
int32
nbands, nGeoTrack;
int32
bands[15]={3,6,9,12,15,18,23,26,29,32,33,34,36,37,39};
char
label[16];
char
unit[16];
char
format[16];
float
dataGeoTrack[20];
swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR );
SWid = SWattach ( swfid, "Swath1“ );
Page 27

nGeoTrack = 20;
for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; }
status = SWsetdimscale ( SWid, "Temperature", "GeoTrack",
nGeoTrack, DFNT_FLOAT32, dataGeoTrack );
nbands = 15;
status = SWsetdimscale ( SWid, "Spectra", "Bands",
nbands, DFNT_INT32, bands );
strcpy (label, "Bands");
strcpy (unit, "none");
strcpy (format, "I2");
status = SWsetdimstrs ( SWid, "Spectra", "Bands", label, unit, format );
SWdetach ( SWid );
SWclose ( swfid );
return 0;
}
Page 28

Writing Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main()
{
herr_t
status = FAIL;
hid_t
swfid = FAIL;
hid_t
SWid1 = FAIL;
int
data[15]={1,2,3,6,9,12,15,18,23,26,29,32, 33,34,35};
char
label[16], unit[16], format[16];
int
nbands;
hsize_t
count[3];
swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR );
SWid1 = HE5_SWattach ( swfid, "Swath1“ );

Page 29

nbands = 15;
status = HE5_SWsetdimscale ( SWid1, "Spectra", "Bands", nbands,
H5T_NATIVE_INT, (void *)data );
strcpy(label, "Bands Dim");
strcpy(unit, "None");
strcpy(format, "I2");
count[0]= 12;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "label",
H5T_NATIVE_CHAR, count, label );
count[0]= 6;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "unit",
H5T_NATIVE_CHAR, count, unit );
count[0]= 4;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "format",
H5T_NATIVE_CHAR, count, format );
Page 30

int datbuf_i1[1] = {-999};
count[0]= 1;
status = HE5_SWwritedscaleattr ( SWid1, "Bands", "MissingValue",
H5T_NATIVE_INT, count, datbuf_i1 );
int datbuf_i2[3] = {-999, 0, 999};
count[0]= 3;
status = HE5_SWwritedscaleattr(SWid1, "Bands", "IntValues",
H5T_NATIVE_INT, count, datbuf_i2 );
HE5_SWdetach ( SWid1 );
HE5_SWclose ( swfid );
return 0;
}

Page 31

Reading Code Example (HDF-EOS2)
#include "hdf.h"
#include "HdfEosDef.h"
Int main()
{
intn
status, i;
int32
swfid, SWid;
int32
*datbuf_int32;
int32
nbands, data_type, dimsize, dimsize1;
intn
len, buffsize;
char
label[16], unit[16], format[16];
swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR );
SWid = SWattach ( swfid, "Swath1“ );

/* get Bands */
dimsize = SWdiminfo ( SWid, "Bands“ );
buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1,
&data_type, NULL );
Page 32

datbuf_int32 = (int32 *) malloc(buffsize);
buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1,
&data_type, (void *)datbuf_int32 );
for (i=0; i<dimsize; i++)
{ printf("i = %d datbuf_int32 = %dn", i, datbuf_int32[i]); }
/* get str attributes of Bands*/
len = 15;
status = SWgetdimstrs ( SWid, "Spectra", "Bands",
label, unit, format, len );
printf(" ttlabel = %s unit = %s format = %s nn", label, unit, format);
free(datbuf_int32);
datbuf_int32 = NULL;
SWdetach ( SWid );
SWclose ( swfid );
return 0;
}
Page 33

Reading Code Example (HDF-EOS5)
#include <HE5_HdfEosDef.h>
int main()
{
herr_t
status = FAIL;
hid_t
swfid = FAIL, SWid = FAIL, numtype;
int
i,j, bufsize;
int
*databuff;
long
nattr, strbufsize;
char
*attrlist, *attrname = (char *)NULL;
size_t
fldnmlen[HE5_HDFE_NAMBUFSIZE];
char
*fldnm[HE5_HDFE_NAMBUFSIZE];
hid_t
*ntype;
hsize_t
count = 0;
void
*attr;
int
*attr_int;
float
*attr_flt;
char
*attr_char;
hsize_t
dimsize;
swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR );
SWid = HE5_SWattach ( swfid, "Swath1“ );

Page 34

bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize,
&numtype, NULL );
databuff = (int *) malloc(bufsize );
bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize,
&numtype, (void *)databuff );
for( i = 0; i < dimsize; i++)
{ printf ("DscaleVal[%d] = %dn",i,databuff[i] ); }
nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", NULL, &strbufsize );
attrlist = (char *) calloc(strbufsize + 1, sizeof(char));
nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", attrlist, &strbufsize );
nattr = HE5_EHparsestr(attrlist, ',', fldnm, fldnmlen);

Page 35

for( i = 0; i < nattr; i++)
{
attrname = (char *) calloc(fldnmlen[i] + 1, sizeof(char) );
memmove ( attrname,fldnm[i],fldnmlen[i] );
ntype = (hid_t *) calloc(1, sizeof(hid_t));
if ( strcmp ( attrname, "REFERENCE_LIST") == 0 ) continue;
status = HE5_SWdscaleattrinfo ( SWid,"Bands", attrname,
ntype, &count );
if( (int)*ntype == 0) {
attr_int = (int *) malloc(count*sizeof(int));
attr = (void *) attr_int; }
attr_flt = (float *) malloc(count*sizeof(float));
attr = (void *) attr_flt; }
attr_char = (char *) malloc((count+1)*sizeof(char));
attr = (void *) attr_char; }
status = HE5_SWreaddscaleattr ( SWid,"Bands", attrname, attr );
printf("tAttribute Name:
%s n",attrname);
printf("tNumber of attr elem: %lu n", (unsigned long)count);
printf("tData type of attribute:
%d n", (int)*ntype);
Page 36

attr_int = (int *) attr;
for (j = 0; j < count; j++) { printf("tAttrvalue[%d]:
}
attr_flt = (float *) attr;
for (j = 0; j < count; j++) { printf("tAttrvalue[%d]:
}

}

}

attr_char = (char *) attr;
attr_char[count]='0';
printf("tAttrvalue:
free ( attr_char );
}

%d n", j,attr_int[j]); }

%f n", j,attr_flt[j]); }

%s n", attr_char);

HE5_SWdetach ( SWid );
HE5_SWclose ( swfid );
return 0;

Page 37

Test File content (he2)
HDFView 2.4:
uses plug-in with
hdf-eos5-1.11,
Hdfeos2.15,
HDF4.2r3,
HDF5-1.6.7

Page 38

HDFView 2.4:
uses plug-in with
hdf-eos5-1.11,
Hdfeos2.15,
HDF4.2r3,
HDF5-1.6.7

Page 39

HDFView 2.6:
Without plug-in
HDF4.2r4,
HDF5-1.8.3

Page 40

HDFView 2.6:
uses plug-in with
hdf-eos5-1.12,
Hdfeos2.16,
HDF4.2r4,
HDF5-1.8.3

Page 41

File content (HIRDLS he5)
HIRDLS he5
Modified with
HDF
Augmentation
Tool

Page 42

Suggestions?

– Do we need new APIs ?
– Improvements for existing APIs ?
– What else?

Page 43

Dimension Scales in HDF-EOS2 and HDF-EOS5

More Related Content

Viewers also liked (7)

Similar to Dimension Scales in HDF-EOS2 and HDF-EOS5 (20)

More from The HDF-EOS Tools and Information Center (20)

Recently uploaded (20)

Dimension Scales in HDF-EOS2 and HDF-EOS5