HDF Tools Tutorial
Download as PPTX, PDF1 like605 views
The document discusses HDF command line tools that can be used to view, modify, and manipulate HDF5 files. It provides examples of using tools like h5dump to view file structure and dataset information, h5repack to optimize file layout and compression, h5diff to compare files and datasets, and h5copy to copy objects between files. The tutorial was presented at the 15th HDF and HDF-EOS workshop from April 17-19, 2012.
HDF Tools Tutorial
- 1. The HDF Group HDF Tools Tutorial Peter Cao The HDF Group The 15th HDF and HDF-EOS Workshop April 17-19, 2012 Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 1 www.hdfgroup.org
- 2. Tools Tutorial HDF5 Command-line tools HDFView GUI Built in Java Text Built in C Separate release Part of the lib release HDF4, HDF5, NetCDF, etc. HDF5 only Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 2 www.hdfgroup.org
- 3. HDFView Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 3 www.hdfgroup.org
- 4. HDFView The best way to get help a) b) c) d) e) Apr. 17-19, 2012 Call Peter Email to Barbara Stop by Mike’s office All of above None of above HDF/HDF-EOS Workshop XV 4 www.hdfgroup.org
- 5. HDFView HDFView Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 5 www.hdfgroup.org
- 6. View File Content File structure File Content Metadata and log info. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 6 www.hdfgroup.org
- 7. Add/delete, Copy/paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 7 www.hdfgroup.org
- 8. Add/delete, Copy/paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 8 www.hdfgroup.org
- 9. Add/Delete, Copy/Paste Objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 9 www.hdfgroup.org
- 10. Modify, Copy/Paste Data Values Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 10 www.hdfgroup.org
- 11. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 11 www.hdfgroup.org
- 12. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 12 www.hdfgroup.org
- 13. Modify, Add, Delete Attributes Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 13 www.hdfgroup.org
- 14. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 14 www.hdfgroup.org
- 15. HDFView • View file content • File structure/metadata • Data values or Images • Change file content • Add/delete objects/attributes • Modify values of datasets or attribute • Import/export • Image files • Text files Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 15 www.hdfgroup.org
- 16. The HDF Group HDF5 Command-line tools Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 16 www.hdfgroup.org
- 17. HDF5 Command-line tools • View or check file content h5dump, h5diff, h5ls, h5check, etc. • Change file content or create a new file h5repack, h5import, h5jam, h5copy, etc. • Convert between HDF5 and another format h4toh5, h5toh4, gif2h5, h52gif, etc. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 17 www.hdfgroup.org
- 18. HDF5 Command-line tools • H5dump • H5repack • H5diff • h5copy Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 18 www.hdfgroup.org
- 19. HDF5 Command-line tools “-h” or “-help” h5dump -h HDF5 tools Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 19 www.hdfgroup.org
- 20. h5dump h5dump -H ami.h5 HDF5 "ami.h5" { GROUP "/" { GROUP "EnSight_model" { ATTRIBUTE "description_1" { DATATYPE H5T_STRING { STRSIZE 256; STRPAD H5T_STR_NULLPAD; CSET H5T_CSET_ASCII; CTYPE H5T_C_S1; } DATASPACE SIMPLE { ( 1 ) / ( 1 ) } } …. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 20 www.hdfgroup.org
- 21. h5dump: Dataset Information H5dump –H -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } } } Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 21 www.hdfgroup.org
- 22. h5dump: Dataset Content h5dump -d /EnSight_model/geometry/parts/part_2/quad4 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } DATA { (0,0): 5, 6, 431, 430, (1,0): 6, 7, 432, 431, (2,0): 7, 8, 433, 432, (3,0): 8, 9, 434, 433, (4,0): 9, 10, 435, 434, (5,0): 10, 11, 436, 435, … Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 22 www.hdfgroup.org
- 23. h5dump: Data Subset h5dump -d /EnSight_model/geometry/parts/part_2/quad4 -s 10,0 -c 10,2 ami.h5 HDF5 "ami.h5" { DATASET "/EnSight_model/geometry/parts/part_2/quad4" { DATATYPE H5T_STD_I32LE DATASPACE SIMPLE { ( 768, 4 ) / ( 768, 4 ) } SUBSET { START ( 10, 0 ); STRIDE ( 1, 1 ); COUNT ( 10, 2 ); BLOCK ( 1, 1 ); DATA { (10,0): 15, 16, (11,0): 16, 17, (12,0): 17, 18, (13,0): 18, 19, (14,0): 19, 20, } Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 23 www.hdfgroup.org
- 24. h5repack Copies a whole file to a new file Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 24 www.hdfgroup.org
- 25. H5repack: Remove Junk Space Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 25 www.hdfgroup.org
- 26. H5repack: change the storage layout 2x2 chunks h5repack -l CHUNK=16x16 tools_bad_layout.h5 tmp.h5 162x16 chunks Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 26 www.hdfgroup.org
- 27. H5repack:Apply compression filter 10 GB h5repack -f GZIP=6 tmp.h5 tmp2.h5 1 GB Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 27 www.hdfgroup.org
- 28. h5diff Show differences between two files or two objects Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 28 www.hdfgroup.org
- 29. H5diff: Two files h5diff tools_diff1.h5 tools_diff2.h5 Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 29 www.hdfgroup.org
- 30. H5diff: Two datasets h5diff tools_diff1.h5 tools_diff2.h5 /dset h5diff tools_diff1.h5 tools_diff2.h5 /g1/uint8 /dset Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 30 www.hdfgroup.org
- 31. h5copy Copy one or more objects to another file h5copy -i tools_input.h5 -o tmp.h5 -s /g1/g2/f -d /float -h Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 31 www.hdfgroup.org
- 32. Acknowledgements This work was supported by Subcontract number 114820 under Raytheon Contract number NNG10HP02C, funded by the National Aeronautics and Space Administration (NASA) and by cooperative agreement number NNX08AO77A from the NASA. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of Raytheon or the National Aeronautics and Space Administration. Apr. 17-19, 2012 HDF/HDF-EOS Workshop XV 32 www.hdfgroup.org
Editor's Notes
- #3: Hdf4, netcdf, etc.
- #11: Copy and paste
- #12: Add general view
- #13: Add general view
- #14: Add general view
- #18: Introduce 4 tools, mention Gerd’spowershell, Dan’s python, and other tools
- #19: Find these are the most useful/used tools
- #21: Does not show content of a dataset
- #25: A little too many details.
- #32: One or more objects to another file
- #33: Mention the updates and other things