HDF Town Hall

Editor's Notes

• #6: Joe will take care of it.
• #8: HDF5 1.8.7 – 1.8.9 Fortran 2003 support, support for Fortran dimension scalesHDF4 releases in support of the H4 mapping projectSupport for Powerpc64 platform (big-endian)Java – addressed all ESDIS requestsBased on the latest available HDF4 and HDF5H4h5tools – updated to 18 APIs, no 18 features were added
• #10: Up to here elena fixes. Add QA person.
• #12: Joe moved this slide after maintenance plan.
• #13: Java HDF4.
• #14: Java HDF4.
• #15: Does this belong to Goal #5?
• #18: HDFView more than 10 years old. Since first implemented, new technologies and techniques have emerged that could help improve HDFView. We surveyed HDFView users last year. A lot of good ideas came out of that.We will not just look at Java, but other alternatives such as QT.This is an internally funded project led by Cao, Heber, Readey (Amazon).This group will:Review our vision for vis tools and how they are aligned with our mission. Review and company goals as regards support for vis tools. Identify needs and opportunities based on current and potential customers and their needs and desires.Review technologies and tools currently available that can help us develop new tools if needed, how the new tools compare with current HDF tools, and what they might offer in terms of improvements.Develop of a set of guiding principles for going forward.Recommend activities, perhaps leading to a roadmap to long-term goals for the visualization tool(s).
• #22: The slide highlights recent accomplishments from the ExaHDF5 project funded by DOE/ASCR Exascale Scientific Data Management award.1) Parallel I/O with HDF5We ran a Trillion particle simulation on 120K cores on hopper. The code produced 30 TB of particle data per timestep, and produced over 350TB of data total- To the best of our knowledge, this is the first time that anyone has demonstrated writes to a single, shared 30 TB HDF5 fileWe hit peak I/O rates on hopper (~35GB/s) during the run, we sustained an average ~23GB/s, which is a new record for parallel HDF5 performance2) FastBit based analysis- We developed a novel hybrid parallel version of FastBit to do the indexing/querying on the datasetThis was the first time that we used FastBit and FastQuery to index and query a dataset with Trillion entriesWe were able to index the dataset in 10 minutes and query the dataset in 3 seconds DOE researchers: Prabhat (PI), Suren Byna, Oliver Rubel and John Wu (LBNL)Scientific collaborators: HomaKarimabadi (UCSD), VadimRoytershteyn (UCSD) and Bill Daughton (LANL)Simulation code used in the study is VPIC, developed at LANL.Please address any questions to Prabhat (prabhat@lbl.gov).
• #23: 3) Scientific insightsThis is the first time that our science collaborators have been able to examine the trillion particle dataset. They had largely ignored the particle data, or looked at a coarse grained version earlier- Our collaborators discovered a power-law distribution in the energy spectrum of the particles. This is the first kinetic plasma physics to demonstrate a power-law distribution; our analysis capabilities directly facilitated this discovery Our collaborators had made a number of conjectures and hypothesis regarding the interplay between particles and the magnetic fields and multi-dimensional phase-space distribution of particles. Using these new tools, they were able to confirm these hypothesis quantitatively. More specifically the scientists found: - a preferential acceleration of particles in a direction parallel to the magnetic field - predominant distribution of energetic particles in the current sheet, suggesting that flux ropes can confine these particlesagyrotropic (asymmetric) distribution of particles near the magnetic reconnection event
• #28: Kent needs to update this.