Scientific Computing and Visualization using HDF
Download as PPTX, PDF1 like768 views
Aashish Chaudhary gave a presentation on Kitware's work with scientific computing and visualization using HDF. HDF is a widely used data format at Kitware for domains like climate modeling, geospatial visualization, and information visualization. Kitware is looking to improve HDF support for cloud and web environments to enable streaming analytics and web-based data analysis. The company also aims to further open source collaboration and scientific computing.
Scientific Computing and Visualization using HDF
- 1. Aashish Chaudhary aashish.chaudhary@kitware.com Technical Leader with Patrick O’Leary, Petr Votava (NASA), Dr. Rama Nemani (NASA), Chris Harris, Chris Kotfila, Doruk Aztek, Andrew Michaelis (NASA) Scientific Computing and Visualization using HDF July 19th 2016 Durham, NC
- 2. What We Do at Kitware? 0pen Source and Open Data is strongly encouraged and practised at Kitware
- 3. It started with VTK
- 4. Parallel Processing and Rendering - Paraview
- 5. Computer Vision Images, Video, Point Clouds Recognition by Function Content- based Retrieval Event & Activity Recognition Anomaly Detection 3D Extraction and Compression Detection & Tracking
- 6. Medical Computing Quantitative imaging Electronic health records Vascular analysis Surgical guidance And simulation Digital pathology Orthopedic analysis Longitudinal and population shape analysis Interactive medical applications and visualizations
- 8. HDF at Kitware Climate Community High Performance Computing Extensible Data Model and Format - Developed to exchange scientific data between HPC codes and tools - Heavy data is stored using HDF5 Network Common Data Form (NetCDF) - Most projects use NetCDF4 Medical Community Vision Community Leading-edge algorithms for registering and segmenting multidimensional data
- 9. ACME The Accelerated Climate Modeling for Energy (ACME) project is sponsored by the Earth System Modeling (ESM) program (Biological and Environmental Research) with eight national laboratories and six partner institutions to develop and apply the most complete, leading-edge climate and Earth system models to challenging and demanding climate-change research imperatives. Most commonly used data format - NetCDF4 Data streaming using OpenDAP Python Interface for most of the tools
- 10. OpenNEX NEX is a platform for scientific collaboration, knowledge sharing and research for the Earth science com Global Daily Downscaled Projections (NEX- GDDP, NetCDF4) MODIS-Land and Atmosphere (HDF)
- 16. Possible Improvements Streaming and Big Data analytics - Any useful ingestion of HDF data into cluster requires ETL pipeline - For some tools, computation cannot move close to the data, streaming support is necessary in such cases - Optimal read/write on cloud storage Web-Support - More tools and projects are moving to support web-enabled data analysis and visualization - Pure JS implementation if possible
- 17. Summary ● HDF is widely data format for scientific computing, climate/geospatial visualization, and in other domains at Kitware ● Recently we have started using HDF for information visualization ● We are looking forward to HDF usage on cloud and web-environment ● Kitware is always looking for strong open source collaborations and is committed to push open-source scientific computing to its next level
- 18. Information Aashish Chaudhary: aashish.chaudhary@kitware.com LinkedIn: www.linkedin.com/in/aachaudhary Kitware: http://www.kitware.com NASA-NEX: https://nex.nasa.gov/nex Kitware-AIST: https://github.com/OpenGeoscience/nex HPC Cloud : http://www.kitware.com/publications/item/view/1784 HPCloud Github: https://github.com/Kitware/HPCCloud