Creating a Better Infrastructure to Manage Big Data
1 like653 views
The document discusses the development of infrastructure to optimize big data management within high-performance computing (HPC) environments. It highlights key concepts such as job-resource alignment, cloud computing integration, and the future of computing based on HPC principles. Additionally, references to visualizations and workflows essential for scientific computing and big data are noted.
Creating a Better Infrastructure to Manage Big Data
- 1. Creating a Better Infrastructure to Manage Big Data SC’13 Exhibitor Forum Software for HPC III Trev Harmon Principal Solution Strategist Adaptive Computing 1 © 2013 ADAPTIVE COMPUTING, INC. 21 Nov 2013 4:00-4:30pm
- 4. Exascale ??
- 5. Let’s Turn Back the Clock Denver, Colorado 5 2010 2013 © 2013 ADAPTIVE COMPUTING, INC.
- 6. Douglas Parkhill’s Vision 6 © 2013 ADAPTIVE COMPUTING, INC.
- 7. 7
- 8. not exactly Cloud ?
- 9. An Over-simplification Visualization HPC Technical Computing Computing Big Data Cloud 9 © 2013 ADAPTIVE COMPUTING, INC. Scientific Computing
- 10. Multiple Fields Multiple Users 10 © 2013 ADAPTIVE COMPUTING, INC.
- 11. Computing Workflows Visualization HPC 11 © 2013 ADAPTIVE COMPUTING, INC.
- 12. Computing Workflows 12 Scientific Computing Big Data Visualization Cloud HPC © 2013 ADAPTIVE COMPUTING, INC.
- 13. Really, we’re optimizing… Job Needs Available Resources 13 © 2013 ADAPTIVE COMPUTING, INC.
- 14. Topology and Company Topology-Awareness Data-Locality Awareness Hardware Preferences 14 © 2013 ADAPTIVE COMPUTING, INC.
- 15. Another Merger Cloud Scientific Computing Big Data HPC Cloud Visualization 15 © 2013 ADAPTIVE COMPUTING, INC. HPC
- 16. Defining HPC Cloud ds… en lly. rea dep It m, the l of ll al we 16 Cloud-based HPC ? Cloud-bursting ? VM-encapsulated Jobs ? © 2013 ADAPTIVE COMPUTING, INC.
- 17. The Future of Computing Is Founded on HPC Principles and Concepts. 17 © 2013 ADAPTIVE COMPUTING, INC.
- 18. Douglas Parkhill’s Vision 18 © 2013 ADAPTIVE COMPUTING, INC.
- 19. 19 © 2013 ADAPTIVE COMPUTING, INC.
- 20. 20 © 2013 ADAPTIVE COMPUTING, INC.
- 21. Attribution & Citation ▪ ▪ ▪ 21 Denver Photo: http://www.flickr.com/photos/dagpeak/ Douglas Parkhill Photo: http://www.spectralumni.ca/CommExpress_pdf/ cemay84e.pdf Topology Map: US Geological Survey, US Department of the Interior/USGS © 2013 ADAPTIVE COMPUTING, INC.