AppScale + Neptune @ HPCDB
- 1. Enabling Automated HPC / Database Deployment via the AppScale Hybrid Cloud Platform Chris Bunch Chandra Krintz HPCDB 2011 @ Seattle, WA November 18, 2011
- 2. Cloud Computing • Three tiers of abstraction: • Infrastructure: Scalable hardware • Platform: Scalable APIs • Software: Scalable application
- 3. HPC in the Cloud • Easy access to vast resources • Hard to automatically configure and deploy libraries • Requires in-depth knowledge of each technology involved • Hard to get high performance on opaque cloud • Wide range of APIs for similar services (e.g., compute, storage)
- 4. Solving these Problems • Need a solution that is automated • Minimize the amount of domain-specific knowledge a scientist has to learn • Need to automate configuration and deployment • Hard since the number of nodes used can be dynamic in the cloud
- 5. Cloud Computing • Infrastructure: Completely self-service • Platform: Automated access to certain APIs • Software: Optimized for one piece of software
- 6. APIs to Support • To support scientists, we need to support: • Web services • HPC codes • Databases • And it must be simple
- 7. Introducing AppScale • An open source implementation of the Google App Engine APIs • Can run over Amazon EC2 or Eucalyptus • Configures and deploys automatically • Scientist only needs to specify the number of nodes to run over
- 8. One Button Deployment •appscale-run-instances • --max 32 • --infrastructure ec2
- 9. AppScale + Neptune • A domain-specific language for running HPC codes • Supports codes written in: • MPI, UPC, X10, and others • Configures and deploys automatically • Scientist only needs to specify the number of nodes to run over
- 10. One Command Deployment •neptune :type => :mpi, • :code => “/code/ring”, • :nodes_to_use => 32, • :output => “/output/ring”
- 11. Database Support • Support for a dozen datastores, including: • Neptune can also store to Amazon S3 • Or anything that uses the same APIs • Completely automated
- 12. Hybrid Clouds • The utilization of more than one cloud • Fault tolerance: Run an app in multiple clouds • Portability: Migrate an app from one cloud to another • Adds another layer of complexity
- 13. Open Hybrid Clouds • AppScale and Neptune both support automated hybrid cloud deployment • Just provide your credentials • Would like to expand both with capabilities for special use cases • Quantify the cost of each use case
- 14. Government Use Case • Compute anywhere, store only in a private, trusted cloud Cloud 1 - EC2 East Coast Cloud 2 - Private Eucalyptus Deployment App Server & DB Node Master App DB Node Server
- 15. One DB in Each Cloud • Compute anywhere, store in the same cloud Cloud 1 - EC2 East Coast Cloud 2 - Private Eucalyptus Deployment App App Server & Server Master DB Node DB Node
- 16. Wrapping it Up • Thanks to the AppScale team, especially co- lead Navraj Chohan and advisor Chandra Krintz • AppScale 1.5 released in August • Neptune 0.1.2 released last week • Visit us at http://appscale.cs.ucsb.edu • E-mail: cgb@cs.ucsb.edu