Open stack @ iiit hyderabad
1 like1,337 views
This document summarizes the work done by OpenStack@IIIT-H, which uses OpenStack to run an Indian language search engine, conduct research in information extraction/retrieval/access and virtualization/cloud computing, and provide users with OpenStack, Hadoop, and other open-source software. Before OpenStack, provisioning resources was ad hoc and unmanaged, user management lacked controls, and storage had reliability and duplication issues. After implementing OpenStack, resources can be quickly and reliably provisioned on demand, usage is monitored and restricted with quotas, and storage uses Swift for reliability without data fragmentation. OpenStack also supports research and teaching over 350 students through hands-on projects.
Open stack @ iiit hyderabad
- 1. OpenStack@IIIT-H Dharmesh Kakadia (@dharmeshkakadia) Shashank Sahni (@shredder12)
- 2. What we do ● Run an Indian Languages Search Engine ● Research ○ Information Extraction ○ Information Retrieval ○ Information Access ○ Virtualization and Cloud ● Users of ○ OpenStack ○ Hadoop ○ and lot of other FOSS
- 4. Before OpenStack source: http://www.codeproject.com/KB/threads/hxgrid/image4.jpg
- 5. Problems ● Provisioning ○ Adhoc ○ Time consuming ○ Unmanaged ● User Management ○ No resource accounting ○ Access Control ○ Usage Restriction ● Storage ○ Data reliability ○ Duplication
- 6. More Problems... ● Cluster ○ Terrible Resource Utilization ○ New deployment => Too much time ○ Data Redundancy ○ Non-optimal deployments ● Academic ○ No cloud platform for experimentation ○ Large Scale sandboxed resource provisioning for students.
- 8. OpenStack(KVM) ● 7 Compute nodes (8GB, quad-core) ● 1 nova-volume(2 TB, Raid-1) Swift ● 3 storage nodes (2TB each) OpenStack(LXC) ● 16 Compute nodes (6GB, dual core)
- 9. Provisioning ● Pre-configured images to quickly get started. ● VM of any capacity available at any time( 2 a.m. Sunday morning) ● Snapshots
- 10. User Management ● Resource restrictions using Quota ● Project based collaboration and private resources ● Usage monitoring
- 11. Storage This wasn't easy. We experimented with ● nova-volume ● Swift(diablo) ● GlusterFS ● Swift(Folsom)(current)
- 12. Storage ● Hadoop compatible distributed storage ● Glance image store ● Desktop backup utility using CloudFuse ● Data reliability ● No more Data Fragmentation
- 13. OpenStack in Academia ● Research ○ Inter cloud migration ○ Inter cloud scheduling ○ Performance Evaluation ● Resource provisioning for course assignments and projects. ○ 3 courses ○ 350+ students ○ 20+ projects
- 14. HadoopStack ● Big Data processing on Demand ● Entire ecosystem for Big Data - Hadoop Family, Spark, Mahout, R ● Multi-Cloud - OpenStack and AWS.
- 15. HadoopStack
- 16. Conclusion ● Using OpenStack ● Working with and around OpenStack ● OpenStack is Awesome !!