UnaCloud: Opportunistic Cloud Computing Infrastructure as a Service
1 like1,543 views
The document discusses 'Unacloud,' an opportunistic cloud computing infrastructure model that addresses the computational needs of e-science projects by utilizing underutilized resources from desktop computers and labs. It highlights the architecture, implementation, and testing of Unacloud, focusing on its ability to provide customizable, on-demand computing services to academic and research groups while ensuring usability, security, and integration with existing infrastructures. The findings suggest that Unacloud offers an economically viable solution for scalable and efficient cloud-based computing, facilitating faster project development cycles.
UnaCloud: Opportunistic Cloud Computing Infrastructure as a Service
- 1. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UnaCloud: Opportunistic Cloud Computing Infrastructure as a Service IaaS Model Type II Hypervisor UnaCloud Clients Operating System Desktop Computers UnaCloud Server IaaS user Grid user Administrator IaaS-Grid user Eduardo Rosales, Harold Castro, Mario Villamizar Department of Systems and Computing Engineering Universidad de los Andes Bogotá, Colombia
- 2. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM THE DESIRED SOLUTION UNACLOUD ARCHITECTURE UNACLOUD IMPLEMENTATION UNACLOUD TESTING AND RESULTS CONCLUSIONS AND FUTURE WORK
- 3. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM The development of e-Science projects requires large processing capabilities. These capabilities are regularly provided by dedicated cluster, grid and cloud computing infrastructures.
- 4. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM In the research environment of our university campus we find that each research group has its own dedicated clusters and some computer labs for students, so: Researchers use and have some experience with specific cluster/grid middlewares to distribute load among the nodes of dedicated clusters (OGE, Condor, etc.). Computer labs have many commodity desktops with different operating systems: Windows (mainly), MAC and Linux, which are idle most of the time. Most e-Science applications are run on Linux.
- 5. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM Researchers require large HPC/HTP computing during some peak periods (a project needs to be delivered, the call for paper will finish, etc.). Additionally, there are a lot of general or public campus computer labs used by students and these labs are also idle most of the time.
- 6. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM More than 2000 CPU cores
- 7. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE CONTEXT An alternative are Desktop Grids and Volunteer Computing Systems (DGVCS’s): Offer large scale computing infrastructures at low cost. Use inexpensive resources, most of them underutilized desktop computers. Interconnect thousands of computing resources available through Internet or Intranet environments. Are based on resources that are non-dedicated, distributed, highly heterogeneous, and part of independent administrative domains.
- 8. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM When a research group wants to use a DGVCS it regularly find that: They will need to recode, modify or adapt every application that is going to be executed on the DGVCS, for several research groups and tens of existing applications it is a complex process. The installation, configuration, maintaining and use of most DGVCSs require of people with some/advanced skills in applications and IT infrastructures. For using the idle processing capabilities of hundreds of commodity desktops, they will need to configure manually every desktop computer with the DGVCS software.
- 9. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE PROBLEM When a research group wants to use a DGVCS it regularly find that: Administrators of different computer labs do not want that external people modify the configurations of the physical machines. Most of the physical desktops (99%) machines available in computer labs have Windows operating systems. They would like to share easily with other research groups the idle capabilities available in computer labs using a shared model.
- 10. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION
- 11. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION
- 12. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Ubuntu with OGE
- 13. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Ubuntu with OGE VMs begin to process jobs of the bioinformatics cluster
- 14. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION
- 15. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Debian with PBS
- 16. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Ubuntu Debian with OGE with PBS Both clusters are being executed on the same physical/shared commodity infrastructure.
- 17. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION UNACLOUD Research groups can deploy on- demand Computing Services, sharing the same commodity infrastructure. This is achieved using an Opportunistic Infrastructure as a Service Strategy.
- 18. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) SOLUTION REQUIREMENTS Our designs have specific requirements: On demand computing services customization and infrastructure assignment to meet specific requirements related to e-Science applications (native executing environments). High usability user interfaces based on a self-service model to intuitive creation, deployment and management of large computing infrastructures to support e- Science projects. Strategies to allow the use of idle computing resources in a non-intrusive manner to provide large scale computing capabilities at low cost. Measured service to record and report the infrastructure user-level consumption. Delegated administration of the underlying infrastructure complexity used to provide customized computing services to researchers, professors and students. Solutions based on ubiquitous, interoperability, loose coupling, extensibility and information security principles.
- 19. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) DGVCS’s / CLOUD COMPUTING STRATEGY CLOUD COMPUTING DGVCS’s -Broad network access -Opportunistic strategies -Measured service -High -Distributed systems usability -On demand services -Highly heterogeneous customization -Interoperability -Self-service UnaCloud systems -Resource optimization and loose coupling -Scalability -Non-dedicated -Low cost infrastructures -Virtualization -Multi tenancy -High scalability -Extensibility -Best effort -Delegated approach -Non-intrusive -Security administration design In this work we analyze the prospect and performance of using an opportunistic underlying infrastructure to support a Cloud Computing IaaS model. This is the main motivation and contribution of this research work.
- 20. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD ARCHITECTURE Virtualization technologies allow UnaCloud IaaS Model UnaCloud to access the following Type II Hypervisor advantages: resource optimization, Operating System execution environments isolation, on demand deployment and high portability to build a cloud computing CPU Memory Storage Network IaaS model over type II hypervisor HARDWARE services. VMs are executed in Virtual Machine background and as low IaaS (background and low priority Model process) priority processes, allowing that UnaCloud can provide Type II Hypervisor Virtualization Technology the following advantages: Main Operating Main Operating System System virtual and physical environments isolation and Opportunistic non-intrusive idle resources CPU Memory Storage Network Infrastructure use.
- 21. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD ARCHITECTURE END USER ROLE IaaS Model Non-located IaaS Type II Hypervisor IaaS customization and/or UnaCloud Clients deployment Operating System Located Grid Desktop Grid customization and/or Computers deployment UnaCloud Server IaaS-Grid Any of the above roles Administrative and Administrator monitoring exclusive IaaS user services Grid user Administrator IaaS-Grid user UnaCloud architecture is based on the integration of a Web information system with an underlying computing and communication infrastructure A Web portal (UnaCloud Server) articulated (through UnaCloud Clients) with a non- dedicated infrastructure provides an opportunistic cloud computing IaaS model
- 22. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION Data Center The UnaCloud server was developed using J2EE and Administrators MySQL, and it was installed on a dedicated machine. Remote Access Mechanism UnaCloud Grid IaaS users Server Masters The UnaCloud client is a J2SE GigE application. The UnaCloud client was Grid users deployed in 3 computer labs using 105 desktops computers. Alan Turing Waira I Waira II Desktops have an Intel Core 2 0 IaaS Cluster Duo (1.86GHz) processor, 4GB of virtual Slaves RAM and Windows XP. machines All nodes were interconnected Computer laboratories with UnaCloud Client using a GigE LAN.
- 23. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION Data Center A Web user interface was used, providing a self-service model. Administrators Deployments on different Remote Access Mechanism UnaCloud Grid Server Masters networks and administrative IaaS users GigE domains were performed. Secure mechanisms were used Grid users in message passing. Monitoring reports and Alan Turing Waira I Waira II statistics were generated for 0 IaaS Cluster UnaCloud users. virtual Slaves machines VMware workstation was used Computer laboratories with UnaCloud Client as type II hypervisor.
- 24. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION • Customizable Virtual Clusters (CVCs) IAAS through 5 settings: software, CUSTOMIZATION hardware, quantity, location (optional) and execution time. • On demand CVC deployment and IAAS provision of necessary data to DEPLOYMENT secure remote access. • VM operations such as: start, stop, IAAS ADMINISTRATION restart, change execution time and monitoring. IAAS • IaaS model traceability at user level TRACEABILITY with basic reports and statistics. PHYSICAL • Physical machine operations such INFRASTRUCTURE as: turn off, restart, logout and near- ADMINISTRATION real time monitoring.
- 25. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION • Customizable Virtual Clusters (CVCs) through 5 IAAS settings: software, hardware, quantity, location CUSTOMIZATION (optional) and execution time.
- 26. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION • Customizable Virtual Clusters (CVCs) through 5 IAAS settings: software, hardware, quantity, location CUSTOMIZATION (optional) and execution time.
- 27. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION IAAS • On demand CVC deployment and provision of DEPLOYMENT necessary data to secure remote access.
- 28. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION IAAS • VM operations such as: start, stop, restart, ADMINISTRATION change execution time and monitoring.
- 29. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION IAAS • IaaS model traceability at user level with basic TRACEABILITY reports and statistics.
- 30. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD IMPLEMENTATION PHYSICAL • Physical machine operations such as: turn off, INFRASTRUCTURE restart, logout and near-real time monitoring. ADMINISTRATION
- 31. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD TESTING AND RESULTS IAAS USERS VIRTUAL MACHINES 70 academic VMs DEPLOYMENT MODE Non-located deployment DEPLOYMENT TIME 13 seconds GRID USERS 70 bioinformatics 35 chemical VIRTUAL MACHINES VMs engineering VMs DEPLOYMENT MODE Located deployment DEPLOYMENT TIME 15 seconds 7 seconds The average time to load the guest operating systems is 4-5 minutes.
- 32. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD TESTING AND RESULTS CPU JOB TIME BY JOB EXECUTION APPLICATION NAME INFRASTRUCTURE USED NUMBER NUMBER (SEC) TIME (DAYS) BSGrid PC 2 150000 35 30,38 Model A Chemical Eng. CVC 70 150000 85 2,11 BSGrid PC 2 150000 63 54,69 Model B Chemical Eng. CVC 70 150000 111 2,75 PC 2 4200 11700 284,40 HMMER Biological Science CVC 140 4200 12900 4,50 Performance degradation perceived by owner users (students or administrative personal) is less than 3%. The maximum overload of grid jobs executed on UnaCloud virtual machines is of 17%. To avoid resource competition among virtual machines only one virtual machine is executed on each desktop.
- 33. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD OPPORTUNITIES AND LIMITATIONS REQUIRED FEATURES UNACLOUD USABILITY High usability Web user interfaces, which operation is almost intuitive, requiring basic IT knowledge SELF-SERVICE Unilaterally computing service provision BROAD NETWORK ACCESS Web portal available over Intranet and Internet ON DEMAND SERVICES CUSTOMIZATION On demand computing services customization, even to meet large scale computational requirements HARDWARE MULTI TENANCY Opportunistic use of idle computing resources VIRTUALIZATION On demand VM deployment through virtualization SCALABILITY Horizontal scaling model based on private clouds INTEROPERABILITY AND LOOSE COUPLING Web and service oriented architecture EXTENSIBILITY Use of open source tools, broadly diffused DELEGATED ADMINISTRATION Services to support common administration tasks SECURITY Authentication, authorization, confidentiality and non- repudiation mechanisms to secure the deployments MEASURED SERVICE Infrastructure traceability at user level with basic reports QOS AND SLAs Best effort approach due to the infrastructure availability x
- 34. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) CONCLUSIONS UnaCloud validates the convergence of cloud computing and DGVCS’s, offering promising opportunities to meet customized computational requirements thought the use of an open source, low cost, extensible, interoperable, efficient, scalable, secure and opportunistic IaaS model. UnaCloud provides a multipurpose cloud computing experimental platform to deploy Customizable Virtual Clusters that support new specific computational requirements of academic and research projects. UnaCloud represents an economically attractive solution for building and deploying large scale computing infrastructures. UnaCloud cloud computing features are promising to reduce the development cycle and the generation of results time of projects depending on the agile and flexible provisioning and sharing of low cost computing resources.
- 35. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) FUTURE WORK RELATED TO CLOUD COMPUTING RELATED TO DGVCS’s QUALITY OF SERVICE MODELS: PaaS and SaaS Statistic QoS approach SERVICE (QoS): DEPLOYMENT Public, community and TESTS AND Comprehensive testing MODELS: hybrid VALIDATIONS: and model validations INTEROPERABILITY: Amazon, Eucalyptus, etc. RELATED TO ADMINISTRATION On demand networking Secure command line IAAS MODEL: INTERFACES: customization interfaces Compatibility with type I and INITIAL First deployment and HYPERVISORS: other type II hypervisors DEPLOYMENT: VM copy automation STORAGE: Decoupled and persistent Based on computing, storage, PAY-PER-USE MODEL: backup and networking resources consumption Stronger information security SECURITY: architecture …
- 36. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THANKS FOR YOUR ATTENTION!
- 37. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) CONTACT INFORMATION mj.villamizar24@uniandes.edu.co mjvc007@hotmail.com http://twitter.com/mjvc007 http://linkedin.com/in/mariojosevillamizarcano Mario José Villamizar Cano
- 38. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Ubuntu with OGS
- 39. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) THE DESIRED SOLUTION Debian with PBS
- 40. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) cmp UnaCloud Serv er Architecture IaaS Administration UNACLOUD SERVER ARCHITECTURE IaaS Customization IaaS Deployment IaaS Traceability IaaS Monitoring Administration Monitoring Physical Physical UNACLOUD SERVER The UnaCloud server architecture INTERFACE LAYER is based on three layers: WI CORE LAYER Layer 1 - Interface Layer CEM PIM VMM PM Layer 2 - Core layer EXTERNAL LAYER SCM <<uses>> Layer 3 - External layer SSM UnaCloud Client Serv ices
- 41. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD SERVER ARCHITECTURE cmp UnaCloud Serv er Architecture IaaS Administration IaaS Customization IaaS Deployment IaaS Traceability IaaS Monitoring Administration Monitoring Physical Physical UNACLOUD SERVER INTERFACE LAYER WI Layer 1 - Interface Layer: A Web Interface (WI) used by users (IaaS, Grid and CORE LAYER Admin) for accessing and consuming the services through authentication and CEM PIM authorization mechanisms. VMM PM
- 42. IaaS Mon Phy Phy Iaa Iaa Iaa Iaa Ad The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD SERVER UNACLOUD SERVER ARCHITECTURE INTERFACE LAYER WI CORE LAYER CEM PIM VMM PM Layer 2 - Core layer: it is responsible for processing all user requirements and delivering the solutions desired. EXTERNAL LAYER components: It has four main Customized Environment Manager (CEM): it prepares user orders SCM Persistence Manager (PM): information database Virtual Machine Manager (VMM): it prepares hypervisor orders <<uses>> Physical Infrastructure Manager (PIM): desktops computers of physical labs
- 43. CORE LAYER The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) CEM PIM VMM PM UNACLOUD SERVER ARCHITECTURE EXTERNAL LAYER SCM <<uses>> SSM UnaCloud Client Serv ices Layer 3 - External layer: it is responsible of providing secure communications with the UnaCloud clients. It has two main components: Server Communication Manager (SCM): connection, disconnection and message passing between UnaCloud Server and UnaCloud Clients Server Security Manager (SSM): it provides security in communications
- 44. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) cmp UnaCloud Client Architecture UNACLOUD CLIENT ARCHITECTURE IaaS Administration IaaS Customization IaaS Deployment IaaS Monitoring Administration The Client architecture is based on Monitoring Physical Physical two layers: UNACLOUD CLIENT Layer 1 - External Layer EXTERNAL LAYER Layer 2 - Core layer CSM <<uses>> It is a lightweight, highly portable CCM and easy to install client which is installed and run directly on each desktop. CORE LAYER CM MM It is responsible for receiving and LEM HM processing all of UnaCloud Server orders to provide a dynamic and on- demand IaaS model. Operating System Hyperv isor Serv ices Serv ices
- 45. The Second International Conference on Cloud Computing, GRIDs, and Virtualization (CLOUD COMPUTING 2011) UNACLOUD CLIENT ARCHITECTURE cmp UnaCloud Client Architecture IaaS Administration IaaS Customization IaaS Deployment IaaS Monitoring Administration Monitoring Physical Physical UNACLOUD CLIENT EXTERNAL LAYER CSM <<uses>> CCM Layer 1 – External Layer: it is responsible for managing secure communication CORE LAYER services on the client side. It has two main components: CM MM LEM HM Client Communication Manager (CCM): message passing with the server Client Security Manager (CSM): it provides security in communications
- 46. The Second International Conference on Cloud Computing, GRIDs, <<uses>> and Virtualization (CLOUD COMPUTING 2011) CCM UNACLOUD CLIENT ARCHITECTURE CORE LAYER CM MM LEM HM Operating System Hyperv isor Serv ices Serv ices Layer 2 - Core layer: it is responsible for attending and meeting UnaCloud Server orders through local operating system and hypervisor invocations. It has for main components: Context Manager (CM): adapt the virtual machine execution context Local Executor Manager (LEM): command execution on the local operating system Hypervisor Manager (HM): commands to invocate the local hypervisor Monitoring Manager (MM): monitoring the physical and virtual machine