week 3 cloud computing northumbria foudation
- 1. LD5012 – Cloud Computing Technology Dr. Rejwan Bin Sulaiman < rejwan.sulaiman@northumbria.ac.uk > Module Leader: Cloud Computing Technology 1
- 2. Learning Objective • Understanding Virtualization • Types of Virtualization • Exploring Hypervisors and Virtual Machines • Cloud Storage Models and Types • Compute Resources in the Cloud • Backend Infrastructure and Cost Efficiency
- 3. Before Virtualization Single OS image per machine Software and hardware tightly coupled Running multiple applications on same machine often creates conflict Inflexible and costly infrastructure
- 4. Hardware-independence of operating system and applications Virtual machines can be provisioned to any system Can manage OS and application as a single unit by encapsulating them into virtual Machines After Virtualization
- 6. Challenges The x86 computer hardware was designed to run a single operating system Low Utilization - on average servers utilization rates of <15% Multiple test / dev environments need multiple machines Data centers needing huge power, high operating expenses and under utilized processing power Inability to scale resources for apps needing more resources during peakloads
- 7. Consolidation Approaches Data Consolidation Combination of data from different sources into a single repository and format Application Consolidation Reducing the number of applications Physical Consolidation Hosting multiple application on one server Centralization Consolidation of management control and system to a central location
- 8. Virtualization Basics A virtual machine is a software computer that, like a physical computer, runs an operating system and applications. Each virtual machine contains its own virtual, or software-based, hardware, including a virtual CPU, memory, hard disk, and network interface card. Virtualization is a technique of how to separate a service from the underlying physical delivery of that service. It is the process of creating a virtual version of something like computer hardware. With the help of Virtualization, multiple operating systems and applications can run on same machine and its same hardware at the same time, increasing the utilization and flexibility of hardware. One of the main cost effective, hardware reducing, and energy saving techniques used by cloud providers is virtualization
- 9. Virtualization Basics Virtualization allows to share a single physical instance of a resource or an application among multiple customers and organizations at one time. It does this by assigning a logical name to a physical storage and providing a pointer to that physical resource on demand. The term virtualization is often synonymous with hardware virtualization, which plays a fundamental role in efficiently delivering Infrastructure-as-a-Service (IaaS) solutions for cloud computing. Moreover, virtualization technologies provide a virtual environment for not only executing applications but also for storage, memory, and networking. The machine on which the virtual machine is going to be built is known as Host Machine and that virtual machine is referred as a Guest Machine.
- 10. BENEFITS OF VIRTUALIZATION More flexible and efficient allocation of resources. 2. Enhance development productivity. 3. It lowers the cost of IT infrastructure. 4. Remote access and rapid scalability. 5. High availability and disaster recovery. 6. Pay peruse of the IT infrastructure on demand. 7. Enables running multiple operating systems.
- 12. Application Virtualization Application virtualization helps a user to have remote access of an application from a server. The server stores all personal information and other characteristics of the application but can still run on a local workstation through the internet. Application virtualization software allows users to access and use an application from a separate computer than the one on which the application is installed. Using application virtualization software, IT admins can set up remote applications on a server and deliver the apps to an end user’s computer. For the user, the experience of the virtualized app is the same as using the installed app on a physical machine.
- 13. Network Virtualization The ability to run multiple virtual networks with each has a separate control and data plan. It co-exists together on top of one physical network. It can be managed by individual parties that potentially confidential to each other. Network virtualization provides a facility to create and provision virtual networks—logical switches, routers, firewalls, load balancer, Virtual Private Network (VPN), and workload security within days or even in weeks. Network Virtualization is a process of logically grouping physical networks and making them operate as single or multiple independent networks called Virtual Networks.
- 14. Desktop Virtualization Desktop virtualization allows the users’ OS to be remotely stored on a server in the data centre. It allows the user to access their desktop virtually, from any location by a different machine. Users who want specific operating systems other than Windows Server will need to have a virtual desktop. Main benefits of desktop virtualization are user mobility, portability, easy management of software installation, updates, and patches.
- 15. Storage Virtualization Storage virtualization pools physical storage from multiple devices into a single, centralized storage resource. Servers access storage as a unified pool without knowing specific data locations. Simplifies storage management by treating diverse sources as a single repository. Software enables this pooled capacity, accessible by both traditional servers and virtual machines.
- 16. Server Virtualization Server virtualization divides a physical server into multiple, independent virtual servers using software. Each virtual server can run its own operating system in isolation from others. This process boosts performance and reduces costs by efficiently allocating main server resources. Benefits include virtual migration, lower energy use, and reduced infrastructure expenses.
- 17. Data Virtualization • This type of virtualization collects and manages data from multiple sources in a single location, abstracting technical details. • Data is logically organized to provide a virtual view accessible to stakeholders via cloud services. • Major providers include Oracle, IBM, and CData, among others. • It supports tasks like data integration, business integration, service-oriented architecture, and organizational data searches.
- 18. Virtualization: Hypervisor The virtualization layer consists of a hypervisor or a Virtual Machine Monitor (VMM). There are two types of hypervisors • Type-1Hypervisors or Native Hypervisors • Type-2Hypervisors or Hosted Hypervisors Type-1 Hypervisors or Native Hypervisors : Type-1 Hypervisors or Native Hypervisors run directly on the host hardware and control the hardware and monitor the guest operating system. Type 2 Hypervisors or Hosted Hypervisors : Type 2 Hypervisors or Hosted Hypervisors run on top of a conventional (main or Host) operating system and monitor the guest operation systems.
- 19. Xen Architecture The Xen Architecture: Xen is an open source hypervisor program developed by Cambridge University. Xen is a micro-kernel hypervisor, which separates the policy from the mechanism. The Xen hypervisor implements all the mechanisms, leaving the policy to be handled by Domain 0. Xen does not include any device drivers natively. It just provides a mechanism by which a guest OS can have direct access to the physical devices. As a result, the size of the Xen hypervisor is kept rather small. Xen provides a virtual environment located between the hardware and the OS.
- 20. Virtual Machine A tightly isolated software container • Has its own operating systems and applications as if it were a physical computer. • Behaves exactly like a physical computer • Contains it own virtual (ie, software- based) CPU, RAM, Hard disk and NIC.
- 21. VM Management and Isolation VM Management: Resource allocation: CPU, RAM, storage, and network bandwidth are managed by the hypervisor. Tools: Hypervisor-based tools (e.g., vCenter, Hyper-V Manager) for provisioning and monitoring. Snapshot and backup capabilities for VM recovery. Isolation:Each VM operates in its own environment, ensuring security and minimizing cross-VM interference. VM isolation prevents direct access to the host machine’s resources, enhancing security. Example: Illustration of isolated VMs on a single
- 22. Use Cases of VMs in Cloud Services Testing and Development: Developers use VMs to test software in isolated environments without affecting the production system. Disaster Recovery: VMs can be replicated and restored easily, ensuring quick disaster recovery. Scalable Applications: VMs allow easy scaling for applications, ideal for handling variable workloads. Hosting and Deployment: Cloud providers offer VM instances (e.g., AWS EC2, Azure VMs) for running applications and hosting websites. Graphic: Depiction of various cloud-based applications using VMs.
- 23. Introduction to Containers vs. Virtual Machines (VMs) Definition of Containers:Containers are lightweight, portable units that package code and its dependencies, allowing applications to run consistently across environments. Key Differences from VMs:VMs run full operating systems (OS) with separate kernel instances. Containers share the host OS kernel, making them more lightweight and efficient. Diagram: Comparison visual – side-by-side illustration showing VMs with individual OS instances vs. containers sharing a single OS kernel.
- 24. How are containers and virtual machines related? Feature Container Virtual machine Operating system Shares the host operating system's kernel Has its own kernel Portability More portable Less portable Speed Faster to start up and shut down Slower to start up and shut down Resource usage Uses fewer resources Uses more resources Use cases Good for portable and scalable applications Good for isolated applications
- 25. Cloud storage Cloud storage is a model of data storage in which the digital data is stored in logical pools, the physical storage spans multiple servers (and often locations), and the physical environment is typically owned and managed by a hosting company.
- 26. History – J.C.R. Licklider – One of the fathers of the cloud-based computing idea. – Global network that allows access from anywhere at anytime. – Technological limits of the 60’s. Concept – Cloud storage is a service model in which data is maintained, managed and backed up remotely and made available to users over a network (typically the Internet). What is cloud storage?
- 27. How does cloud storage work Remote Storage: Cloud storage uses remote servers to save your data. These servers are located in data centers. Data Upload: You upload your data to these servers using an internet connection. This data can be anything from files to photos. Data Distribution: Cloud providers often distribute your data across multiple servers. This ensures high availability and redundancy. Data Access: You can access your data from any device with an internet connection. This makes it easy to collaborate and access files from anywhere. Security: Cloud providers implement security measures to protect your data. These measures include encryption, access controls, and regular backups
- 28. Types of Cloud Storage The three main types of cloud storage are: 1.Block Storage : Suitable for applications requiring fast and consistent I/O performance, like databases (e.g., AWS EBS, Azure Disk Storage). 2.Object Storage : Ideal for storing unstructured data, such as media files and backups, often used for scalable storage solutions (e.g., AWS S3, Azure Blob Storage). 3.File Storage : Provides a shared file system, typically used in environments needing centralized file access across multiple machines (e.g., Azure File Storage, Amazon FSx).
- 29. Object Storage: Definition and Structure Object storage stores data as discrete units known as “objects.” Each object contains the data, metadata, and a unique identifier. Objects are stored in a flat structure within “buckets” rather than a hierarchical directory, making them ideal for unstructured data. Characteristics: • Scalability: Easily scales horizontally to accommodate vast data volumes. • Accessibility: Accessed over HTTP/S, suitable for web apps. • Durability: Redundancy and data replication ensure data integrity.
- 30. Object Storage - Use Case: Amazon S3 Service Overview: Amazon S3 (Simple Storage Service) provides highly scalable object storage with a pay-as-you-go model, ideal for large unstructured data. Features: • Storage Classes: Options like S3 Standard, S3 Glacier for cost- effective long-term storage. • Durability: 99.999999999% durability with automatic replication across regions. • Data Access Controls: Supports fine-grained permissions and access policies for security. Example Use Cases: • Media Streaming: Store and deliver high-resolution video content. • Data Lakes for Analytics: Ideal for big data processing, ML, and IoT data storage.
- 31. Block Storage - Introduction • Block storage splits data into fixed-size blocks, each with a unique identifier. Data is stored in “blocks” across a storage area and can be accessed individually. Structure: Offers high-speed, low-latency access, making it suitable for structured data. Characteristics: • High Performance: Fast data retrieval, ideal for databases. • Resilience: Block-level replication and snapshot options for data recovery. • Flexible Volume Size: Supports resizing based on application needs.
- 32. Block Storage - Use Case: Amazon EC2 with EBS Amazon Elastic Block Store (EBS) provides persistent block storage for EC2 instances, optimized for performance-sensitive applications. Features: Volume Types: Options like General Purpose SSD, Provisioned IOPS SSD for various performance needs. Data Persistence: Storage persists independently from the EC2 instance lifecycle. Scalability: Resize volumes and adjust performance without disrupting workloads. Example Use Cases: Database Storage: High-performance SSD for RDBMS and NoSQL databases. Enterprise Applications: ERP and CRM systems requiring rapid data access.
- 33. File Storage File storage organizes data in a hierarchical structure using files and folders, like traditional file systems, accessible over network file protocols. Structure: File data is managed via standard protocols (e.g., SMB, NFS) and accessible by multiple users simultaneously. Characteristics: • Familiar Structure: Folder-based organization is user-friendly for teams. • Shared Access: Supports multi-user access, making it ideal for collaboration. • Network Access: Files are accessible via network paths across environments.
- 34. File Storage - Use Case: Amazon FSx • Service Overview: Amazon FSx offers fully managed file storage optimized for applications that require shared file systems. • Features: • FSx for Windows: SMB protocol support, integration with Active Directory. • FSx for Lustre: High-performance file system for compute-intensive workloads. • Scalability and Security: Easily scales with managed encryption and backup. • Example Use Cases: • Shared Team Storage: Collaborative file storage for distributed teams. • Data Processing for HPC: Ideal for high-performance computing with FSx for Lustre.
- 35. Data Redundancy in Cloud Storage Data redundancy involves storing multiple copies of data across different locations to ensure availability and prevent data loss. Types of Redundancy: • Geo-Redundancy: Storing data across geographically distributed locations for disaster recovery. • Zone-Redundancy: Within a single region but across multiple availability zones (e.g., AWS AZs). • Replication: Real-time duplication of data in the same or different locations. Purpose: Ensures data integrity, high availability, and continuity in the event of failures.
- 36. Backup Mechanisms in Cloud Storage Snapshot-Based Backups: Capture incremental changes in data, efficient for restoring previous states. Scheduled Backups: Automated regular backups for consistent data protection. Versioned Backups: Store multiple versions of files for retrieval in case of accidental deletion or modification. Key Services: Examples like AWS Backup, Azure Site Recovery, Google Cloud Backup & DR.Benefits:Data protection from accidental deletions, disasters, and cyber threats. Flexibility in recovery points and reduced downtime in recovery.
- 37. Role of Backend Infrastructure in Cloud Services Overview of Core Backend Components: Virtualization: Enables multiple virtual machines (VMs) to run on a single physical machine, improving resource utilization. Storage Solutions: Object, block, and file storage options allow flexible and scalable data storage. Computing Resources: On-demand computing power (e.g., EC2, Azure VMs) provides scalable processing capability. How These Components Work Together:Virtualization optimizes resource use, storage ensures data availability, and computing power enables scalable application deployment. Together, they form the backbone of cloud environments, supporting multi-tenancy, scalability, and agility.
- 38. Security Considerations in Cloud Infrastructure Key Security Measures: • Data Encryption: Encrypts data in transit and at rest (e.g., AWS KMS, Azure Key Vault). • VM Isolation: Separates virtual machines on the same physical host to prevent unauthorized access. • Access Control: Fine-grained access policies (e.g., IAM roles) restrict access to authorized users only. Additional Security Layers: • Firewall Rules and Network Security: Controls data traffic with virtual firewalls, VPNs, and security groups. • Multi-Factor Authentication (MFA): Adds a second layer of security to access management.
- 39. Cost Efficiency in Cloud Infrastructure Cost-Optimizing Strategies: • Virtualization: Allows multiple VMs on a single physical machine, reducing hardware costs. • Storage Options: Offers cost-effective storage tiers (e.g., S3 Glacier for long- term archival). • On-Demand vs. Reserved Instances: On-demand for flexible usage, reserved instances for predictable workloads to save costs. • Autoscaling: Automatically adjusts resources to meet demand, optimizing costs for fluctuating workloads. Example Cost-Saving Services: AWS Cost Explorer, Azure Cost Management.
- 40. Reference • Virtualization vs. Cloud computing. (2023). Oracle.com. Retrieved 29 October 2024, from https://www.oracle.com/uk/cloud/what-is-cloud-computing/virtualization-vs-cloud-computing/ • What is Virtualization? (2024). Amazon.com. https://aws.amazon.com/what-is/virtualization/#:~:text=Virtualization%20improves %20hardware%20resources%20used,to%20the%20pool%20as%20required. • What is virtualization? (2024, August 12). Ibm.com. https://www.ibm.com/topics/virtualization
- 41. Lab Learning Objective • Understand the importance of securing cloud resources. • Implement encryption for data at rest and in transit. • Configure Identity and Access Management (IAM) for access control. • Set up firewall rules to control network traffic. • Apply best practices for a secure cloud environment.
- 42. Lab Videos Identity and Access Management for Beginners (IAM) Moving to Cloud: Security Basics and Configuration AWS IAM User - Step-by-Step Tutorial AWS Network Firewall Step-By-Step Demo Reading Materials AWS Certified Solutions Architect Official Study Guide* (Chapter 5) • https://learning.oreilly.com/library/view/aws-certified-solutions/9781119982623/
- 43. Thank you Any question? 10/30/2024 Sample Footer Text 43