Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar
6 likes3,695 views
The document discusses the deployment of Ceph block storage within an OpenStack environment using Crowbar, highlighting Dell's integrated solutions for cloud infrastructure. It emphasizes the benefits of automated deployment, scalability, and cost-effectiveness, illustrated through a case study from a university requiring a centralized data repository. Future developments and support for OpenStack versions and Crowbar enhancements are also mentioned, along with related resources for further information.
Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar
- 1. Wicked Easy Ceph Block Storage & OpenStack Deployment with Crowbar Kamesh Pemmaraju, Dell Neil Levine, Inktank
- 2. Outline • Introduction – Dell OpenStack-Powered Cloud Solution – Ceph – Crowbar • Why Crowbar + Ceph in OpenStack? – OpenStack Block Storage gap – Automation, scale • What we have done to enable things? • Customer benefits, drivers – an example • What’s coming next?
- 3. • Clouds demand significant operational and process controls • Operational decisions drive hardware and software decisions • We are finding ways to productize operations into best practices HW SW OPS Clouds Require an Operational Focus
- 4. OPS SW HW SW OPS “Crowbar” CloudOps Software Services & Consulting Reference Architecture Dell OpenStack-Powered Cloud Solution
- 5. Dell OpenStack-Powered Cloud Solution HW + SW + Services Hardware HW reference architecture • C6220, C6105,R720, R720XD servers • Storage and compute • Force 10 S60 and PowerConnect Configuration • Min 6 nodes. Max 60 nodes Software Software • OpenStack Installer and continuous integration (Crowbar) • OpenStack cloud SW • Ceph for distributed storage Operating system • Ubuntu 12.04 (host) • Windows (guests) • Linux (guests) Hypervisor • KVM Services Deployment and consulting • Hardware integration – onsite or merge center • Software installation – onsite • OpenStack consulting services − Assessment, design and installation − Delivered via partnerships with Mirantis and Canonical Support • HW/Crowbar: Dell ProSupport • OpenStack support via Canonical (coming soon)
- 6. OBJECTS VIRTUAL DISKS FILES & DIRECTORIES CEPH FILE SYSTEM A distributed, scale-out filesystem with POSIX semantics that provides storage for a legacy and modern applications CEPH OBJECT STORAGE A powerful S3- and Swift- compatible gateway that brings the power of the Ceph Object Store to modern applications CEPH BLOCK STORAGE A distributed virtual block device that delivers high- performance, cost-effective storage for virtual machines and legacy applications Ceph – Unified Storage Software
- 7. Ceph – Key Differentiators • Unified storage platform (Object + Block + File) • Ceph Intelligent Placement (CRUSH) • Ceph Intelligent Devices (Self-healing, P2P) • Ceph Intelligent Objects (Embedded Software Classes) • Ceph Integration (Linux Kernel, OpenStack, Cloudstack)
- 8. Ceph Storage Cluster (RADOS) A reliable, autonomous, distributed object store comprised of self-healing, self- managing, intelligent storage nodes LIBRADOS A library allowing apps to directly access RADOS, with support for C, C++, Java, Python, Ruby, and PHP Ceph Block Device (RBD) A reliable and fully- distributed block device, with a Linux kernel client and a QEMU/KVM driver Ceph File System (CephFS) A POSIX-compliant distributed file system, with a Linux kernel client and support for FUSE Ceph Object Gateway (RGW) A bucket-based REST gateway, compatible with S3 and Swift APP APP HOST/VM CLIENT Ceph Architecture
- 9. Ceph Monitor Nodes: – Maintain cluster map – Provide consensus for distributed decision-making – Must have an odd number – These do not serve stored objects to clients M Ceph OSD Nodes: • One Ceph OSD daemon per disk (recommended) • At least three nodes in a cluster • Serve stored objects to clients • Intelligently peer to perform replication tasks • Supports object classes Ceph RADOS Components
- 11. Mission: “A Zero Touch Cloud Installer” Servers in boxes to full function cloud in under 2 hours • Fast & Flexible – Bare metal install including BIOS & RAID config – Users can choose how their system is configured (“barclamps”) • DevOps Embracing – Ongoing Operations Model (DevOps for Clouds) – Leverages & Wraps Opscode Chef • Open – Not specific to OpenStack – Dell using for other Apps – Not restricted to Dell hardware – Apache 2 licensed What is Crowbar?
- 12. Images vs. Layers: Overview Images: Single Unit Layers: Stacked Pieces Integrations + Applications + Utilities + Operating System Configuration Operating System Utilities Application Bar Application Foo Integrations Configuration
- 13. Images vs. Layers: Lifecycle Images: Replacement Layers: Upgrade I+A+U+O/ S Config OS U Bar v1 Foo I Config I+A+U+O/ S Config I+A+U+O/ S Config I+A+U+O/ S Config OS U Bar v2 Foo I Config Bar v2
- 14. Crowbar Uses the Layer Model for Deployment Dell“Crowbar” OpsManagement Core Components & Operating Systems Cloud Infrastructure & Dell IP Extensions Physical Resources APIs, User Access, & Ecosystem Partners * Crowbar has potential to service other programs beyond OpenStack • Includes all the components required to implement an entire cloud infrastructure including ecosystems partners • Pluggable components deploy cloud infrastructure • Allow for addition of Dell IP and expansion by the community services and customers • Can integrate with Dell existing products • Delivers basic data center services and required cloud infrastructure. • Provision bare-metal servers from box to cloud WITHOUT user intervention (other than racking/cabling and some minimal configuration questions) NOT EXCLUSIVE TO DELL HARDWARE
- 15. Server How Does Crowbar Work? Dell“Crowbar” OpsManagement Operating System Network ConfigChef Server Network Config Apps DHCP CrowbarMagic Happens! Orchestration State Machine Run List Config. Hardware Config (uses “Sledgehammer”) Apps
- 16. Dell“Crowbar” OpsManagement What is a Barclamp? Barclamp Chef Recipes O/S App Packages Crowbar API & Partial UI Components & Scripts • Crowbar API & Partial UI • Adds states and transitions to Orchestration • Customizes UI specific to barclamp • Visual extensions to nodes/networks/utils • Chef Recipes • DevOps description of application • Enforced repeatable configuration • O/A App Packages • Operating system dependencies (Debs & RPMs) • Components & Scripts • Ad hoc configuration not covered by Chef
- 17. Modular Design: Barclamps The Configuration You Want to DeployDell“Crowbar” OpsManagement Core Components & Operating Systems Cloud Infrastructure & Dell IP Extensions Physical Resources APIs, User Access, & Ecosystem Partners Crowbar Deployer Provisioner Network RAID BIOS IPMI NTP DNS Logging Nova Swift Glance Nagios Ganglia enStratus RightScale Dashboard Ceph Horizon KeystoneForce10
- 18. “We should do things the new way!” Use Object Storage “App can’t be re-architected!” Use local storage! “No SPOFs please!” Use this expensive SAN! “How do the Web 2.0 folks do it?” Use distributed storage on utility hardware! The 4 Stages of Cloud Storage Thinking
- 20. Ceph Block Storage for Dell Openstack Integrated: with Cinder and Nova (via qemu-kvm) Compatible: with Glance Foundations: no SPOFs, self-healing, CRUSH Features: Copy-on-Write, Snapshots, Cloning Integrated: with Crowbar for automation
- 21. If you can’t automate it, you can’t scale it OS U Nova Node I Config OS U Nova API I Config OS U Nova Node I Config OS U Nova Node I Config OS U Nova Node I ConfigOS U Nova Node I Config OS U Nova Node I Config OS U Nova DB I Config OS U Nova Cntrl I Config OS U Dash Board I Config OS U Nagios I Config OS U Crowbar Chef I Config OS U Ceph Mon Node I Config OS U Ceph Mon Node I Config OS U Ceph Mon Node I Config OS U Ceph OSD Node I Config OS U Ceph OSD Node I Config OS U Ceph OSD Node I Config OS U Cinder API I Config API & Services Compute Nodes Storage Nodes Scale-Out Operations With OpenStack
- 22. Ceph Related Barclamps for Crowbar • https://github.com/ceph/barclamp-ceph – ceph.com packages into the Crowbar Admin Node – Ceph OSD roles (ceph-store) – Ceph Mon roles (ceph-mon) • https://github.com/crowbar/barclamp-nova – Hypervisor with RBD (ceph-client)
- 23. • Dell and Inktank have partnered to bring a complete solution for OpenStack + Ceph + Automated deployment with Crowbar • The joint solution provides: – Crowbar barclamp to – Deploy Ceph clusters automatically in a very short time – Connect those clusters to OpenStack – Professional Services, Support, and Training › Collaborative Support for Dell hardware customers – Joint Solution › Validated against Dell Reference Architectures via the Technology Partner program What Have We Done to Enable?
- 24. Customer: University to Deploy OpenStack + Ceph Using Crowbar Situation • University employs close to 900 researchers and receives hundreds of million dollars in Grants as a Top 10 Research Institution primarily working on Cancer and Genomic Projects. The University has a need to provide a centralized data repository for Researchers in order to insure compliances concerning retention of data. • The intent of the Data Repository is to provide 2TB of free storage space to each Researcher, with opportunity to purchase from the university more capacity at a very reasonable cost. Decision Drivers • University investigated using an traditional SAN storage solution which was very expenses on a per TB basis. They also investigated using public cloud storage options which also proved to be too expensive. Finally, they looked at Hadoop for the project but found that it was not a good fit for this use case. • In the end, University chose Dell/Inktank to architect a platform that would be very scalable and provide lost costs per TB and was the best of all worlds that provide compute and storage on the same hardware.
- 25. What’s Next? • May – Ceph Cuttlefish • Summer 2013 – Dell Grizzly support – Ceph Dumpling (August) • https://github.com/crowbar/crowbar/wiki/Crowbar-2.0 – Crowbar v2.0 work is happening in the open. Check it out and participate!
- 26. Don’t Miss the OpenStack Summit Sessions Havana Sessions: – Planning the Ceph Roadmap for Openstack Wednesday April 17, 2013 1:50pm - 2:30pm in Room - B113 – Features for Ceph with Cinder and Beyond Thursday, April 18, 2013 9:50 – 10:30am in Rooms – C120+121+122
- 27. Contact Information Reach Kamesh and Neil for additional information: Dell.com/OpenStack Dell.com/Crowbar Inktank.com/Dell Kamesh_Pemmaraju@Dell.com @kpemmaraju Neil.Levine@Inktank.com @neilwlevine Visit the Dell and Inktank booths in the OpenStack Summit Expo Hall