Turning OpenStack Swift into a VM storage platform
- 1. OpenStack Meetup: Turning OpenStack Swift into a VM storage platform Wim Provoost (@wimpers_be) Open vStorage (@openvstorage)
- 2. CloudFounders • CloudFounders (2007) – 120 people in the US, Belgium, Romania and India • Products – Open vStorage (open source under an Apache 2.0 license) • Ultra reliable, high performance, open storage layer for virtual machines. • Compatible with all major hypervisors. • Supports OpenStack – vRun = Converged Infrastructure • Software that transforms server hardware to converged infrastructure with unparalleled performance & flexibility. • Software can run on any x86 server or as a virtual machine (VMware/OpenStack) • Uses Open vStorage as basis • Competes with Nutanix, Simplivity and upcoming EVO:RAIL
- 3. Object storage: the future of scale out storage Object storage is the future for storing massive amounts of data: • PRO – Build scale-out storage by adding more disk/servers when needed – High reliability by distributing content across nodes – Easy to manage (no knowledge needed about RAID or volumes) – Standard API as all solutions support S3 API, no vendor lock-in – Cost-effictive: use low cost, large capacity SATA disks, increase performance by adding spindles/gateways • CON – NO support for Virtual Machine Disk Storage (see next slide)
- 4. Object storage challenges as VM storage • Eventual consistency (the CAP Theorem) • Latency & performance – VMs require low latency and high performance – Object stores are developed to contain lots of data (large disks, low performance) – Additional latency as Object Store is on the Local LAN instead of attached to the host like DAS • Different Management Paradigms – Object Stores understand Objects <> Hypervisors understand blocks, files What is needed is a technology whereby Virtual Machines can use object stores instead of a SAN and get the benefits of the low cost and scale-out capabilities of object stores
- 5. Traditional OpenStack Setup Nova Instance Management Swift Object Storage Cinder Block Storage Provides image for Glance Image store VM Provides volume for Provisions Stores image in Stores backups in Provides disk space Ceph, SAN, NAS, ... 2 storage platforms?!
- 6. The answer: Open vStorage Nova Instance Management Swift Object Storage Provides image for Glance Image store VM Provides volume for Provisions Stores image in Stores backups in Open vStorage Cinder Block Storage Provides disk space Converts Object storage into Block storage
- 7. What is Open vStorage Open vStorage is an open source “Storage Router“ and is installed on a host or a cluster of hosts to create a VM-centric, clustered, reliable, scale out and high performance storage system for virtual machines.
- 8. Open vStorage Intro • +40.000 IOPS per Hypervisor • Made for virtual machines • Unified Namespace • Ultra Reliable • Unlimited Snapshots • Endless Scalable • Lowest Management Cost In Market Hypervisor Hypervisor SAN / NAS Hypervisor S3 compatible Object Based Storage Hypervisor Open v Storage Open v Storage Open v Storage Open v Storage Distributed File Systems
- 9. Ultra Scalable Storage For Virtual Machines KVM Scale-out VM VM VM VM SSD SSD Open vStorage KVM VM VM VM VM SSD SSD Open vStorage KVM VM VM VM VM SSD SSD Open vStorage Virtual File System interface
- 10. Solving Eventual Consistency using time based approach LBA 1: 4k block 1 SSD or PCI Flash LBA 2: 4k block 2 LBA 3: 4k block 3 LBA 4: 4k block 4 LBA 5: 4k block 5 LBA 1: 4k block 6 LBA 1: 4k block 7 LBA 3: 4k block 8 LBA 6: 4k block 9 LBA 7: 4k block 10 LBA 8: 4k block 11 LBA 2: 4k block 12 LBA 9: 4k block 13 LBA 10: 4k block 14 New writes 4k block 1 4k block 2 4k block 3 4k block 4 4k block 5 4k block 6 4k block 7 SCO 1 4k block 8 4k block 9 4k block 10 4k block 11 4k block 12 4k block 13 4k block 14 SCO 2 LBA 5: 4k block 15 LBA 10: 4k block 16 4k block 15 4k block 16 New writes SCO 3 SCO1 SCO2 Transfer SCOs once they are full (4MB) to the Storage Backend at slow pace Each write is appended to the current Storage Container Object (SCO)
- 11. Open vStorage <> distributed file system VFS2 VFS3 VSA 1 VSA 2 VSA 3 Arakoon – (config params, metadata, ...) vDisk 1 vDisk 2 Internal Bucket vDisk 3 xml VOL DRV VM VOL DRV Object Router FILE DRV FILE DRV Object Router VOL DRV FILE DRV KVM1 KVM2 KVM3 VFS1 Object Router
- 12. Live Motion – In depth (Phase 1) VSA 1 VSA 2 VSA 3 Arakoon – (config params, metadata, ...) vDisk 1 vDisk 2 Internal Bucket vDisk 3 VFS3 vmx VOL DRV VOL DRV Object Router FILE DRV FILE DRV VM Object Router VOL DRV FILE DRV KVM1 KVM2 KVM3 VFS1 Live Motion VM Object Router VFS2
- 13. Live Motion – In depth (Phase 2) VFS2 VFS3 VSA 1 VSA 2 VSA 3 Arakoon – (config params, metadata, ...) vDisk 1 vDisk 2 Internal Bucket vDisk 3 xml VOL DRV VOL DRV FILE DRV FILE DRV VM Object Router VOL DRV FILE DRV KVM1 KVM2 KVM3 VFS1 Live Motion VM Object Router Handover Object Router
- 14. How does Open vStorage solve the problem • Open vStorage is a middleware layer in between the hypervisor and the object store. (Converts object storage into block storage) – On the host: location based storage (block storage). – On the backend: time based storage (ideal for objects stores). – Open vStorage turns a volume into a single bucket. • OpenStack Cinder Plugin for easy integration (snapshots, ...). • Distributed file systems don’t work! Open vStorage is not a distributed file sysem! – All hosts ‘think’ they see the same virtual file systems. – Volume is ‘live’ on 1 host instead of all hosts. – Only the virtual file system metadata is distributed. • Caching inside the host fixes impedance mismatch between slow, high latency backend and fast, low latency requirement of Virtual Machines.
- 15. The benefits • Single distributed storage platform (Swift) to maintain. This platform is very performant and almost infinitely scalable. • Better performance due to caching inside the host where the Nova instance runs. • VM-centric management on storage level. – Thin-cloning, zero-copy snapshots, ... • Add storage which isn’t compatible with Cinder.
- 16. Supported OpenStack actions • Volumes – Add / Remove – Create / Delete snapshot – Clone from snapshot – Clone volume from volume – Import image to volume – Export volume to image • Instances – Live Migration – Evacuate
- 17. Deployments • Our internal environments are now running Open vStorage + swift • Open vStorage is available in beta – Certified OpenStack Cinder plugin is expected with Kilo release – Feel free to use in test environments and provide feedback – Supported production release Q1
- 18. Next steps • The code: https://bitbucket.org/openvstorage/openvstorage/ • Download the packages and deploy in your OpenStack environment • Documentation: http://doc.openvstorage.com • Support: https://groups.google.com/forum/#!forum/open-vstorage • Follow us on Twitter (@OpenvStorage) or read our blog (http://blog.openvstorage.com)
- 19. OpenStack Paris November, 3-7 We will be there! Look out for the “Cloudfounders” booth.