Storage for containers and cloud-native deployments - Rancher Online Meetup - March 2017
Download as PPTX, PDF5 likes2,204 views
The document outlines a Rancher Meetup focused on storage for containers and cloud-native deployments, featuring presentations from co-founders of Rancher Labs, Quobyte, and Portworx. It emphasizes Rancher's role as a container management platform, detailing features such as volume plugin management, lifecycle management, and the support for various storage vendors and cloud providers. Additionally, it covers Quobyte's data center file system and Portworx's app-aware storage solutions for cloud-native infrastructure.
Storage for containers and cloud-native deployments - Rancher Online Meetup - March 2017
- 1. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Storage for Containers and Cloud-Native Deployments March 28, 2017 #ranchermeetup
- 2. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Shannon Williams Co-Founder/VP Sales @smw355 1 Will Chan Co-Founder/VP Engineering @willchan00 #ranchermeetup
- 3. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Bjorn Kolbeck Co-Founder Quobyte @quobyte 2 #ranchermeetup Gou Rao Co-Founder and CTO Portworx @raogou
- 4. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . First things first… This is a not a ! 3 #ranchermeetup
- 5. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .4 There are rules for a meetup! • We won’t be done on time • Questions are always welcome • Demo, then demo some more • Things will break, be patient #ranchermeetup
- 6. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .5 Join the conversation on Twitter #ranchermeetup
- 7. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Agenda • Rancher Overview – Shannon • Storage Concepts and Data Persistence in Rancher – Will • Quobyte Overview – Bjorn • Portworx Overview – Gou • Latest Rancher Release Details – Will 6 #ranchermeetup
- 8. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Rancher Labs 7 #ranchermeetup An open-source software platform for managing containers A minimalist OS built explicitly to run Docker
- 9. © 2017 Rancher Labs, Inc. A complete container management platform that makes it easy to… 8 INNOVATE WITH CONTAINERS without compromising flexibility by empowering developers with fast access to the latest tools MANAGE APPLICATIONS by simplifying day to day application lifecycle management RUN CONTAINERS with the most complete set of container and infrastructure management capabilities Production ready ✔ 20 million+ downloads ✔ Open platform for innovating ✔ Easy to use interface ✔ Multi-tenant ✔ Role based access ✔ 24X7 support ✔ And more….
- 10. © 2017 Rancher Labs, Inc. Complete Container Management Platform 9 Application Catalog Container Orchestration and SchedulingUser Mgmt RBAC AD/LDAP SAML Ops Mgmt CI/CD Registries Monitoring Networking Multi-tenant Environments Environment 1 Environment N Infrastructure Services Storage ……. ..Environment 2 Security DNS/LB
- 11. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .#ranchermeetup Storage for Containers and Cloud- Native Deployments
- 12. © 2017 Rancher Labs, Inc. Docker Volume Plugins • Released as part of Docker 1.8 in Aug 2015. • Since then, there has been many volume plugins created including storage plugins to: • cloud providers (AWS, GCE, Azure, DO) • storage vendors (NetApp, EMC, Quobyte, Portworx) • more frameworks (rex-ray, openstorage) • While is this great, you still have to download it, install it, and manage it…
- 13. © 2017 Rancher Labs, Inc. Volume Plugin Manager • Rancher is your Docker volume plugin manager and provides the following: • Distribution – automatically deploys volume plugins to desired hosts managed by Rancher so they are made available immediately • Version Management and Upgrades – provides version management and allows users to automatically update plugins on all hosts with a single click (or API call) • Lifecycle Management – Provides health checks and automatic redeployment of plugins if things go wrong • Plugin-aware Container Scheduling – provides container scheduling based on deployment of plugins so you never schedule containers on hosts that have no access to storage
- 14. © 2017 Rancher Labs, Inc. How is this done? • Leverage Rancher’s catalog to for version management, upgrades, and distribution. • Leverage Rancher for lifecycle management of the volume plugin which needs to be packaged as a Docker container. • Leverage Rancher for container scheduling.
- 15. © 2017 Rancher Labs, Inc. Supported Plugins
- 16. © 2017 Rancher Labs, Inc. Supported Plugins • Rancher already supports NFS with EBS and EFS made GA in 1.6 (end of April) • Open source contributions include plugins for Rex-ray • Vendor contributions include Netapp, Portworx, and Quobyte
- 17. © 2017 Rancher Labs, Inc. Docker Plugins Storage Vendors Rancher Supported Cloud Providers
- 18. QUOBYTE - THE DATA CENTER FILE SYSTEM Fast and Reliable SoftwareStorage
- 19. Introduction • Björn Kolbeck Co-Founder atQuobyte 2006 2010-2013 2013 1 8
- 20. Google-Style Infrastructure Data Center “Warehouse-scale” Container Infrastructure Linux + Containers +Borg Google File System Replication + ErasureCoding Applications 1 9
- 21. GIFEE … but I’m not Google?! 2 0 • Scalability • Agility /Flexibility • Fault-tolerance • Commodityhardware • Containers (Kubernetes) • Storage? – Google has customstack – Everyone else: POSIX & legacy apps
- 22. Quobyte Data Center File System Scale-out shared POSIX filesystem Linux Linux Linux Applications MySQL, LAMP, Cassandra, ElasticSearch… 2 1 Commodity Servers no RAID, NVRAM, journalingdevice…
- 23. Data Center File System Reliability in Software: Storage must be cattle! • Split-brain safereplication • Erasure Coding • End-to-end checksums • Never trust thehardware! • Ignore: Broken/slow drives, servers, network cards,packet loss, network splits, brokenswitches… 2 2
- 24. Data Center File SystemLinear Scalabiliy without Bottlenecks • 2x4x6xServers = 2x4x6xIOPS, throughput, capacity • No bottleneck: Big data, video, webserver, hpc… • Grow compute &storage together 2 3
- 25. Data Center File System High Performance for All Workloads • High performance for all workloadtypes – Throughput – IOPS – Metadata intensive (small fileworkloads) – Consistent low latency <1ms • Share data viainterfaces – File system (Linux, Mac,Win) – S3 – Hadoop stat read(4k) write(4k) 420µs 2 4 150µs100µs
- 26. Data Center File System Smart Data Placement: Optimize your Apps ➡ Tiering ➡ Isolation ➡Application-aware data placement • Filename • Extension • Application • Age /Access 2 5 HDDSSDDatabase
- 27. Data Center File System Smart Data Placement: Cassandra fast and cheap Commit log(*.log) • Appendworkload • Replication:x3 • NVMe • Transactions <500usec SSTables(*.db) • Read-only •EC: x1.375 • Read optimizedSSDs • Lowest $/IOPS 10
- 28. Quobyte + Containers Integrated • Persistent Volume Plugin (>=1.4) – Global, replicas,container… – Automatic volume provisioning • UserId mapping (Quobyteblog) • Volume plugin • https://github.com/quobyte/docker-volume 27
- 29. Get Quobyte Today! • www.quobyte.com/get-it • Soon: RancherCatalog • hello@quobyte.com • Meet usat 28
- 30. © 2017 Rancher Labs, Inc. Data Services for Cloud Native Applications
- 31. © 2017 Rancher Labs, Inc. Portworx + Rancher: The Transformation to Cloud Native DevOps Infrastructure X86 Commodity Servers or Cloud Monitoring Orchestration/Scheduling Minimal OS Portworx Cloud Native Infrastructure Fixed Infrastructure Security Kubernetes w/ Rancher Containers HP EMC NetApp IBM Cisco VMware Dell Oracle RedHat Microsoft
- 32. © 2017 Rancher Labs, Inc. The Problem: Static, Out of Band Storage Provisioning SSD HDD nginx python mysql SAN html5 rest sql Apps hit a wall going to production Traditional Storage forced into a container, Out-of-Band Applications App Unaware Storage Cloud Connectors like RexRay SSD EBS
- 33. © 2017 Rancher Labs, Inc. Persistence IOPs Connectors SSD HDD SAN EBS S3 Admins Realize that Apps Don’t Want Volumes 1. Don’t make data protection a science project 2. Stop trying to connect legacy SAN/NAS/SDS to containers – They are not container or application aware 3. Your applications want on-demand services delivered programmatically via the scheduler 4. Storage services need to be “app-aware”, for example Cassandra-aware – Traditional storage is not.
- 34. © 2017 Rancher Labs, Inc. The Solution: Data Services built for DevOps for ANY Infrastructure nginx python mysql html5 rest sql SSD HDD SAN Cloud Portworx SSD EBS • App-aware • Any infrastructure • Programmatic No more volumes or storage per application to manage Rancher with Kubernetes
- 35. © 2017 Rancher Labs, Inc. Programmatic Data Services for DevOps WordPress WordPress PX MySQL MySQL persistence encryption CoS namespace Auth/ACLs HA Operational Experience Delivered as a lightweight container with support for volumes and graph. Runs on any server, any cloud. Converged Performance Data collocated with container compute Container Granular Controls Encryption, CoS, replication, snapshots are done at container granularity Hybrid Cloud Ops Cloud agnostic deployment and data accessibility across multiple clouds S3 Object
- 36. © 2017 Rancher Labs, Inc. Portworx is Topology and Application Aware Application … x86 Servers x86 Servers rack x86 Servers x86 Servers rack x86 Servers x86 Servers rack vol volvol vol vol Scheduler Portworx places an application’s data ON THE NODE that it executes on Container granular volumes on ANY infrastructure
- 37. © 2017 Rancher Labs, Inc. Portworx vs ScaleIO or CEPH x86 Servers x86 Servers rack … x86 Servers x86 Servers rack Portworx Places an Application’ss Data Local to a Few Topology Constrained Servers. Cassandra runs converged. x86 Servers x86 Servers rack … x86 Servers x86 Servers rack x86 Servers ScaleIO and CEPH shard an application’s data across many servers to get capacity. This breaks convergence. You also don’t have application granular settings on the volumes.
- 38. © 2017 Rancher Labs, Inc. Portworx is Enterprise Grade Software
- 39. © 2017 Rancher Labs, Inc. Demo: Portworx in the Rancher Catalog Demo
- 40. © 2017 Rancher Labs, Inc. Why Rancher: from the Portworx Perspective ▪ Start to finish • Launch Containers and Infrastructure services in seconds • Rich Community + Enterprise Catalog of services ▪ Deploy anywhere • Support for top cloud providers like Amazon, Digital Ocean, Packet … • Broad community of users moving to cloud-native microservices
- 41. © 2017 Rancher Labs, Inc. Portworx is Open Core OpenStorage.org Scheduler Data Layer: OpenStorage is a massively scalable data layer for Containers and Schedulers like Kubernetes and Mesosphere LCFS Data Path: LCFS provides core data path functionality for Docker images. Check it out on https://github.com/portworx/lcfs PORX Bundled Data Services: Try out the Portworx community edition at https://github.com/portworx/px-dev
- 42. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc .41 Demo #ranchermeetup
- 43. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Latest Release 42 Rancher 1.5 – March 3, 2017 Key Features: - Templating support for catalog items - API Interceptor - Enhanced Network Policies - Webhooks – Host Scaling, DockerHub upgrade - Metadata Improvements - Portainer.io for Swarm UI
- 44. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Next Releases 43 Rancher 1.6 – Late April Key features: - Catalog Enhancements – Ability to add catalogs per environment - HAProxy Enhancements - - Certifying Rancher-EBS - - Scheduling Enhancements - schedule containers evenly across pools of hosts - K8S 1.6 Support with Docker 1.13/Docker 17.03 and etcdv3
- 45. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Getting Started Rancher and RancherOS are in GitHub – Get Involved! 44 #ranchermeetup http://github.com/rancher
- 46. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Even better - try.rancher.com… 45
- 47. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Then join a free training class… 46 http://rancher.com/training
- 48. © 2017 Rancher Labs, Inc.© 2017 Rancher Labs, Inc . Thank you rancher.com #ranchermeetup
Editor's Notes
- #10: Rancher is a complete container management platform that makes it easy to… Run containers in production with the most complete set of container and infrastructure management capabilities Manage applications by simplifying day to day application lifecycle management Innovate with containers without compromising flexibility by empowering developers with fast access to the latest tools And Rancher is production ready Enterprise-class features such as role-based access control, integration with LDAP and Active Directories, detailed audit logs, high-availability management servers and encrypted networking are available out of the box. With over 2.4 million downloads and optional 24x7x365 support, Rancher has quickly become the platform of choice for devops and IT teams who are serious about running containers in production More than 30 customers are using Rancher in production including large enterprise such as US Bank, IBM, Cisco, Invisalign and more.
- #32: DevOps driven IT = Services… Terraform, Kubernetes, Docker, Data Services – Think commodity HW
- #33: DevOps started with stateless – now with state, there are issues.
- #35: Data layer that neutralizes the physical infrastructure
- #36: Specific services provided by PXß