CloudByte Technology Whitepaper
1 like710 views
CloudByte's ElastiStor storage controller uses a patented TSM architecture that fully isolates each application at all storage stack levels, allowing resources to be intelligently provisioned based on an application's performance needs. This enables shared storage to deliver guaranteed quality of service to thousands of applications. ElastiStor controllers can scale linearly and use standard servers with no proprietary hardware, reducing costs by 80-90% over 3-5 years compared to dedicated storage islands.
CloudByte Technology Whitepaper
- 1. White Paper Technology STORAGE ARCHITECTED FOR THE NEW-AGE DATACENTERS CloudByte ElastiStor makes Storage Predictable, Affordable and Easy even as Datacenters Scale to Thousands of Applications
- 2. Table of Contents Executive Summary ……………………………………………………………………………………. 3 Legacy Solutions are a Misfit in the New-Age Datacenters …………………….. 4 CloudByte Technology TSM Architecture ……………………..…………………………………………………………. 5 Heuristics Based Performance Control ……………..………………………………… 6 CloudByte Deployment Architecture …………………….……..………………………….... 7 ElastiStor: Key Features …………………….……..……………………………..………………….. 8 ElastiStor: Standard Storage Features ……..…………………………………………………. 9 2
- 3. Executive Summary Legacy storage solutions fail to scale up to the demands of new-age datacenters, which are witnessing a rapid increase in number of applications and new levels of performance requirements. While virtualization has made rapid strides on the server side, storage technology witnessed just incremental innovations in the past decade. By baking in advanced technologies of virtualization and software-defined intelligence, ElastiStor is architected to make storage predictable, affordable, and easy, even as datacenters scale to thousands of applications. CloudByte ElastiStor controllers are built for multi-tenancy i.e., every application hosted on a shared storage platform is completely isolated and dedicated its own set of storage resources. ElastiStor intelligently provisions these storage resources based on an application’s performance (QoS) demands. This allows datacenters to realize the cost efficiencies of shared storage, while delivering guaranteed QoS to every application. In addition to affordably delivering performance to a large number of applications, ElastiStor offers complete security, comprehensive management tools and superior reliability, that’s expected in carrier-grade storage solutions. Software-only and software-defined, ElastiStor is installable on industry-standard servers. With zero-proprietary hardware and OpenStorage, ElastiStor frees datacenters from any proprietary lock-in and allows storage infrastructure to be custom-built, whether it’s SATA, SAS or SSD. 3
- 4. Legacy Solutions are a Misfit in the New-Age Datacenters Legacy storage solutions were designed to handle stable workloads from applications hosted on dedicated physical servers. However, with server virtualization and the new class of enterprise applications, datacenters have to deal with rapidly increasing number of applications, new levels of performance demands, which are often dynamic and hosted on virtual servers. Due to their architecture limitations, legacy solutions cannot scale well in these virtualized environments. Tradeoff between Performance and Affordability Legacy solutions are restricted by monolithic controller architecture, making it impossible to completely isolate applications within shared storage. As a result, applications contend for the shared storage resources and due to this noisy neighbor effect, no application can be guaranteed performance. In short, to ensure predictable performance, applications require dedicated physical storage. The cost structure and complexity to host these dedicated storage islands is prohibitively high, especially with rapidly increasing number of applications. Datacenters are forced to choose between delivering predictable performance and balancing their cost structure. Current workarounds include overprovisioning shared storage resources, grouping similar workloads together, and dedicating storage to performance-sensitive applications. These are far from perfect and fail to either deliver predictable performance or optimally utilize resources. Management Nightmare With legacy solutions, provisioning performance (QoS) requires hardwiring of storage, which is just not scalable with today’s dynamic workloads. Fragmented management of multiple storage islands can be tedious and daunting even for datacenters with large management overheads. Further, within shared storage, legacy solutions are incapable of providing granular resource usage analytics, leading to complexities in identifying bottlenecks and billing customers (for a CSP). Several storage management tools have popped up over the last decade due to the sheer number of inefficiencies in the legacy systems. While they help alleviate the above-mentioned pain points, none of them address the core problems– eliminating the need for storage fragmentation or enabling on-demand provisioning. Security Storage assigned to a customer/application must be free from data snooping, unauthorized manipulation, and deletion. In many scenarios, data confidentiality maybe required even from the service provider or enterprise IT. With the lack of complete isolation and encrypted access, legacy solutions do not meet these standard security requirements. In the SAN world, LUN masking is used to some extent to separate an application’s storage at the switch level. However, this is neither comprehensive nor scalable. Service providers cannot provide comprehensive security without expanding their physical infrastructure. 4
- 5. CloudByte Technology: Patented TSMTM Architecture Legacy Architecture Legacy storage solutions are restricted by their monolithic controller, where applications share the same access layer, file system and disk subsystem. Further, physical resources of a controller such as CPU and memory are shared without any reference to an application’s performance needs. With the monolithic approach, it is impossible to guarantee QoS to applications within shared storage. CloudByte TSM Architecture The CloudByte storage controller is designed and built from the ground up for multi-tenancy. In a CloudByte controller, each application is fully isolated at all storage stack levels and unified under a Tenant Storage Machine (TSM). With completely isolated applications, controller resources allotted to each application can now be easily monitored, controlled, and provisioned. As we’ll see in detail in the next section, CloudByte also intelligently provisions controller resources to each application, based on the application’s QoS requirements (IOPS, latency and throughput). Further, each application’s data can now be optionally encrypted to provide additional security. Figure 1: Legacy Monolithic Architecture vs. ElastiStor TSM Architecture 5
- 6. CloudByte Technology: Heuristics Based Performance Control CloudByte Intelligence Dynamically Provisions Resources based on QoS Requirements For the first time ever, CloudByte delivers tailored QoS (IOPS, throughput, latency) to every application within shared storage. CloudByte software includes an intelligent heuristics daemon which continuously learns the quantity of various controller resources needed to deliver the required QoS and accordingly provisions them to each application. Controller resources needed to deliver a specific QoS are not static, but vary depending on factors such as the configuration of disks, the location of tenant data on disks, the amount of data in cache, and the amount of data that needs to be fetched from the disks. The equation continually changes and this pattern is learnt by CloudByte heuristics daemon. This pattern is then used to provision the resources of the storage controller such as CPU, memory, cache lengths in the file system, and network bandwidth. While the heuristics learning happens at the storage controller level, resource enforcement happens at the TSM level. Figure 2: Resource provisioning to achieve QoS requirements 6
- 7. CloudByte Deployment Architecture Figure 3: CloudByte deployment architecture Scale to Thousands of Applications An ElastiStor controller is built by deploying ElastiStor OS on industry-standard server. These controllers can be linearly scaled to form an ElastiStor cluster, with one ElastiStor controller dedicated as an administration node. With each controller supporting several applications/VMs, an ElastiStor cluster can scale to thousands of applications. With zero-proprietary hardware and OpenStorage, ElastiStor frees datacenters from any proprietary lock-in and large upfront investments. Further, infrastructure can be custom-built based on the datacenter demands, whether it’s SATA, SAS or SSD. Storage made Predictable, Affordable, and Easy For the first time ever, ElastiStor delivers guaranteed QoS to every application right from shared storage, eliminating the need for any dedicated storage islands. By un-fragmenting storage islands and optimally utilizing resources, ElastiStor steeply cuts down datacenters’ storage footprint, leading to 80-90% cost savings over 3-5 years. With its on-demand provisioning, ElastiStor breaks the need for hardwiring storage to deliver an application's performance. Further, ElastiStor makes storage management incredibly easy with vCenter-like administration, delegated administration and REST APIs (see “key features” for more details). 7
- 8. ElastiStor: Key Features QoS-Configurable Storage Endpoints Share your storage and deliver predictable performance to every application. For the first time ever, ElastiStor allows storage LUNs to be defined beyond capacity, in terms of IOPS, throughput and latency. This allows applications with diverse workloads to be guaranteed QoS right from a shared storage platform. Together with linear scaling, a single extensible shared storage platform from ElastiStor can now replace legacy solutions’ dedicated storage islands. By un-fragmenting storage islands and optimally utilizing resources, ElastiStor steeply cuts down your storage footprint, leading to 80-90% cost savings over 3-5 years. On-Demand Storage Provisioning Do you still manually configure hardware to provision storage for any new application? Break the need for hardwiring storage with ElastiStor's on-demand provisioning. Just enter the required SLA/QoS parameters and let ElastiStor automate node selection and resource allocation for you. ElastiStor includes an intelligent heuristics daemon which continuously learns the quantity of various controller resources needed to deliver the required QoS. vCenter-like Administration Console ElastiStor makes managing storage as easy as managing VMs, even when as you scale to hundreds of applications. Storage admins can now comprehen- sively manage the entire storage cluster, spanning across multiple sites, from a single web-based console. Further, ElastiStor gives you unprecedented ac- cess and control over resource usage within shared storage, right down to the application-level granularity. REST APIs and Plugins for Easy Integration Every action performed at ElastiStor admin console translates into a REST based API call in the backend, enabling admins to fully manage ElastiStor right from their existing portals. Our plugin for VMware vCenter (also based on REST API) enables storage management right from the vCenter, right from setting QoS policies to monitoring resource usage. N-way High Availability ElastiStor enables N-way High Availability, exponentially increasing reliability (mean time to failure), compared to the standard 2-way HA provided by existing solutions. CloudByte's storage un-fragmentation and its patented TSM architecture make N-way HA affordable and feasible. Delegated Administration A much requested feature from the cloud service providers, delegated administration empowers both CSPs and its customers to monitor and control storage, as necessary. Management privileges vary based on the admin functionality – for example, a super admin can manage the entire storage cluster, whereas a customer admin can manage just the storage resources allotted to that particular customer. 8
- 9. ElastiStor: Standard Storage Features Scalability 128-bit file system Zettabyte storage capacity Unlimited file size Access Protocols NFSv3, NFSv4, CIFS, iSCSI, FC Storage Connectivity SAS JBODS, iSCSI targets, FC targets Storage Resilience RAIDZ1, RAIDZ2 Storage Efficiency De-duplication Compression Thin provisioning Backup Efficient Snapshots, Unlimited Efficient Clones, Unlimited Tape Backup Availability N-way high availability (N-way HA) Partial failure transfer to the available node Active-Active mode HA with/without storage redundancy Disaster Recovery Tenant level disaster recovery High availability across primary and DR sites Block level replication Synchronous mirroring Asynchronous mirroring RPO—Last minute RTO—Few minutes Data Integrity Protection against silent data corruption Fixes corrupt block without having to take the file system offline CloudByte’s file system is built on ZFS and hence, it inherits all the standard ZFS features 9
- 10. multi-tenant storage Service Providers can now provide full benefits of dedicated storage to their customers on a shared storage platform For more information, visit www.cloudbyte.com or follow us on Twitter @CloudByteInc 10