Multi Tenancy In The Cloud

Editor's Notes

• #2: This definition states that Clouds have five essential characteristics: on-demand self-service(real time), broad network access, resource pooling, rapid elasticity, and measured service.This definition by default assumes the Cloud services as Infrastructure as a Service (IaaS). IaaS could be very successful in development and testing environment, because those public Cloud offering cheap and instant infrastructure services could ease the reach to IT resources.
• #4: Multi-tenancyallows a single application to emulate multiple application instances.Enterprise IT infrastructure incurs many costs ranging from hardware costs and software licenses/maintenancecosts to the costs of monitoring, managing, and maintaining IT infrastructure.Tenant could be an Application owner. But from a cloud providers perspective, the application owner could be a cloud provider like SalesForce.comin the cloud the data is the only thing that matters: for SaaS, this is the data stored within the Cloud Application, for PaaS that could be the application written by the Tenant and the associated data, and for IaaS this will be the virtual machines, virtual networks, and other bits below the application. Our definition of Tenant must handle all these cases.The Tenant is the Legal Owner of the data that resides within the system.SMT ( Secure Multi Tenancy) for a cloudMulti Tenancy and PaaS:PaaS Generation I : Simply offer a Physical Machine or a Cluster as per user’s needPaas Generation II : Use OS level Virtualization to create Virtual Machines per customer that share a single server or a cluster of real servers. Amazon Relational data service, provides elasticity as wellPaas Generation III: Deploy multi tenancy, essentially virtualize the middleware servers and allow a single instance to be securely shared among tenants or different customers. IaaS : The definition presented in the first slide, by defaut assumes the Cloud Services as IaaS.The AppFabric Container provides base-level application infrastructure such as automatically ensuring scale out, availability, multi-tenancy and sandboxing of your application components.(Microsoft, Windows Azure)Cloud-enabling infrastructure to allow secure multi-tenant deployments, including fully integrated management, monitoring, metering and billing infrastructure (CloudBees)If you are running numerous applications/application instances, XAP’s fine-grained multi-tenancy allows you to share them across all available machines, instead of running only one instance per machine. This allows you to support more users on each machine. (GigaSpaces)Amazon’s EC2 is also a multi-tenant platformMulti-tenant components:  application code, operating system, data storage software, and computing resources.
• #6: Multi-tenancy can be applied at the database layer of a hosted service, where it can be very effective due to the high cost of provisioning and operating databases.multi-tenant databases are essential for hosted services to manage high traffic volumes at low cost.The administrative framework of a multi-tenant database should maintain metadata about customers, such astheir contact information, their location in the farm, and the features they are allowed to access. Certainadministrative operations will need to access this metadata alongside of customer data. For example, a serviceprovider might want to scan the metadata to find all customers in a given region and then determine which ofthose customers have more than a certain amount of data. To support such operations, the administrativeframework should offer a unified query language that obviates the need for a general-purpose programminglanguage with embedded queries. In addition to being easier to use, this approach makes it simpler to executeadministrative operations in bulk on individual databases in the farm.
• #7: Advantages: 1. Veryeasy to implement asit does not require modifying the implementation of the database.2. In addition, it does not substantially reduce customer isolation, particularly if each process is placed in its own virtual machine.3. Metadata associates each database with the correct tenant, and database security prevents any tenant from accidentally or maliciously accessing other tenants' data.4. Giving each tenant its own database process makes it easy to extend the application's data model to meet tenants' individual needs, and restoring a tenant's data from backups in the event of a failure is a relatively simple procedure.5. Restoring a single tenant's data means simply restoring the database from the most recent backup.6. Customer migration is straightforward, just move files from one server to another.Disadvantages:1. This approach tends to lead to higher costs for maintaining equipment and backing up tenant data. Hardware costs are also higher than they are under alternative approaches, as the number of tenants that can be housed on a given database server is limited by the number of databases that the server can support.2. Because isolation between customers is strong in this approach, executing administrative operations in bulk is not feasible: each database will execute queries on its own.3. providing seamless access to shared data requires a mechanism to access it remotely, otherwise it will end up being replicated multiple times on the same server.4.limitation of this approach is that each database requires its own connection pool on each application server, so sockets will not be shared among customers. To improve this situation, the implementation could be modified to share memory and sockets among co-located database processes, possibly at a level below those processes.Example, customers in fields such as banking or medical records managementAn example of this approach is CasJobs, which supports analysis of data in the Sloan DigitalSky Survey database
• #8: Advantages:1. it is useful if each customer gets their own physical table space so that customer migration entails simply moving files from one server to Another.2. it allows the system administrator to balance the I/O load by distributing customers across different backing disks.3. This approach is considerably better at pooling memory.4. Since there is only one database in this approach, customers can share connection pools.There is a well-known downside to connection pooling however: all connections must be associated with a fixed principal who can access everything. This means that both security and the management of resource contention have to be handled at the application layer. Thus errors in the application code could allow one customer to access another customer’s tables or prevent them from getting their fair share of resources. This problem could be mitigated by allowing the principal associated with a database connection to be picked up from the application server. Support for this feature is beginning to appear in databases today.5. This approach should easily scale up to thousands of active customers per server, a two orders of magnitude improvement over the shared machine approach.Disadvantages:1. Tenant data is harder to restore in the event of a failure. 2.With a separate-table application, restoring the entire database would mean overwriting the data of every tenant on the same database with backup data, regardless of whether each one has experienced any loss or not. Therefore, to restore a single customer's data, the database administrator may have to restore the database to a temporary server, and then import the customer's tables into the production server—a complicated and potentially time-consuming task.
• #9: Every application query is expected to specify a single value for theTenantIDTo allow customers to extend the base schema, each table is given a fixed set of additional generic columns. These columns might be of type VARCHAR,Adv:1.This approach is clearly the best at pooling resources.2. Its ability to scale up is limited only by the number of rows the database can hold, which should offer several orders of magnitude improvement over the shared process approach.3. Administrative operations can be executed in bulk simply by executing queries that range over the TenantId column.4. since there is only one database, customers can share connection pools. DisAdv:since files on the disk have intermingled data from multiple customers, migration requires executing queries against the operational systemsuch intermingling can impact the performance of accessing a customer’s data, since it may be spread out across many pagessecurity can be pushed down into the database only if different access privileges can be assigned to different rows in the same table.the use of generic columns is feasible only if the database has a compact representation for sparse tablesif typing of the generic columns has been abandoned, it will be hard to use column-oriented features such as indexes and integrity constraintsqueries intended for a single customer have to contend with data from all customers, which compromises query optimization.
• #13: A service is a well-defined, self-contained unit of functionality, independent of the state of other services.Concept of SOA comes into picture.Integration between two or more applications running on different platforms would require a middle component-dependent intercommunication layerservices can be used to create distributed applications, integrate component-based applications, and make them communicate with each otherTypically services reside on different machines. They are exposed to the outside world by service interfaces. A service provider provides its functionality using the service interfaces that are used or consumed by the service consumers. A service consumer sends a service request to a service interface and receives a service response.Software as a Service (SaaS) is a model of software deployment whereby a provider licenses an application tocustomers for use as a service on demand.“On-demand” licensing and use alleviates the customer's burden of equipping a device with everyconceivable application. It also reduces traditional End User License Agreement (EULA) software maintenance, ongoingoperation patches, and patch support complexity in an organization.SaaS – old concept, gained momentum in around 2001
• #14: In the hosted service a service provider develops an application and operates the system that hosts it. Customers access the application over the Internet using industry-standard web browsers or Web Services clients.Evolution: Started with the Application Service ProviderAdv:1. reduce the total cost of ownership of an application by aggregating customers together and leveraging economy of scale.2. This principle applies to both capital expenditures, e.g., for hardware and software, and operational expenditures, e.g., for bandwidth and personnel. Because a hosted service is focused on one application, the infrastructure and the procedures for managing it can be highly optimized:Multi-tenancy is an optimization for hosted services in which multiple customers are consolidated onto the same operational system, a technique pioneered by salesforce.com.Multi-tenancy allows pooling of resources, which improves utilization by eliminating the need to provision each customer for their maximum load.Multi-tenancy can also improve management efficiencies by providing a uniform framework for administering the systemA multi-tenant system should support both scale up and scale out, scale up by consolidating multiple customers onto the same server and scale out by having the administrative framework span a farm of such servers. Scale out is required because it is not cost effective to scale up a single server IndefinitelyThe administrative framework of a multi-tenant system should support the ability to migrate a customer from one server to another within the farm.For example, a customer might start out on a server that manages trial accounts, be moved to a server that manages small production accounts, and grow until it is moved to a dedicated server that manages only one account.Migration should also be possible between farms, for example, to allow customers to be moved from one data center to another.The administrative framework should also support rolling upgrade, where the servers in a farm are upgraded to a new version of the application one at a time
• #15: Multi-tenant is appropriate for business tasks that are common across many customers or divisions of a company.Multi-tenant architectures for ERP
• #17: Virtualization: ability to run a high number of virtual machines than what is physically available as hardware servers.H/W Virtualization deals with creation of Virtual Machines.S/W Virtualization: Operating system-level virtualization, a type of server virtualization technology which works at the OS layer. The physical server & single instance of the operating system is virtualized into multiple isolated partitions where each partition replicates a real server. The OS Kernel will run a single Operating System and provide that Operating System functionality to each of the partitions.Operating system-level virtualization is a server virtualization method where the kernel of an operating system allows for multiple isolated user-space instances, instead of just one. These instances are called containers. So, the software installed needs to be complaint with the Operating System. This leads to a loss in Operating System flexibility. This is an acceptable tradeoff if you consider Web Applications only.H/W Virtualization: serious limitations when it comes to scaling to large numbers of concurrent virtual machinesThis results in a higher per virtual machine instance cost compared to a traditional shared hosting solution.
• #18: Hardware based virtualization has a serious limitation when it comes to scaling of large number of concurrent virtual machines running on a given physical server. This results in a higher per virtual machine instance cost compared to a traditional shared hosting solution. There are lots of opinions running around about Virtualization – some say that it is a part of Cloud Computing, some say that it is not important and adds extra complexity to cloud computing , vendors say that Virtualization is inherently a cloud infrastructure, some even say that Virtualization & cloud computing are one and the same.
• #19: Next generation multi-tenant virtualization cloud computing platform - they call it Uranus
• #20: Uranus focuses on increased scalability and security and remaining flexible by not requiring applications to be modified in any special way in order to support multi tenancy.Used for web applications. The platform is built on popular Linux, Mysql, Apache, PHP with the aim of supporting around 1000 existing LAMP based applications.LAMP is an acronym for a solution stack of free, open source software, referring to the first letters of Linux (operating system), Apache HTTP Server, MySQL (database software) and PHP (or sometimes Perl or Python), principal components to build a viable general purpose web server.Another surprising thing about a typical LAMP server (for example one you might rent in the cloud or as a dedicated server) is that they do not actually have the AMP components installed (only the L, that is, the Linux is actually there)LAMP based web sites or applications appear to be thousands of pages but these pages are actually assembled at the moment of request from just a few templates. When you click the "Find" button on Ebay, for example, the web server execute program code that does the lookup by calling the SQL server and assembles a page showing the items it found. Likewise, when one of these links is clicked the web server executes program code to lookup the details of the item. In a LAMP system the SQL database product employed is MySQL and the program code woven into the pages is written in a language named PHP (more information below).The SQL server, like Apache, runs all the time waiting for database requests. However unlike Apache, SQL only accepts SQL requests from programs written in a language like PHP. When a web page contains a PHP program the PHP interpreter is launched on the server and it interprets the PHP instructions on the page and carries them out. It is thus the PHP program code that interacts with MySQL and ApacheThousands of new companies have sprung up to provide services to those adopting LAMP, making it easier, more scalable, cheaper, fault tolerant, etcCloud computing is an example, now with an account at a cloud provider like Amazon, Rackspace you can create a virtual server and install Linux (with a choice of a dozen different distributions) in less than a minute. Your virtual server can be connected to a URL, backed up, templated to create others and removed quickly. And these virtual servers scale to meet traffic demands, have load balancing and integrate with specialized database servers.Reverse Proxy Layer: This is where the users HTTP request first encounters the system. This layer consists of one or more reverse proxy servers that route the users HTTP Calls to the corresponding back end LAMP server found in the virtualization layer.These servers are implemented using Apache Web server running in a reverse proxy mode.
• #22: The ProxyPass Port: 9101 is the port of the Apache server within the Virtualization layer to which the HTTP request is being handed over toa reverse proxy is a type of proxy server that retrieves resources on behalf of a client from one or more servers. These resources are then returned to the client as though it originated from the reverse proxy itself
• #24: Each tenant exists as a separate system user with a dedicated home directory which is shared by the tenants own Apache & Mysql processes.This is unlike H/W virtualization where each VM has a complete copy of an OS image including all applications. In this implementation they share the systems application files.They don’t share the same application process but they share the physical executable binary files and therefore benefit from being able to update just one system and have all the virtual systems take advantage of it as opposed to having to update several VM’s separately.Process & Data isolation The dedicated home directory of the tenant is accessible only to that specific tenant’s processes and all data such as websites, databases , log files are completely isolated in a sand box like environment. Thus Uranus provides Process & Data isolation for a high level of security.
• #25: Apache Virtualization:Each Apache process of a tenant runs on its own unprivileged port as the tenant user. This gives process isolation. Thus each process instance can be customized based on the required resources as well as being able to prioritize its CPU allocation in a much more granular manner.The system also provides PHP level customization using a custom php.ini file for each tenant.Mysql Virtualization:Usedmysqld_multi tool for launching multiple mysql instances.Functionally similar to Apache in the sense that a separate Mysql process is started for each tenant running on its own mysql port and having its own database.Mysql process is started using a common system user called mysql and requires a common database user account and password within each tenants mysql database.Ex: user = multi_adminPassword = aim3Mer6This common database account is required by the mysqld_multi script to have control over shutting down the database instance.Does sound like a security risk, however this database account does not have any permissions other than to shut down the database server. So it cannot be used to access the tenants application data. Its not absolutely required, but it offers convenience to managing Mysql instances centrally & can be shut off.
• #26: a sandbox is a security mechanism for separating running programs. It is often used to execute untested code, or untrusted programs from unverified third-parties, suppliers, untrusted users and untrusted websitesa sandbox is a security mechanism for separating running programs. It is often used to execute untested code, or untrusted programs from unverified third-parties, suppliers, untrusted users and untrusted websites
• #33: http://www.ibm.com/developerworks/webservices/library/ws-multitenantpart2/index.htmlFoundation of SaaS multi-tenancyIsolation & customization of web applicationsSeparation of concern of diff rolesIntroduction of Multi-tenancy should not bother other roles.Ex: A developer does not need ot learn multi-tenancy but should focus on the applicationMT should be like a plugin, the middleware provider would like to have an add on without maintaining two editions.The SaaS cloud platform operator would like to minimize the effort of deploying and provisioning the multi-tenant application without additional deployment and administration effort.
• #35: Web Application Server has a Servlet Container as well as an EJB containerAt the Application Server tier the tenant content information will be carried by a thread and be used to route to tenant specific application server classes throughout the life cycle of the thread.
• #36: Isolation & Customization are the foundations of SaaS Multi-tenancy
• #37: Security Isolation:one also needs to consider additional potential security risk introduced by other tenants who share the same application instance and resourcesPerformance Isolation:The tenants as well as the system administrators are naturally concerned about how to prevent one tenant’s behaviors from affecting the performance of other tenants. In addition, the tenants have the rights to receive the service levels that they have paid for.Availability Isolation:When many tenants share the same application instance, it is critical for the environment to detect any faults and stop the propagation of the faults at the earliest possible time.
• #48: General Application model that can be applied to all Web based modelsParameters for the AppNodeParameters for the DBNodeParameters for the Link
• #49: A topology model is an aggregation of definition of all nodes that make up the cluster built with virtual images.
• #50: After the Topology file is parsed the ImageID and Image location as well as parts of the information will be derived. The files will be retrieved from storage and will be installed on the base image.
• #51: An offering management application may cover many functions such as creating a new offering, updating an offering, etc. When creating a new offering, some compulsory information must be filled or selected such as the name of the offering, types of SLA this offering supports, whether this offering needs to be activated immediately, entry point of provisioning service (to be used to make the tenants onboarding).
• #52: The contents shown in the UI in Fig. 10 vary from applications to applications. The contents will be automatically generated based on the metadata of the multi-tenant application which is derived from the application transformation phase
• #56: Multi tenancy:Every tenant’s monitoring information flows through the same stream management system, but the information is only to be seen by its intended recipient(s). The system therefore has to make sure that information about the intended recipient(s) is transferred with the messages and that messages are appropriately filtered before being passed to user applications. Isolation in terms of tenant visibility is thereby achieved.Such filtering is performed by the message propagation system before delivering messages to their recipients. The data format as described before contains a field that denotes the intended recipients. This field is stripped from the notification before delivery to the recipients in order to avoid recipients being able to learn about other recipients of a particular message. The handling of tenant information is therefore completely shielded from the tenants, making the multi-tenancy property invisible to the tenants.
• #57: A main building block of the architecture is a generic event notification propagation system, called a “message bus”.Filtering and aggregation engine: As the name suggests, it filters event notifications based on certain criteria and in particular aggregates notifications.The aggregation facility is mostly used to aggregate streams of individual notifications over time, meaning that it creates coarse-grained streams of notifications from finer grained ones. For example, it can be used to create notifications of CPU utilization with a granularity of one minute from a stream of CPU utilization measurements with the granularity of one second.Similarly, it is used to aggregate streams of measurements of the same type originating from different agents into one aggregated stream. For example, CPU utilization measurements from different servers can be accumulated into one stream expressing the aggregated CPU utilization of the server farm.Every monitorable resource in the data center has one or more agents attached to it. The agents are the source of all monitoring information.They measure certain parameters and periodically or on-demand pass this information on to the message bus. Once on the message bus, the notifications are filtered and aggregated and, depending on the configuration, they are propagated to specific monitoring and control applications and are stored for later use in a data store. The monitoring and control applications are external to the monitoring infrastructure and are supplied and operated by either the data center operator (SaaS Operator) or the tenants.Both agents and monitoring applications are decoupled from the message bus by an adaptation layer – adapterit abstracts from the actual core monitoring system implementation and thereby insulates the agents and applications from the particular implementation technology used for the message bus and aggregation engine.Agents announce their capabilities and the resources they monitor to the configuration system. The information about the agents is stored in a Configuration Management Database (CMDB). Applications can query the CMDB, e.g., to find out which agents are available for a certain tenant. the agents have no information about their own location or about the tenant(s) they are associated with.The monitoring system, on the other hand, needs to associate each data message with the sending agent as well as with the corresponding tenants.Longer-term storage of events (e.g. longer than one hour) happens in a data store, which can be a regular data base or a cloud-like storage infrastructure, such as BigTable.
• #59: The SLA for this application requires that a request to the application needs to be returned within 2 secondsAt a certain snapshot, suppose15 requests are sent the application per second. <Case 1>:Twice as many intermediate requests are generated for B as C: RB= 2 Rc. After A, 10 RB are sent to B, and 5 RC are sent to C, therefore, the application can return these requests within 2 seconds successfully. The application is at the optimal condition, since all of its components are working at the maximal capacity. <Case 2>: if twice as many intermediate request are generated for C as B: RC= 2 RB. After A, 10 RC are sent to C, 5 RB are sent to B. All the requests to B can be processed in one second. Because C’s throughput is 5, it takes 2 seconds to process these 10 intermediate requests, or it can only process 5 of them in 1 second. Therefore, only 10 RAPP are successfully returned in 2 seconds. In this situation, B is under loaded, and C is overloaded.
• #65: Access Control:implicit filter: when one tenant requests to access shared resources, a common platform level account is delegated to handle this request. The delegated account is shared by all tenants and has the privileges to access resources of all tenants. explicit permission : In this pattern, access privileges for the resources have been explicitly pre-assigned to the corresponding tenant accounts by using the Access Control List (ACL) mechanism. In essence, how can I protect my data from those who are not allowed access to the data. In other words: a trust no one model specifically the cloud administrator(s).a level of auditing and monitoring of  administrator actions, to see if they have touched the data. Auditing is after the fact, what we need here is preventionExample:The current SMT design from Cisco-VMware-NetApp (CVN) looks to be all about availability of storage IO and ignores Data Integrity and Confidentiality. Cisco even claimed “we did not think about security” when designing the initial solution. Cisco is worried about Quality of Service, I.E. Bandwidth through out the system to the disk. Furthermore
• #66: the goal of tenant oriented fault isolation is to reduce X/N, the ratio of fault propagation among tenants, as much as possible.