Windows Azure SQL Database Federations
Download as PPTX, PDF1 like1,368 views
SQL Database Federations provide a sharding-as-a-service capability for elastically scaling out relational data in Windows Azure. A federation comprises a root database and federation member databases (shards) that store distributed data based on a distribution key range. Transact-SQL statements and dynamic management views provide capabilities for defining, managing, and querying across federations and their member databases.
Windows Azure SQL Database Federations
- 1. SQL Database Federations Scaling out relational data in Windows Azure NEIL MACKENZIE
- 2. Who Am I? Neil Mackenzie Azure Architect @ Satory Global Windows Azure MVP Blog: http://convective.wordpress.com/ Twitter: @mknz Book: Microsoft Windows Azure Development Cookbook
- 3. Content Windows Azure Platform Motivation for SQL Database Federations Overview of SQL Database Federations Transact-SQL Statements DMVs Fan-out queries
- 4. Windows Azure Cloud Services: Platform-as-a-Service Web roles, worker roles Infrastructure-as-a-Service Virtual Machines, SQL Server 2012 Connectivity Endpoints, Virtual Network, Service Bus Storage Windows Azure Storage Service Windows Azure SQL Database
- 5. SQL Database Database-as-a-Service Multi-tenanted Priced in GB/month Database One primary and two secondary databases Quorum commit Maximum size: 150GB Programmability TDS protocol SQL Server authentication Change connection string and go
- 6. Scalability Scale up Use more powerful hardware Price/performance problem Fails at internet scale Scale out Use commodity hardware Cost-effective way to increase performance Resilient towards failure
- 7. Scale-Out Data Use multiple databases (shards) Sharding provides: Larger data sizes Higher performance Issues: Data distribution Connection routing
- 8. SQL Database Federations Sharding-as-a-Service Federated database comprises: Root database Federation member databases (shards) Federation specified by: Federation name Distribution key Routing Connection pooling
- 9. Federation Members Federation member Federated data distributed by distribution key range Error to insert or update distribution key outside range Member databases can have: different sizes different schemas Distribution key in: Clustered index Each unique index
- 10. Table Types Federated tables In federation members CREATE TABLE ( … ) FEDERATED ON (CustomerId = custId) Reference tables In federation members CREATE TABLE ( … ) Common tables In root database
- 11. Federation Operations Transact SQL support for federations: CREATE FEDERATION USE FEDERATION ALTER FEDERATION DROP FEDERATION
- 12. CREATE FEDERATION CREATE FEDERATION federation_name (distribution_name <data_type> RANGE) Data types: INT BIGINT UNIQUEIDENTIFIER VARBINARY(n) - n<=900 Root database can support several federations e.g. – customer federation, product federation
- 13. USE FEDERATION USE FEDERATION ROOT WITH RESET Routes connection to root database USE FEDERATION federation_name (distribution_name = value) WITH RESET, FILTERING={ON|OFF} Routes connection to appropriate federation member FILTERING= OFF allows connection to be used for any data in the federation member FILTERING=ON restricts connection to a specified distribution key
- 14. ALTER FEDERATION: SPLIT Range: 100 400 500 ALTER FEDERATION CustomerFederation SPLIT AT (CustomerId = 200) Range: 100 200 400 500
- 15. ALTER FEDERATION: DROP 100 200 400 500 ALTER FEDERATION CustomerFederation DROP AT (LOW CustomerId = 200) 100 200 400 500 ALTER FEDERATION CustomerFederation DROP AT (HIGH CustomerId = 200) 100 200 400 500
- 16. Dynamic Management Views Definition e.g. sys.federation_member_distributions History e.g. sys.federation_member_distribution_history Operations e.g. sys.dm_federation_operations Operation errors e.g. sys.dm_federation_operation_errors
- 17. sys.federation_member_distributions Columns federation_id int member_id int (database id) Distribution_name sysname Range_low sqlvariant Range_high sqlvariant Range: includes Range_low excludes Range_high
- 18. Security Principal Root database CREATE USER user_name FROM LOGIN login_name Federation member CREATE USER user_name Security is otherwise like SQL Database
- 19. Fan-Out Queries Loop over: sys.federation_member_distributions.Range_low Perform queries in parallel Two-step process Member query Summary query (if necessary)
- 20. Summary SQL Database Federations Sharding-as-a-Service Elastic scalability for SQL Database databases Management support Developer support
- 21. More Information MSDN Wiki http://bit.ly/A7sUdo Cihan Biyikoglu blog: http://blogs.msdn.com/b/cbiyikoglu/ Post based on this presentation: http://bit.ly/wqD4Xo
Editor's Notes
- #7: Performance / database sizeNew pricing model complicates things since price is not linear in DB sizeFlavorous example
- #8: Data Distribution:Range partitioningRound robinHash bucketRouting:Database discoveryConnection managementFlavorusTo support the spike in first-day ticket sales, Flavorus used the Windows Azure platform to host 750 web role instances of the Jetstream application and 550 SQL Azure databases. The application wrote chunks of customer data to multiple sharded SQL Azure databases in parallel. Jetstream was live for two days, after which the company returned the event data to its on-premises servers and continued selling tickets.http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=4000011072
- #10: A range distribution can be affected by:Data skew – data concentrated in one federation memberOperational skew – data operations concentrated in one federation member
- #11: FederatedReferenceCommonIndividual members contain distinct schema
- #14: DB_NAME() provides the name of the current federation member.FEDERATION_FILTERING_VALUE(DistributionName) provided the current FILTERING=ON valuehttp://msdn.microsoft.com/en-us/library/windowsazure/hh597465.aspxFederation atomic unit is explicitly not a security boundaryError to insert/update data in wrong member
- #15: Only valid when connected to root databaseOrdering of uniqueidentity:00000000-0000-0000-0000-080000000000http://sqlblog.com/blogs/alberto_ferrari/archive/2007/08/31/how-are-guids-sorted-by-sql-server.aspx
- #17: Metadatasys.federationssys.federation_memberssys.federation_distributionssys.federation_member_distributionssys.federation_historysys.federation_member_historysys.federation_distribution_historysys.federation_member_distribution_historysys.dm_federation_operationssys.dm_federation_operation_memberssys.dm_federation_operation_errorssys.dm_federation_operation_error_members
- #20: LOW is in memberHIGH is NOT in memberIterate over LOW value for member distributions
- #22: SQL Azure Federation Data Migration Wizardhttp://sqlazurefedmw.codeplex.com/