Hochskalierbare, relationale Datenbanken in Microsoft Azure
1 like293 views
The document discusses scalable relational databases in Microsoft Azure, focusing on Azure SQL Database and Azure SQL Data Warehouse. It covers topics such as transactional processing, service tiers, performance insights, and querying multiple shards. Best practices for design and data loading in Azure SQL Data Warehouses are also provided, along with resource links for further information.
![Compute
Dist_DB_1
Dist_DB_2
Dist_DB_6
SQL DW - Architecture
Blob storage [WASB(S)]
Queries Control
Engine
DMS
SQL DB
DMS
SQL DB
…
Compute
Dist_DB_7
Dist_DB_8
Dist_DB_12
DMS
SQL DB
…
Compute
Dist_DB_13
Dist_DB_14
Dist_DB_18
DMS
SQL DB
… Compute
Dist_DB_19
Dist_DB_20
Dist_DB_24
DMS
SQL DB
…
Compute
Dist_DB_25
Dist_DB_26
Dist_DB_30
DMS
SQL DB
…
Compute
Dist_DB_31
Dist_DB32
Dist_DB_26
DMS
SQL DB
…
Compute
Dist_DB_37
Dist_DB_38
Dist_DB_42
DMS
SQL DB
…
Compute
Dist_DB_43
Dist_DB_44
Dist_DB_48
DMS
SQL DB
…
Compute
Dist_DB_49
Dist_DB_50
Dist_DB_54
DMS
SQL DB
…
Compute
Dist_DB_55
Dist_DB_56
Dist_DB_60
DMS
SQL DB
…](https://image.slidesharecdn.com/hochskalierbarerelationaledatenbankeninmicrosoftazure-170628120002/85/Hochskalierbare-relationale-Datenbanken-in-Microsoft-Azure-22-320.jpg)
Hochskalierbare, relationale Datenbanken in Microsoft Azure
- 1. Hochskalierbare, Relationale Datenbanken in Microsoft Azure Sascha Dittmann Data Solution Architect, Microsoft Twitter: @SaschaDittmann
- 2. Use Case
- 3. Microsoft Azure Data Services transactional processing rich query managed as a service elastic scale Internet accessible http/rest schema-free data model arbitrary data formats
- 4. Azure SQL Database The developer’s intelligent cloud database service
- 5. Demo Creating a Azure SQL DB
- 7. Azure SQL DB – Service Tiers
- 8. Database Transaction Units (DTU) Reads CPU Writes Memory Utilization 30% 75% 50% 75%
- 14. Performance Management Singleton databases Elastic databases Elastic Database Pool
- 16. Querying a Shard var customerId = 12345; var newPersonId = 4321; var connectionString = Configuration.GetCredentialsConnectionString(); using (SqlConnection conn = customerShardMap.OpenConnectionForKey(customerId, connectionString, ConnectionOptions.Validate)) { SqlCommand cmd = conn.CreateCommand(); cmd.CommandText = @"UPDATE Sales.Customer SET PersonID = @newPersonID WHERE CustomerID = @customerID"; cmd.Parameters.AddWithValue("@customerID", customerId); cmd.Parameters.AddWithValue("@newPersonID", newPersonId); cmd.ExecuteNonQuery(); }
- 17. Querying multiple Shards using (MultiShardConnection conn = new MultiShardConnection(myShardMap.GetShards(), myShardConnectionString)) { using (MultiShardCommand cmd = conn.CreateCommand()) { cmd.CommandText = "SELECT c1, c2, c3 FROM ShardedTable"; cmd.CommandType = CommandType.Text; cmd.ExecutionOptions = MultiShardExecutionOptions.IncludeShardNameColumn; cmd.ExecutionPolicy = MultiShardExecutionPolicy.PartialResults; using (MultiShardDataReader sdr = cmd.ExecuteReader()) { while (sdr.Read()) { var c1Field = sdr.GetString(0); var c2Field = sdr.GetFieldValue<int>(1); var c3Field = sdr.GetFieldValue<Int64>(2); } } } }
- 18. Demo Azure SQL Elastic Pools
- 19. Do we need a Data Warehouse?
- 20. Azure SQL Data Warehouse
- 21. Compute Dist_DB_1 Dist_DB_2 Dist_DB_12 SQL DW - Architecture SSD storage Queries Control Engine DMS SQL DB DMS SQL DB … Compute Dist_DB_13 Dist_DB_14 Dist_DB_24 DMS SQL DB … Compute Dist_DB_25 Dist_DB_26 Dist_DB_36 DMS SQL DB … Compute Dist_DB_37 Dist_DB_38 Dist_DB_48 DMS SQL DB … Compute Dist_DB_49 Dist_DB_50 Dist_DB_60 DMS SQL DB …
- 22. Compute Dist_DB_1 Dist_DB_2 Dist_DB_6 SQL DW - Architecture Blob storage [WASB(S)] Queries Control Engine DMS SQL DB DMS SQL DB … Compute Dist_DB_7 Dist_DB_8 Dist_DB_12 DMS SQL DB … Compute Dist_DB_13 Dist_DB_14 Dist_DB_18 DMS SQL DB … Compute Dist_DB_19 Dist_DB_20 Dist_DB_24 DMS SQL DB … Compute Dist_DB_25 Dist_DB_26 Dist_DB_30 DMS SQL DB … Compute Dist_DB_31 Dist_DB32 Dist_DB_26 DMS SQL DB … Compute Dist_DB_37 Dist_DB_38 Dist_DB_42 DMS SQL DB … Compute Dist_DB_43 Dist_DB_44 Dist_DB_48 DMS SQL DB … Compute Dist_DB_49 Dist_DB_50 Dist_DB_54 DMS SQL DB … Compute Dist_DB_55 Dist_DB_56 Dist_DB_60 DMS SQL DB …
- 24. Table Distribution Options Replicated (Future improvement) Round Robin (Default) Data distributed evenly across nodes Easy place to start, don’t need to know anything about the data Simplicity at a cost Data repeated on every node Simplifies many query plans and reduces data movement Best with joining hash table Hash Distributed Data divided across nodes based on hashing algorithm Same value will always hash to same distribution Single column only Check for Data Skew, NULLS, -1 Will incur more data movement at query time Consumes more space Joining two Replicated Table runs on one node
- 25. Best Practices • Designed for DW and not OLTP • Do not use singleton DML heavy operations • Load Data in „batch mode“ • PolyBase • BCP, SQL BulkCopy, etc. • Azure Data Factory • SQL Server Integration Services • …
- 26. PolyBase Import Readers / Writers DWU Max External Readers Max Writers DW100 8 60 DW200 16 60 DW300 24 60 DW400 32 60 DW500 40 60 DW600 48 60 DW1000 80 60 DW1200 96 60 DW1500 120 120 DW2000 160 120 DW3000 240 240 DW6000 480 480
- 28. Azure SQL Database https://docs.microsoft.com/en-us/azure/sql-database/ http://dtucalculator.azurewebsites.net/ Azure SQL Database https://docs.microsoft.com/en-us/azure/sql-database/sql-database-elastic-scale-introduction https://docs.microsoft.com/en-us/azure/sql-database/sql-database-wtp-overview Azure SQL Data Warehouse https://docs.microsoft.com/en-us/azure/sql-data-warehouse/ Resources