Sql azure dec_2010 Lynn & Ike

Editor's Notes

• #2: Lynn then Ike
• #3: Lynn
• #4: LynnMany solutions will use a hybrid (i.e. mixed) approach
• #5: LynnWe have included this feature comparison table in anticipation of your likely questions about differences between using a relational database table as you may be currently doing with your SQL Server databases and the new Windows Azure Tables included in Windows Azure.
• #6: LynnCode Far works with any type of code – i.e. Java, etc.. Does NOT have to be Azure or .NET
• #7: IkeSLA InformationSQL Azure customers will have connectivity between the database and our Internet gateway. SQL Azure will maintain a “Monthly Availability” of 99.9% during a calendar month. “\\Monthly Availability Percentage” for a specific customer database is the ratio of the time the database was available to customer to the total time in a month. Time is measured in 5-minute intervals in a 30-day monthly cycle. Availability is always calculated for a full month. An interval is marked as unavailable if the customer’s attempts to connect to a database are rejected by the SQL Azure gateway.Download SQL Azure SLA.
• #8: Ikehttp://social.technet.microsoft.com/wiki/contents/articles/inside-sql-azure.aspx
• #9: Ike
• #10: Lynnhttp://www.microsoft.com/windowsazure/evidence/
• #11: Lynn talks, Ike types
• #12: Lynn talks, Ike types
• #13: Lynn
• #14: IkeSample code to handle re-try attempts for dropped connections - http://blogs.msdn.com/b/sqlazure/archive/2010/05/11/10011247.aspx
• #15: IkeIkeSteve Hale (Microsoft) authored this article at http://blogs.msdn.com/sqlnativeclient/archive/2010/02/12/using-sql-server-client-apis-with-sql-azure-vversion-1-0.aspxAlso updated MSDN Dev Guide 'How to connect' http://msdn.microsoft.com/en-us/library/ee621787.aspxUsing SQL Server Client APIs with SQL Azure Version 1.0 This post describes some of the issues that the application developer should be aware of when writing ODBC, ADO.NET SQLClient, BCP, and PHP code that connects to SQL Azure. See also SQL Azure documentation on MSDN (http://msdn.microsoft.com/en-us/library/ee336279.aspx). Supported components1. Microsoft SQL Server Native Client 10.0 ODBC Driver or newer. This is a free download.2. Microsoft PHP 1.1 driver or newer. Again a free download.3. SQL Server 2008 BCP.EXE or newer. The caution here is that the queryout option is not fully supported for SQL Server 2008 RTM. Queryout is expected to be fully supported in a future release of SQL Server.4. ADO.NET 3.5 SqlClient or newer.Any other components are NOT currently supported for connecting to SQL Azure. In particular it should be noted that OLEDB connections are not currently supported from any OLEDB provider, even SQL Server 2008 Native Client OLEDB provider. ODBC connections using the on-the-box ODBC driver “SQL Server” are not currently supported.Connecting1. valid server name, user ID, and password for SQL Azure2. On the SQL Azure portal, set the firewall rules to allow your client IP address to connect to SQL Azure. Additionally, in some corporate environments, you might need to install Microsoft Firewall Client for ISA Server on the client machines or connections will not succeed. And in those environments, using IIS might require the default IIS settings for Application Pool Identity to be changed. Here is how that setting is changed: Open IIS Manager in Admin Tools, select Application Pools, Pick the pool, Advanced Setting and under Process Model change Identity from BuiltInAccount\\ApplicationPoolIdentity to Custom account domain account. Restart IIS.SQL Azure supports only TCP/IP connections using port 1433 (default for most SQL Server APIs) if other protocol choices (named pipes or using other ports) Additionally, most client APIs do not send the server name when connecting, thus the server name should be included as part of the user ID. Currently the only exception to this is .NET 4.0 which does send the server name. ODBC, BCP, PHP, and .NET 3.5 all require the server name to be sent as part of the user ID. This may change in a future release. Connection strings are all similar for each API stack and require the server name, user ID, password, and optionally the database. Trusted connections (also known as Integrated Security or Windows Authentication) are not supported. MARS is not yet supported by SQL Azure so make sure MARS is not enabled in the connection string.Creating and Connecting to a DatabaseSQLCmd example: sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword>Note the required server name (in yellow) included as part of the user ID.In the above example the database was not specified. In that case the default database “master” is used. You must connect to the master database to create databases or drop databases. Since the “USE database” command is not supported, you must disconnect after creating a database and reconnect directly to the user database. In this example we connect directly to the Northwind database.sqlcmd -S o8uyxb64fe.ctp.database.windows.net -U <MyUserID>@o8uyxb64fe -P <MyPassword> -d NorthwindCreating tablesTable creation is restricted to user databases; you cannot create tables in the master database. Table creation is substantially the same as on SQL Server, with the exception that certain data types are not supported, for example CLR User Defined Types, Filestream, and row guid types. Also, it should be mentioned that SQL Azure requires a clustered index or primary key (which uses a clustered index by default) before data can be inserted into a table. For this reason a clustered index or primary key must always be created for any table intended to contain data.Example:Create table employees(EmployeeIDint primary key, EmployeeNamevarchar(30))OrCreate table employees(EmployeeIDint, EmployeeNamevarchar(30))Create clustered index empIndex on employees(EmployeeID) Unsupported features· “USE database” is not supported· 3-part names are not supported unless the database being referenced is the current database.· Creating or dropping a database when not in the master database· Insertion of data into a table without a clustered index.· MARS connections· Trusted connections· CLR UDTs· XML schema collections· Distributed transactions (local transactions are supported)· AttachDB· Certain stored procedures, for example sp_who and undocumented stored procedures used to support OLEDB schema rowsets. Deprecated functionalitySome features are deprecated in SQL Server and should not be used in new application development. In most cases SQL Azure does not support deprecated SQL Server features. Some examples of these are:· System tables such as sysobjects, sysusers, or sysdatabases. Use the new system views instead. See http://msdn.microsoft.com/en-us/library/ms187997.aspx for a mapping to the new system views.· Numbered stored procedures· “select into” syntax· “COMPUTE BY” clause· Two-part drop index statements See http://msdn.microsoft.com/en-us/library/ee336281.aspx for more detailed Transact-SQL support.Connection behaviorSQL Azure connections will behave somewhat differently than SQL Server connections. They can be dropped for a number of reasons:· Excessive resource usage· Long-running queries· Long-running single transactions· Idle connections· Failover due to server failuresFor this reason it’s best to avoid caching connections for later use. A better strategy is to open the connection when needed, use it, and then close it. Additionally, it is recommended that applications provide retry/reconnect logic to recover from dropped connections.LatencyLatency may be a concern when using SQL Azure unless the SQL Azure server is located near the application (“code near” scenario), such as with Windows Azure and a co-located SQL Azure. Making many round trips to the server can introduce noticeable delay in an application that previously performed adequately when running against a SQL Server on the intranet. Ensure the application connects to the nearest SQL Azure server and that round trips are minimized. SUMMARYü Use only supported client components to connectü Connect using TCP/IP and port 1433ü Use SQL Authentication only, not Trusted Connections/Integrated Authentication/Windows Authentication.ü Specify the server name in the user ID.ü Ensure the firewall rules in the SQL Azure portal permit your IP address.ü Install Microsoft Firewall Client for ISA Server if needed.ü Change the IIS Application Pool Identity if needed.ü Make sure MARS is not enabled.ü Connect to the master database to create or drop databasesü Create a user database for your application since tables cannot be created in masterü Disconnect and reconnect to change databasesü Create a clustered index or primary key on a table before inserting data.ü Remove any unsupported or deprecated features from the application.ü Avoid caching connections. Open the connection, use it, and then close it.ü Avoid making numerous round trips to the server.
• #16: Ike talk, Lynn typeSample Connection StringServer=tcp:server.ctp.database.windows.net;Database=<databasename>;User ID=user@server;Password=password;Trusted_Connection=False;
• #17: Ike talk, Lynn type
• #18: Ike talking, Lynn is typing
• #19: Ike talking
• #20: Lynn talkingFan out queries: where a query can be run against every container in parallel and the results returned and a union-all performed at the client.  This is an important scenario for us and we are looking at how we can make this pattern run efficientlyAuto partitioning. Multi-spindle storage solution. 10 drives look like 1 logical drive.“Sharding” Utility written with .NET 4.0 TPL at http://enzosqlshard.codeplex.com/
• #21: LynnClustered indexes sort and store the data rows in the table based on their key values. There can only be one clustered index per table, because the data rows themselves can only be sorted in one order. For more information about clustered index architectureWith few exceptions, every table should have a clustered index defined on the column, or columns, that offer the following: Can be used for frequently used queries. Provide a high degree of uniqueness. Can be used in range queries.Distributed Query – reference multiple linked servers and can perform either updae or read operations against each individual linked serverDistributed transactions span two or more servers known as resource managers. The management of the transaction must be coordinated between the resource managers by a server component called a transaction manager.DDL – data definition language statement to create drop or alter mining models and multidimensional objects such as cubes and dimensions
• #22: Lynn
• #23: LynnDatabase files can be grouped together in filegroups for allocation and administration purposes. Some systems can improve their performance by controlling the placement of data and indexes onto specific disk drives. Filegroups can aid this process. The system administrator can create filegroups for each disk drive, then assign specific tables, indexes, or the text, ntext, or image data from a table, to specific filegroups.
• #24: Lynn talks, Ike typesUnder “Set Scripting Options”, click Advanced and set “Script for database engine type” to SQL Azure DatabaseFor BCPScenario - BCP / Bulk CopyHigh speed programmable data import and export Best Practices Optimize Databases for Data ImportDisable/Delay creation of none Clustered IndexesOrder Data on the Clustered IndexDisable Triggers and Constraint checking-N Native mode so no data type conversion needed. -c Character mode for interoperability-b batchsize will let you control transaction size-h”TABLOCK, ORDER(…)” optimize writesFor SSISSSIS Design Surface - Data Flow TaskDiverse Source and destinationsTo/From Flat Files, ADO.Net, OleDBFully programmable flow Loops, Sorts, Conditional operators, XML/WebServices Processing etcVS Debugging support with data viewers, watches and conditional breakpointsBest Practices – Data Flow TaskRemember; Optimize Databases for Data ImportDisable/Delay creation of none Clustered IndexesOrder Data on the Clustered IndexDisable Triggers and Constraint checkingBatch Size: Adjust the transaction sizeBuffer and Blob temp storage area; spread over to multiple drives for large data transfersParallelization – based on execution trees, task will auto parallelize
• #26: SQL Azure firewall prevents all access to a SQL Azure server. The IP addresses of the machines that need access need to be granted access before they can attempt to connect to SQL Azure server. This can be done via the Firewall Settings tab under the Server Administration page on the SQL Azure portal. There is also a checkbox “Allow Microsoft Services access to this server”. Checking this will allow connections coming from your applications hosted in Windows Azure. The article How to Configure SQL Azure Firewall has more detailed information and instructions on configuring the firewall.http://msdn.microsoft.com/en-us/library/ee621782.aspxThe SQL Azure Database service is only available with TCP port 1433After you have used the SQL Azure portal to create a firewall setting that enables connection to your SQL Azure server, you can use the server-level principal login and the master database to view and edit your firewall settings.In the master database, the firewall settings are referred to as rules. The sys.firewall_rules view displays the current firewall settings and the sp_set_firewall_rule and sp_delete_firewall_rule stored procedures allow you to change the firewall settings. For Web roles - Check “Allow Microsoft Services access to this server” option under the Firewall Settings tab in the Server Administration page on the portal.
• #29: IKE talksFull list athttp://www.microsoft.com/sqlserver/2008/en/us/R2.aspx
• #30: Lynn
• #31: Lynn sys.dm_exec_connections – This view returns information about the connections established to your database.· sys.dm_exec_requests – This view returns information about each request that executes within your database · sys.dm_exec_sessions – This view shows information about all active user connections and internal tasks. · sys.dm_tran_database_transactions – This view returns information about transactions at the database level.· sys.dm_tran_active_transactions – This view returns information about transactions for your current logical database.· sys.dm_db_partition_stats – This view returns page and row-count information for every partition in the current database.
• #32: LynnFrom SSDS blog http://blogs.msdn.com/sqlazure/archive/2010/04/16/9997517.aspx
• #33: LynnFrom SSDS blog http://blogs.msdn.com/b/sqlazure/archive/2010/06/07/10020947.aspxSQL Azure Database is extending five-fold to offer 50GB of the database storage capacity to offer even more scalability to applications and data.  Provides higher scalability and easier management of applications and services. Available on June 28th worldwide, flexibility to buy 50GB database.  New SQL Azure Offer: SQL Azure Development Accelerator Core will be available on August 1st with a new promotional discount to customers and partners, allowing them to subscribe to larger SQL Azure database size at a 25 percent discount.  This new six month offer provides flexibility to purchase commitment quantities of SQL Azure Business Edition databases independent of other Windows Azure platform services at a deeply discounted monthly price.  You can purchase in 10 GB increments the amount of our Business Edition relational database that you require (each Business Edition database is capable of storing up to 50 GB).Please refer to our offer comparison table for more details by offer on our pricing.Support for Spatial Data: SQL Azure now offers support for the Geography and Geometry types as well as query support in T-SQL. This is a significant feature and now opens the Windows Azure Platform to support spatial and location aware applications.Data Sync Service for SQL Azure allows for more flexible control over data by deciding which data components should be distributed across multiple datacenters in different geographic locations, based on the customer’s internal policies and business needs.  Available in community technology preview at http://www.sqlazurelabs.com .Microsoft SQL Server Web Manager (SSWM) isan easy to use management tool for SQL Azure databases, and specifically designed for Web developers seeking a straightforward solution to quickly develop, deploy and manage their data-driven applications in the cloud. Available as a community technology preview in summer 2010. Access 10 Support for SQL Azure offers native data connectivity support from Microsoft Office 2010 to SQL Azure. This new feature provides an easy way to connect directly from Office applications to cloud-based relational databases, and more choices to integrate rich Office Business Applications that can connect directly to both on-premises or cloud databases.
• #34: LynnMSDN article: http://msdn.microsoft.com/en-us/library/ff951624.aspxSupport for database copy: Database copy allows you to make a real-time complete snapshot of your database into a different server in the data center. This new copy feature is the first step in backup support for SQL Azure, allowing you to get a complete backup of any SQL Azure database before making schema or database changes to the source database. The ability to snapshot a database easily is our top requested feature for SQL Azure, and goes above and beyond our database center replication to keep your data always available. The MSDN Documentation with more information is entitled: Copying Databases in SQL Azure.Update on Project Houstonhttp://blogs.msdn.com/b/sqlazure/archive/2010/08/25/10054230.aspx
• #35: LynnMSDN article: http://msdn.microsoft.com/en-us/library/ff951624.aspxSupport for database copy: Database copy allows you to make a real-time complete snapshot of your database into a different server in the data center. This new copy feature is the first step in backup support for SQL Azure, allowing you to get a complete backup of any SQL Azure database before making schema or database changes to the source database. The ability to snapshot a database easily is our top requested feature for SQL Azure, and goes above and beyond our database center replication to keep your data always available. The MSDN Documentation with more information is entitled: Copying Databases in SQL Azure.Update on Project Houstonhttp://blogs.msdn.com/b/sqlazure/archive/2010/08/25/10054230.aspx
• #36: Lynn
• #37: Lynn
• #38: Ike talks, Lynn demosFor MonitoringGRANT VIEW DATABASE STATE TO database_user;Example: Space UsedMonitor space used via ‘sys.dm_db_partition_stats’Trap ‘out of space exception’ – SQL Error 40544
• #39: Lynn talks, then IkePricing table - http://www.microsoft.com/windowsazure/pricing/#sqlWeb Edition: Up to 1 GB relational database = $9.99 / month Up to 5 GB relational database = $49.95 / month** Business Edition: Up to 10 GB relational database = $99.99 / month Up to 20 GB relational database = $199.98 / month** Up to 30 GB relational database = $299.97 / month** Up to 40 GB relational database = $399.96 / month** Up to 50 GB relational database = $499.95 / month** Data transfers = $0.10 in / $0.15 out / GB - ($0.30 in / $0.45 out / GB in Asia)* * No charge for inbound data transfers during off-peak times through October 31, 2010** SQL Azure 50 GB Business Edition Database and 5 GB Web Edition Database will be available starting on June 28, 2010.
• #40: Lynn
• #41: Lynn talks, Ike typeshttp://msdn.microsoft.com/en-us/library/ff951624.aspx-- execute on the master database-- retrieve state of the new databaseselect name, state, state_desc from sys.databaseswhere name = 'databaseBackup'-- retrieve copying detailsselect * from sys.dm_database_copieswhere database_id = DB_ID('databaseBackup')When the database copying completes, the new database will enter one of two states, as indicated by the state_desc column of the sys.databases view. If the copying was successful, the new database will enter the Online state.
• #42: Ike talks
• #43: Ike talks, Lynn types
• #46: Ike talks
• #47: http://www.silverlight.net/content/samples/odataexplorer/ - Odata viewer
• #48: Lynn talksCons – of ‘Generate Script’ areVerbose – INSERT Statements instead of raw dataData is scripted with a fixed 100 row batch size.Edit “GO” statements between small batchesRound-trip EfficiencyUse “SET NOCOUNT ON”
• #49: Lynn talks
• #52: Lynn talks
• #53: Lynn talks
• #54: Lynn talks
• #55: Lynn talks
• #57: Lynn talks
• #58: IkeFrom this URL - http://www.microsoft.com/sqlserver/2008/en/us/R2-editions.aspxWhat's New in SQL Server 2008 R2 EditionsWith SQL Server 2008 R2, Microsoft continues to be the value leader, offering rich functionality to support OLTP and BI workloads out of the box at a low cost of ownership relative to competitors. With increasing hardware innovations, SQL Server continues to be the only major database vendor who does not price per core for multi-core processors. What’s New in SQL Server 2008 R2 EditionsBuilt on SQL Server 2008, SQL Server 2008 R2 delivers higher mission-critical scale, more efficient IT, and expanded reporting and analytics through self-service business intelligence. SQL Server 2008 R2 introduces two new premium editions to meet the needs of large scale datacenters and data warehouses.SQL Server 2008 R2 DatacenterSQL Server 2008 R2 Parallel Data WarehouseNew Premium EditionsDatacenterBuilt on SQL Server 2008 R2 Enterprise, SQL Server 2008 R2 Datacenter is designed to deliver a high-performing data platform that provides the highest levels of scalability for large application workloads, virtualization and consolidation, and management for an organization’s database infrastructure. Datacenter helps enable organizations to cost effectively scale their mission-critical environment. Key features new to Datacenter:Application and Multi-Server Management for enrolling, gaining insights and managing over 25 instancesHighest virtualization support for maximum ROI on consolidation and virtualizationHigh-scale complex event processing with SQL Server StreamInsight™Supports more than 8 processors and up to 256 logical processors for highest levels of scaleSupports memory limits up to OS maximum Parallel Data WarehouseSQL Server 2008 R2 Parallel Data Warehouse is a highly scalable data warehouse appliance-based solution. Parallel Data Warehouse delivers performance at low cost through a massively parallel processing (MPP) architecture and compatibility with hardware partners – scale your data warehouse to tens and hundreds of terabytes.Key features new to Parallel Data Warehouse:10s to 100s TBs enabled by MPP architectureAdvanced data warehousing capabilities like Star Join Queries and Change Data CaptureIntegration with SSIS, SSRS, and SSASSupports industry standard data warehousing hub and spoke architecture and parallel database copyInvestments in Core EditionsSQL Server 2008 R2 EnterpriseSQL Server 2008 R2 Enterprise delivers a comprehensive data platform that provides built-in security, availability, and scale coupled with robust business intelligence offerings—helping enable the highest service levels for mission-critical workloads.The following capabilities are new to Enterprise:PowerPivot for SharePoint to support the hosting and management of PowerPivot applications in SharePointApplication and Multi-Server Management for enrolling, gaining insights and managing up to  25 instancesMaster Data Services for data consistency across heterogeneous systemsData Compression now enabled with UCS-2 Unicode support High-scale complex event processing with SQL Server StreamInsight™SQL Server 2008 R2 StandardSQL Server 2008 R2 Standard delivers a complete data management and business intelligence platform for departments and small organizations to run their applications—helping enable effective database management with minimal IT resources.The following capabilities are new to Standard:Backup Compression to reduce data backups by up to 60% and help reduce time spent on backups *Can be managed instance for Application and Multi-Server Management capabilitiesHigh-scale complex event processing with SQL Server StreamInsight™
• #59: Lynn
• #60: Lynn
• #61: Both
• #62: Lynn