SQL 2014 In-Memory OLTP

Editor's Notes

• #2: Next user group meetings 20th October. Register interest using the link; https://www.eventbrite.co.uk/e/oct-hants-sql-user-group-meeting-tickets-25771056925?utm_source=eb_email&utm_medium=email&utm_campaign=event_reminder&utm_term=eventname
• #3: Increase in memory capacity: Rise in 64 bit process and fall in memory prices. Cost per GB of RAM dropped to a few £. Number of cores on processors increasing. The In-Memory data structures designed to use very large number of cores.
• #4: Other vendors offering In-Memory OLTP data processing. Main-memory databases: Oracle TimesTen, IBM SolidDB, Microsoft SQL Server that target embedded database space. Applications caches or key-value stores: The Velocity, App Fabric Cache and Gigaspaces uses middle-tier memory to offload work from databases by offering transactions, range indexing, query.
• #5: Pre 2014: Traditional SQL Server engine write 8KB data pages into memory and then back out to disk can incur higher latency cost. SQL was designed originally when memory was expensive and followed the disk-based tables . Used locks and latches to prevent interference between concurrent transaction on the same data rows, thus preserve ACID (Atomic, Consistency, Insolation, Durable) characteristics.
• #6: 1) Memory -optimized tables and indexes: whole table and its indexes are stored in memory all the time. It is using multiple versions of each row's data. It is characterized as non-blocking multi-version optimistic concurrency control. Eliminates both locks and latches to make it faster. Referred to as MVCC model (Multi-version Concurrency Control) 2) Natively-compiled stored procedures: Only access memory-optimized data structures e.g. memory-optimized tables, table variables. The key features - Rows in table read/written to memory, entire table in memory, Non-blocking multi-version optimistic concurrency control, have option of durable (second copy on disk) & non-durable data, in memory data only read from disk in DR.
• #8: Disk based Extents: 8 physically contiguous pages or 64 KB. SQL DB has 16 Extents per MB. Each table : Data stored in data or index 8K pages. Each database can contain 128 pages per MB. Each page starts with a 96-byte header, consisting system information about the page (page number, page type, amount of free space, allocated unit ID of object that owns page). A page stores only rows from the same table. DML operation requires page to be loaded into buffer-pool, which resides in RAM. In–Memory data store New row format. Uses multi-version for each record. Rows are stored as individual rows. 2 rows in a data file can belong to different memory optimized tables.
• #9: SCHEMA_ONLY: (non-durable table): If SQL restarted data is lost as the Schema is recreated to avoid transaction logging and checkpoint reducing I/O operations. SCHEMA_AND_DATA: Durability for schema and data. 2 Levels of data durability available - fully durable or delayed durability. Full durable tables same as the disk-based table. Delay durability is quicker but risk of data loss if server crash/fail over.
• #10: Speed – 1) Faster data access compared to disk based tables. 2) T-SQL compiled to machine code so fewer instructions/layers so business logic process faster. Full ACID (Atomicity, Consistency, Isolation, Durability) Support – Ensure all database transaction are reliably processed. Obey to process all actions within the transaction or non rule. Integrated into SQL Instance: Can deploy using T-SQL or SSMS for new or existing DBs. SQL Support and Manageability tools provided – 1) Native Compilation Advisor: Shows sps that are incompatible with natively compiled procedures. 2) Resource Governor: Provide ability to set memory quota. 3) Memory usage SSMS standard database report. 4) Metadata Enhancements –Catalogue view enhancements (sys.table and sys.table) and Dynamic Management Objects (sys.dm_db_xtp_checkpoint_stats, sys.dm_xtp_gc_stats). 5) Extended Events – 3 extended event packages help monitor and troubleshooting. 6) Performance Counters. 7) AMR (Analysis, Migrate, Report) tool
• #11: Very New: 1) Not supported pre-2014. 2) Not all applications will support In-Memory even if they support SQL 2014. 3) Cannot use replication, mirroring and database snapshots. 4) Only available in premium service Azure SQL instance and ones created after In-Memory launched. Large organisations tend to use a lot of 3rd party applications, therefore little/no control over source database. If QQ alter 3rd party application user databases could violate software support agreements. Troubleshoot future performance problems: 1) No Execution Plans available as it uses compiled assembly instructions so no execution plan created. 2) No Recompile, therefore have to drop and recreate stored procedures and lose permissions. Sharing RAM with SQL Instance: 1) In Memory OLTP bedded within SQL Server. 2) Setting SQL Max memory sets it for both SQL Instance and In-Memory OLTP engine. 3) Both engines compete for same resources – ENTIRE SQL INSTANCE CRASHES? Limited SQL object support: 1) replication, mirroring and database snapshot tables unavailable. 2) No Schema changes – No Alter table. 3) Cannot convert disk-based tables to become In-Memory tables. 4) In-Memory doesn’t recognise – ALTER TABLE, CREATE INDEX, DROP INDEX and ALTER INDEX. 5) FOREIGN KEYS (losing referential integrity) and CHECK CONSTRAINTS unsupported. 6) T-SQL operators and functions not supported. 7) No UNIQUE indexes a part from PRIMARY KEY. 8) Max 8 indexes including PK supported. 9) Cannot create DML triggers. 10) Data type restrictions – Doesn’t support LOB, XML, CLR or max data types. 11) Mirroring and replication of memory- optimized tables unsupported. RAM requirement: 1) Microsoft recommend doubling RAM allocate for version In-Memory OLTP use. 2) In-memory data stored in non-compressed format. SQL 2014 In-memory table limited to 250GB.
• #12: Lock or latch contention – Reduce excessive lock or latch wait times, In-Memory preserves ACID transaction properties without requiring locks. High I/O and Logging: Doesn’t require disk reads. Streaming checkpoint operations are min resources to write durable data to disk for checkpoint files. In-memory doesn’t write index information to disk. Transaction Logging: Disk-based tables required actions written to T-Log on disk. In-Memory enable SCHEMA_ONLY tables which require no logging. Logging overhead for SCHEMA_AND_DATA is reduced because memory optimized can contain information about many rows and index changes are not logged. Hardware resource limitations: CPU stressed causing slowdown. Natively compiled code uses fewer CPU instructions to interpret code and less CPU use. High volume of INSERTs and SELECTs: In-Memory eliminates locks and latch requirements. Also table logs are more optimized. No index changes are logged. SCHEMA_ONLY help reduce overhead with table row logging. Suitable only for many concurrent SELECTs executed on a single thread as parallel operations unsupported. Session State management: Apps use it to retain session information, e.g. website maintain information about user choices and actions across multiple HTTP requests and doesn’t need to be preserved across server restarts.
• #13: Launch SQL2014 SSMS Connect to RL-4009480\SQL2014INST Use script; C:\Users\aekeyse\Documents\Technical\Database\SQLUserGroup\ExternalInMemoryOLTP\Demo1DeployingExample-Local
• #14: Transaction performance collector: Gather information about existing tables and sps and recommends which ones to migrate to In-Memory. Memory Optimization Advisor: Guides through migrating a table to memory-optimized table. Native Compilation Advisor: Identify T-SQL required changes to migrate sps to natively complied sps. Example from - http://sqlmag.com/sql-server-2014/sql-server-2014-analysis-migrate-report-tool-amr
• #15: Stored procedure usage analysis: Contains store procedure information for migration to natively compiled stored procedures. Table usage analysis: Contains disk-based tables that can be migrated to memory optimized tables.
• #16: Demo Create AMR_DW Db using Wizard (Object Explorer in SSMS >Management folder >Right-click Data Collection> configure Management Data Warehouse) Configure Data Collection Process (SSMS> Management folder> Data Collection> Tasks>Configure Data Collection) but click cancel. Show the AMR_DW DB in Databases. Blue Peter style –Show existing SQL agent jobs: collection_set_6_collection, collection_set_6_upload, collection_set_7_collection, collection_set_7_upload Blue Peter style – Show Transaction Performance Analysis Overview (Right click MDW database>Reports>Transaction Performance Analysis Overview> Click on URLs) Demo example from http://sqlmag.com/sql-server-2014/sql-server-2014-analysis-migrate-report-tool-amr
• #18: Show AdventureWorks2008R2.dbo.ErrorLog and it will fail (Right click on ErrorLog and choose Memory Optimization Advisor) Show using AdventureWorks2008R2.dbo.AKEmp, which has no keys or constraints. (Right click table> Memory Optimization Advisor>Set Latin1_General_100_BIN2 for NationalIDNumber and set it as part of primary key> Next>Migrate>Script>Migrate>Click O.K.>Show Db files) Steps used from https://www.pythian.com/blog/sql-server-2014-ctp2-memory-optimization-advisor
• #20: Right click on InsertName sp>Native Compilation Advisor>Click Cancel. This should work. Use uspGetBillOfMaterials that will fail (Right Click uspGetBillOfMaterials >Native Compilation Advisor>Generate Report>Open the File in Internet Explorer). Demo example from http://sqlserver-help.com/tag/native-compilation-advisor/
• #21: The process flow I would follow for estimating memory requirements in SQL Server 2014 is this: Get current number of rows in the table. Get current size of row by adding up data types, row header (additional 24 bytes) and 8 bytes for each index. Multiply (1) and (2) and set aside the result. For each hash index, get the bucket count, round up to the next highest power of 2, and then multiply by 8. Get sum of all hash indexes in (4) and set aside. For each non-clustered index, get the width of the key columns, add 8, then multiply that by the number of unique rows. Get sum of all non-clustered indexes in (6) and set aside. Add (3), (5), and (7) together. Multiply the result from (8) by 2, this is the guesstimate that includes row versioning. Example calculation from http://thomaslarock.com/2014/06/estimating-memory-requirements-in-sql-server-2014/
• #22: There is a memory threshold available depending on the target pool memory Instance target committed memory can be checked by viewing sys.dm_os_sys_info Dynamic Management View Figures from https://www.mssqltips.com/sqlservertip/3368/setup-a-memory-quota-for-sql-server-memory-optimized-databases/
• #23: Demo using script file; C:\Users\aekeyse\Documents\Technical\Database\SQLUserGroup\ExternalInMemoryOLTP\DemoSetMemoryQuota -Local Can achieve same results using SSMS GUI 1) Open Object Explorer. 2) Expand the Management. 3) Select Resource Governor. 4) Right-click the resource pool to be deleted, and then click Delete. 5) To delete the resource pool, click OK. Demo example from https://www.mssqltips.com/sqlservertip/3368/setup-a-memory-quota-for-sql-server-memory-optimized-databases/
• #25: Do you think you would use In-Memory OLTP?