![create table letsgomavs
(col1 int not null identity primary key
nonclustered hash with (bucket_count =
1000000),
col2 int not null,
col3 varchar(100)
)
with (memory_optimized = on, durability
= schema_and_data)
struct NullBitsStruct_2147483654
{
unsigned char hkc_isnull_3:1;
};
struct hkt_2147483654
{
long hkc_1;
long hkc_2;
struct NullBitsStruct_2147483654 null_bits;
unsigned short hkvdo[2];
};
Header hkc_1 hkc_2
Null
bits
Offset array col3
âyour dataâ
nullable](https://image.slidesharecdn.com/insidesqlserverin-memoryoltpsqlsatnyc2017-170520132654/85/Inside-sql-server-in-memory-oltp-sql-sat-nyc-2017-13-320.jpg)
Inside sql server in memory oltp sql sat nyc 2017
- 1. This Photo by Unknown Author is licensed under CC BY-SA
- 3. The What and Why Data, Indexes, and Natively Compiled Stored Procedures Concurrency and Transactions Logging and Checkpoint Based on SQL Server 2016Check out the Bonus Material Section
- 5. Project Verde âą 2007 âEarlyâ Hekaton âą 2009/2010 Hekaton becomes In- Memory OLTP âą SQL Server 2014 In-Memory OLTP Unleashed âą SQL Server 2016 OLTP Through the Looking Glass, and What We Found There by Stavros Harizopoulos, Daniel J. Abadi, Samuel Madden, Michael Stonebraker , SIGMOD 2008
- 6. XTP = eXtreme Transaction Processing HK = Hekaton
- 8. âFeelsâ like normal disk-based tables but in memory Internally completely different Hash or non-clustered index choices (at least one required)
- 10. USER GC USER USER USER USER USER Node 0 GC Node <n> user transactions Garbage collection one per node database CHECKPOINT CONTROLLER CHECKPOINT CLOSE LOG FLUSH LOG FLUSH thread Data Serializer thread thread In-Memory Thread Pools CHECKPOINT I/O U U P XTP_OFFLINE_CKPT H H P U H P P U U H P user scheduler hidden scheduler preemptive You could see session_id > 50 Data Serializer dm_xtp_threads
- 11. Your data. âbytesâ to HK engine Points to another row in the âchainâ. One pointer for each index on the table Timestamp of INSERT Halloween protection Number of pointers = # indexes Rows donât have to be contiguous in memory Timestamp of DELETE. â = ânot deletedâ
- 12. âą The HK Engine doesnât understand row formats or key value comparisons âą So when a table is built or altered, we create, compile, and build a DLL bound to the table used for interop queries dbid object_id Symbol file Rebuilt when database comes online or table altered cl.exe output Generated C source t = schema DLL p =native proc
- 13. create table letsgomavs (col1 int not null identity primary key nonclustered hash with (bucket_count = 1000000), col2 int not null, col3 varchar(100) ) with (memory_optimized = on, durability = schema_and_data) struct NullBitsStruct_2147483654 { unsigned char hkc_isnull_3:1; }; struct hkt_2147483654 { long hkc_1; long hkc_2; struct NullBitsStruct_2147483654 null_bits; unsigned short hkvdo[2]; }; Header hkc_1 hkc_2 Null bits Offset array col3 âyour dataâ nullable
- 14. 100,â Bob Dallas hash on name columns = name char(100), city char (100) hash index on name bucket_count = 8 hash index on city bucket_count = 8 hash on city 0 0 4 5 200,â Bob Fort Worth 90,â Ginger Austin 50,â Ryan Arlington 70,â Ginger Houston 3 4 5 chain count = 3 Hash collision duplicate key != collision Begin ts reflects order of inserts ts
- 15. Mixed Abstract Tree (MAT) built âą Abstraction with flow and SQL specific info âą Query trees into MAT nodes âą XML file in the DLL dir Converted to Pure Imperative Tree (PIT) âą General nodes of a tree that are SQL agnostic. C like data structures âą Easier to turn into final C code Final C code built and written âą C instead of C++ is simpler and faster compile. âą No user defined strings or SQL identifiers to prevent injection Call cl.exe to compile, link, and generate DLL âą Many of what is needed to execute in the DLL âą Some calls into the HK Engine All files in BINNXTP VC has compiler files, DLLs and libs Gen has HK header and libs Call cl.exe to compile and link Done in memory xtp_matgen XEvent
- 17. In-memory OLTP versioning In-Mem TableDisk based table 1, âRow 1â 100 101 102 103 104 105 106 Time T1 T2 BEGIN TRAN BEGIN TRAN 2, âRow 2â SELECT COMMIT SELECT COMMIT 100 101 102 103 104 105 106 Time T1 T2 BEGIN TRAN BEGIN TRAN 2, âRow 2â SELECT COMMIT SELECT COMMIT read committed and SERIALIZABLE = blocked XRCSI = Row 1 Snapshot = Row 1 RCSI = Row 1 and 2 Snapshot = Row 1 SERIALIZABLE = Msg 41325 Snapshot always used = Row 1 X
- 18. âą Code executing transactions in the Hekaton Kernel are lock, latch, and spinlock free Locks âą Only SCH-S needed for interop queries âą Database locks Latches âą No pages = No page latches âą Latch XEvents donât fire inside HK Spinlocks âą Spinlock class never used The âhostâ may wait â tLog, SQLOS We use Thread Local Storage (TLS) to âguardâ pointers We can âwalkâ lists lock free. Retries may be needed Atomic âCompare and Swapâ (CAS) to modify IsXTPSupported() = cmpxchg16b Spinlocks use CAS (âcmpxchgâ) to âacquireâ and âreleaseâ Great blog post here *XTP* wait types not in transaction code deadlock free CMED_HASH_SET fix
- 19. Release latches and locks Update index pages ( more locks and latch ) Modify page Latch page Obtain locks INSERT LOG record In-Memory OLTP INSERT Maintain index in memory Insert ROW into memory COMMIT Transaction = Log Record and Flush COMMIT Transaction = Insert HK Log Record and Flush Page Split Sys Tran = Log Flush Spinlocks No index logging SCHEMA_ONLY no logging No latch No spinlock
- 20. LOP_BEGIN_XACT LOP_INSERT_ROWS â heap page LOP_INSERT_ROWS â ncl index LOP_BEGIN_XACT LOP_MODIFY_ROW â PFS LOP_HOBT_DELTA LOP_FORMAT_PAGE LOP_COMMIT_XACT LOP_SET_FREE_SPACE - PFS LOP_COMMIT_XACT 100 rows Log flush PFS latch multiple times 213 log records @ 33Kb PFS latch Metadata access 1 pair for every row Alloc new page twice
- 21. LOP_BEGIN_XACT LOP_HK LOP_COMMIT_XACT 144 11988 84 HK_LOP_BEGIN_TX HK_LOP_INSERT_ROW HK_LOP_COMMIT_TX 100 Only inserted into log cache at commit. Log flush ROLLBACK = No log records for HK If LOP_HK too big we may need more than one
- 22. Typically 8Mb but can be 128Mb Why CHECKPOINT? All data written in pairs of data and delta files Typically 128Mb but can be 1Gb No WAL protocol
- 23. CHECKPOINT FILE types and states PRECREATED ACTIVE UNDER CONSTRUCTION WAITING..TRUNCATION ROOT FREE After first CREATE TABLE DELTA DATA ROOT FREE INSERT data rows DELTA DATA ROOT ROOT CHECKPOINT event DELTA DATA 0 to 100 TS 0 to 100 TS ROOT ROOT DELTA DATA INSERT more data rows DATA DELTA 101 to 200 TS 101 to 200 TS These will be used to populate table on startup and then apply log Checkpoint File Pair (CFP) and are what is in tlog before CHECKPOINT This can be reused or deleted at log truncation We constantly keep PRECREATED, FREE files available Any tran in this range FREEFREE
- 27. âą SQL Server In-Memory OLTP Internals for SQL Server 2016 âą In-Memory OLTP Videos: What it is and When/How to use it âą Explore In-Memory OLTP architectures and customer case studies âą Review In-Memory OLTP in SQL Server 2016 and Azure SQL Database âą In-Memory OLTP (In-Memory Optimization) docs âą Blog post on In-Memory OLTP and checkpoint files âą Retry logic for In-Memory OLTP Transactions http://speakerscore.com/7KKX
- 29. Always On Availability Groups HTAP applications Azure SQL Database Cross container transactions Table variables here BACKUP/RESTORE Transaction Performance Analysis Report Hear the case studies
- 30. How do you find data rows in a normal SQL table? âą Heap = Use IAM pages âą Clustered Index = Find root page and traverse index What about an in-memory table? âą Hash index table pointer known in HK metadata for a table. Hash the index key, go to bucket pointer, traverse chain to find row âą Page Mapping Table used for range indexes and has a known pointer in HK metadata. Traverse the range index which points to data rows âą Data exists in memory as pointers to rows (aka a heap). No page structure All data rows have known header but data is opaque to HK engine âą Schema DLL and/or Native Compiled Proc DLL knows the format of the row data âą Schema DLL and/or Native Compiled Proc DLL knows how to find âkeyâ inside the index Compute your estimated row size here âbag of bytesâ Hash index scan is possible
- 31. TS = 243 SELECT Name, City FROM T1 âwrite setâ = logged records âread setâ BEGIN TRAN TX3 â TS 246 SELECT City FROM T1 WITH (REPEATABLEREAD) WHERE Name = 'Jane'; UPDATE T1 WITH (REPEATABLEREAD) SET City âHelinskiâ WHERE Name = 'Susan'; COMMIT TRAN -- commits at timestamp 255 Greg, Lisbon Susan Bogata Jane Helsinki FAIL = âJaneâ changed after I started but before I committed and Iâm REPEATABLEREAD. With SQL update to Jane would have been blocked Commit dependency
- 32. create procedure cowboys_proc_scan with native_compilation, schemabinding as begin atomic with (transaction isolation level = snapshot, language = N'English') select player_number, player_name from dbo.starsoftheteam .. end Compile this Into a DLL Required. No referenced object/column can be dropped or altered. No SCH lock required Everything in block a single tran These are required. There are other options Iso levels still use MVCC Your queries
- 33. Hekaton implements its own memory management system built on SQLOS âą MEMORYCLERK_XTP (DB_ID_<dbid>) uses SQLOS Page allocator âą Variable heaps created per table and range index âą Hash indexes using partitioned memory objects for buckets âą âSystemâ memory for database independent tasks âą Memory only limited by the OS (24TB in Windows Server 2016) âą Details in dm_db_xtp_memory_consumers and dm_xtp_system_memory_consumers In-Memory does recognize SQL Server Memory Pressure âą Garbage collection is triggered âą If OOM, no inserts allowed but you may be able to DELETE to free up space Allocated at create index time Locked and Large apply
- 34. Remember this is ALL memory size Binding to your own Resource Pool What about CPU and I/O? no classifier function Not the same as memory
- 35. Check out this blog Upgrade from 2014 to 2016 can take time Large checkpoint files for 2016 https://support.microsoft.com/en-us/kb/3090141 From the CSS team
- 37. 37 Client Access · Backup/ Restore · DBCC · Buik load · Exception handling · Event logging · XEvents SOS · Process model (scheduling & synchronization) · Memory management & caching · I/O Txnâs Lock mgr Buffer Pool Access Methods Query Execution & Expression Services UCS Query Optimization TSQL Parsing & Algebrizer Metadata File Manager Database Manager Procedure Cache · TDS handler · Session Mgmt Security Support Utilities UDTs & CLR Stored Procs SNI Service Broker Logging & Recovery 10%10% Network, TDS T-SQL interpreter Query Execution Expressions Access Methods Transaction , Lock, Log Managers SOS, OS and I/O 35%45%
- 38. SQLOS task and worker threads are the foundation âUserâ tasks to run transactions Hidden schedulers used for critical background tasks Some tasks dedicated while others use a âworkerâ pool SQLOS workers dedicated to HK
- 39. NON-PREEMPTIVEPOOL Background workers needed for on-demand Parallel ALTER TABLE WORKERSPOOL Other workers needed for on- demand (non- preemptive) Parallel MERGE operation of checkpoint files PREEMPTIVEPOOL Background workers needed on-demand - preemptive Serializers to write out checkpoint files Command = XTP_THREAD_POOL wait_type = DISPATCHER_QUEUE_SEMAPHORE Command = UNKNOWN TOKEN wait_type = XTP_PREEMPTIVE_TASK Ideal count = < # schedulers> ; idle timeout = 10 secs HUU H U hidden scheduler user scheduler
- 40. âą Multi-version Optimistic Concurrency prevents all blocking âą ALL UPDATEs are DELETE followed by INSERT âą DELETED rows not automatically removed from memory âą Deleted rows not visible to active transactions becomes stale âą Garbage Collection process removes stale rows from memory âą TRUNCATE TABLE not supported Page deallocation in SQL Server
- 42. Query Plans and Stats here Query Store XEvent and SQLTrace Read the docs
- 43. Multi-row insert test Remember SCHEMA_ONLY has no logging or I/OBULK INSERT for in- mem executes and logged just like INSERT Minimally logged BULK INSERT took 271 log records @ 27Kb Latches required for GAM, PFS, and system table pates
- 44. Multiple Log Writers in SQL Server 2016 You could go to delayed durability Log at the speed of memory video
- 45. Over time we could have many CFPs
Editor's Notes
- #8: Following readme.txt in demo1_inmem_oltp_justshowus
- #11: CHECKPOINT controller will look at committed log records and use a data serializer thread from the pool. These pool threads put work in queues and the LOG FLUSH threads actually perform the checkpoint I/O (which is continuous) The CHECKPOINT CLOSE thread is the one that actually executes a checkpoint event. A request with command = XTP_CKPT_AGENT is used to keep FREE files replenished for CHECKPOINT files
- #13: We will discuss the compilation process when talking about natively complied DLLs
- #15: Hash collision on name index with Ryan because it is a different value than Bob. The first two Bob rows are technically not a collision because they are the same key value
- #17: Follow the instructions in readme.txt in demo2_inmem_oltp_debugging
- #19: T-SQL IsXTPSupported() = Windows API IsProcessorFeaturePresent(PF_COMPARE_EXCHANGE128) = _InterlockedCompareExchange128() = Cmpxchg16b instruction
- #20: Notice that log records are not even created until COMMIT.
- #23: We can switch to âlargeâ files (1Gb) if checkpoint has determine it can read 200Mb/sec and the machine has 16 cores and 128Gb for SQL Server What is the disk space required for checkpoint files? It is roughly the total size of memory consumed by your table data I need to make comments here about log truncation and checkpoint files âhanging aroundâ. This has been a problem for some customers