Welcome to the nightmare of locking, blocking and isolation levels!
Download as PPTX, PDF2 likes2,259 views
The document outlines a presentation on SQL Server transactions, locks, and isolation levels by experts Magi Naumova and Boris Hristov, highlighting their extensive experience and roles in SQL training and consulting. It covers essential concepts such as transaction definitions, types of locks, locking mechanisms, and transaction isolation levels, emphasizing the importance of managing locks to avoid blocking and deadlocks in database environments. Recommendations for minimizing locking issues and utilizing optimistic concurrency approaches are also discussed.
![Implicit Transactions
SQL Server is responsible for opening the transaction
We are responsible for committing or rolling it back
Can be turned on from Connections tab in Server Properties
SET IMPLICIT_TRANSACTIONS ON SET IMPLICIT_TRANSACTIONS ON
USE AdventureWorks2012 USE AdventureWorks2012
GO GO
UPDATE [Person].[Address] UPDATE [Person].[Address]
SET AddressLine1='Microsoft, Bulgaria' SET AddressLine1='Microsoft, Bulgaria'
WHERE AddressID=2 WHERE AddressID=2
COMMIT – this will write a change to the db ROLLBACK – this will not write a change to the d](https://image.slidesharecdn.com/lockingblockingisolationlevelsugmeetingapril2013-130405081008-phpapp01/85/Welcome-to-the-nightmare-of-locking-blocking-and-isolation-levels-9-320.jpg)
![Let’s update a row. What do we need?
S
A query!
IX
USE AdventureWorks2012
GO
UPDATE [Person].[Address]
SET AddressLine1='Microsoft, Bulgaria' Header IX
WHERE AddressID=2
Row
Row
Row
Row X
Row](https://image.slidesharecdn.com/lockingblockingisolationlevelsugmeetingapril2013-130405081008-phpapp01/85/Welcome-to-the-nightmare-of-locking-blocking-and-isolation-levels-17-320.jpg)
Welcome to the nightmare of locking, blocking and isolation levels!
- 1. Българска SQL & BI потребителска група http://bgsqlgroup.com SQL & BI User Group Bulgaria on FB
- 2. WELCOME TO THE NIGHTMARE OF LOCKING, BLOCKING AND ISOLATION LEVELS! Magi Naumova Boris Hristov BG SQL User Group Meeting, 4/4/2013
- 3. About Magi Working with SQL Server from v6.5 MCT from 1998 SQL Server Trainer and Consultant for more than 15 years, having more over 100 projects in Bulgaria, Finland, Germany, UK, Greece.. 5 years Senior Consultant in Microsoft Microsoft Certified Master SQL Server 2008 MVP SQL Server SolidQ Mentor & Trainer Now I am running SQL Master Academy training program www.SQLMasterAcademy.com
- 4. About Bobi SQL Server DBA at HP Trainer at New Bulgarian University and FMI MCT & MCITP DBA SQL Server 2008 brshristov@live.com www.borishristov.com
- 5. Agenda… Transactions. What are they? Locks. What is there for us? Troubleshooting locking problems Transaction Isolation Levels
- 6. Transactions. What are they?
- 7. What Are Transactions? A transaction is an Atomic unit of work A transaction leaves data in a Consistent state A transaction is Isolated from other concurrent transactions A transaction is Durable
- 8. Auto Commit Transactions Default transaction mode Every TSQL statement is committed or rolled back on completion Compile errors result in entire batch not being executed Run time errors may allow part of the batch to commit ---run time error -- compile error - partially executed USE AdventureWorks2012; GO CREATE TABLE TestBatch (Cola INT PRIMARY KEY, Colb CHAR(3)); GO INSERT INTO TestBatch VALUES (1, 'aaa'); INSERT INTO TestBatch VALUES (2, 'bbb'); INSERT INTO TestBatch VALUSE (3, 'ccc'); ---Duplicate key error. VALUES (1, -- error, error! GO SELECT * FROM TestBatch; -- Returns no rows. 2. rows 1 and GO
- 9. Implicit Transactions SQL Server is responsible for opening the transaction We are responsible for committing or rolling it back Can be turned on from Connections tab in Server Properties SET IMPLICIT_TRANSACTIONS ON SET IMPLICIT_TRANSACTIONS ON USE AdventureWorks2012 USE AdventureWorks2012 GO GO UPDATE [Person].[Address] UPDATE [Person].[Address] SET AddressLine1='Microsoft, Bulgaria' SET AddressLine1='Microsoft, Bulgaria' WHERE AddressID=2 WHERE AddressID=2 COMMIT – this will write a change to the db ROLLBACK – this will not write a change to the d
- 10. Explicit Transactions A transaction in which start and end of transaction is explicitly declared BEGIN TRANSACTION COMMIT TRANSACTION OR ROLLBACK TRANSACTION XACT_ABORT ON/OFF – control the rollback behavior SET XACT_ABORT ON – if run time error is generated everything is rolled back USE AdventureWorks2012 GO BEGIN TRANSACTION FundsTransfer GO EXEC HumanResources.DebitAccount '100', 'account1'; EXEC HumanResources.CreditAccount '100', 'account2'; COMMIT TRANSACTION;
- 11. Locks. What is there for us?
- 12. Methods of Concurrency Control Two main concurrency control types: 1. Pessimistic – SQL Server uses locks, causes blocks and who said deadlocks? 2. Optimistic 2. Optimistic – SQL Server generates versions for everyone, but the updates…
- 13. What Are Locks and what is locking? Lock – internal memory structure that “tells” us what we all do with the resources inside the system Locking – mechanism to protect the resources and guarantee consistent data
- 14. Common lock types Shared (S) Update (U) Used for: Reading Used for: Preparing to modify Duration: Released almost immediately Duration: End of the transaction or until (depends on the isolation level) converted to exclusive (X) Exclusive (X) Intent Used for: Modifying Used for: Preventing incompatible Duration: End of the transaction locks Duration: End of the transaction
- 15. Lock Compatibility Not all locks are compatible with other locks. Lock Shared Update Exclusive Shared (S) X Update (U) X X Exclusive (X) X X X
- 16. Lock Hierarchy Database Table Page Row
- 17. Let’s update a row. What do we need? S A query! IX USE AdventureWorks2012 GO UPDATE [Person].[Address] SET AddressLine1='Microsoft, Bulgaria' Header IX WHERE AddressID=2 Row Row Row Row X Row
- 18. Methods to View Locking Information Dynamic SQL Server Performance Management Profiler or monitor or Activity Views Extended Monitor Events
- 20. Locking and blocking Locking and blocking are often confused! Locking • The action of taking and potentially holding locks • Used to implement concurrency control Blocking is result of locking! • One process needs to wait for another process to release locked resources • In a multiuser environment, there is always, always blocking! • Only a problem if it lasts too long
- 21. Lock granularity and escalation SQL Server decides (during compilation) the granularity of locks to be used: • Row • Page Lock granularity can be controlled via LOCK HINTS (PAGLOCK, TABLOCKX, etc…) Escalation always happens this way: Row -> table lock (or partition lock if possible) Page -> table lock (or partition lock if possible) Lock escalation can be disabled: • Trace flag 1211 – disables lock escalation on server level • Trace flag 1224 – disables lock escalation on server level until 40% of the memory used is consumed
- 22. Lock escalation S S IX X IX X X >= 5000 X X
- 23. Controlling Lock escalation Switch the escalation level (per table) SELECT lock_escalation_desc FROM sys.tables WHERE name = 'Person.Address' ALTER TABLE Person.Address SET (LOCK_ESCALATION = {AUTO | TABLE | DISABLE) AUTO – Partition-level escalation if the table is partitioned TABLE – Always table-level escalation DISABLE – Do not escalate until absolutely necessary
- 24. Lock Duration SET LOCK_TIMEOUT 5000 SET LOCK_TIMEOUT specifies number of milliseconds to wait BEGIN TRAN Default LOCK_TIMEOUT is -1 = wait indefinitely UPDATE Production.ProductInventory Session level option SET Quantity = 500 WHERE ProductID = 1; When timeout expires, error 1222 is returned -- what happens if the update times out? Not a good idea to be used! Does not resolve blocking problems! DELETE FROM Production.ProductModel WHERE Name LIKE 'Mountain%'; COMMIT
- 25. Blocking a.k.a live locking Blocking occurs because of locked resources First incompatible lock request waits All other locks requests (even if compatible) WAIT How to resolve blocking problems: 1. Keep the transactions as short as possible 2. No user interactions required in the middle of the transaction 3. Reduce row by row operations (cursors) 4. Use indexes 5. Consider a server to offload some of the workloads 5. Choose isolation level
- 26. What Are Deadlocks? Who is victim? • Cost for Rollback • Deadlock priority – SET DEADLOCK_PRIOIRTY Task A Resource 1 Task B Resource 2
- 27. DEMO Capturing locking information Who is blocking who Lock escalation – both to table and partition Deadlock and the SET DEADLOCK_PRIORITY option
- 28. Transaction isolation levels Transaction Isolation Levels control when locks are taken and how long they are held
- 29. Pessimistic Isolation Levels Changing Isolation level ◦ SET TRANSACTION ISOLATION LEVEL ….. on session level eXclusive lock Transaction 1 Read Update Select Uncommitted Transaction 2 Lowering the default Read Committed ◦ NOLOCK hint – set on the table level ◦ SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED You have Dirty Reads in this isolation level! Saying you want less blocking at the cost of inconsistent data! If you need to use it a lot, that means, you need a reporting database (or optimistic locking)
- 30. Pessimistic Isolation Levels Increasing the default level to Repeatable Read Repeatable Transaction 1 S(hared) lock Read select Transaction 2 Update S lock is held until the end of the transaction in order to read the same data Every transaction that has to update rows has to wait until T1 completes Inserts (phantoms) performed by another transaction can still occur in the table
- 31. Pessimistic Isolation Levels Increasing to Serializable Transaction 1 S(hared) lock Serializable select Transaction 2 Insert Repeatable Read + Holding the locks for a zone of rows If there is no index (or not used) the whole tale will be locked Highest level of isolation, lowest level of concurrency Be careful with using Frameworks (EF, LYNC)
- 33. Optimistic Concurrency Based on Row versioning ◦ When updating a row, the previous version is stored in the version store ◦ The new row contains a pointer to the old row in the version store Transaction 1 V1 V2 Select Transaction 2 Readers do not block writers and writers do not block readers. BUT writers can and will block writers, this can cause conflicts. Adds 14bytes to every row Needs Version Store i.e. TempDB space
- 34. Implementation – RCSI and SI RCSI ◦ Statement-Level versioning, i.e. any query sees the last version of data as of the beginning of the statement ◦ Requires ALTER ATABASE SET READ_COMMITTED_SNAPSHOT ON Transaction 1 V1 V2 Select Select in RCSI Transaction 2 Select in SI Snapshot Isolation Level ◦ Session Level versioning, i.e. the most recent committed version of the rows as of the beginning of the transaction ◦ Requires ALTER DATABASE SET ALLOW_SNAPSHOT_ISOLATION ON ◦ Requires SET TRANSACTION ISOLATION LEVEL SNAPSHOT ◦ If used then better together with RCSI! Else, you have drawbacks of versioning only!
- 35. DEMO Optimistic Concurrency Isolation Levels
- 36. Managing Version store The Version Store ◦ SQL Server starts generating versions in tempdb as soon as a database is enabled for one of the snapshot-based isolation levels ◦ Maintains link lists of rows, the end of the list is the oldest version of that particular row Managing Version Store ◦ SQL Server maintains a clean-up thread ◦ If file cannot grow, a snapshot query fails (tempdb growth errors) ◦ Monitoring counters ◦ Free Space in tempdb ◦ Version Store Size ◦ Version Generation Rate and Version Cleanup Rate ◦ Longest Transaction Running Time ◦ Snapshot Transactions
- 37. DEMO Versioning details and managing version store
- 38. Summary If you experience blocking consider implementing Optimistic Concurrency Consider RCSI only, it is way cheeper RCSI + SI only if needed higher isolation level per transaction (RR in pesimistic) If you have to use often no blocking hints the consider either reporting DB or optimistic concurrency Be careful with READPAST, NOLOCK, ….LOCK TIMEOUT and any other work arounds If you keep default concurrency, then follow the recommendations for minimizing blocking and lock escalations Be careful with hidden hints/isolation levels changes using EF, LYNC, etc
- 39. Resources MCM Readiness videos on locking lecture and demo MCM Readiness video on Snapshot Isolation Level http://blogs.msdn.com/b/bartd/archive/tags/sql+lockin g/ http://www.sqlskills.com/blogs/paul/category/locking/ Lock hints - http://www.techrepublic.com/article/control-sql-server- locking-with-hints/5181472
- 40. Thank you. • Magi.Naumova@SQLMasterAcademy.com ? • www.maginaumova.com • brshristov@live.com • www.borishristov.com
Editor's Notes
- #3: Когато започнах се надявах да отида на безплатен ивент на подобна тема за това, за да мога да я разбера по – добре, а сега имам щастието самия аз да ви дам моите знания. Наистина е леко по – специална за мен и има огромна стойност Та...
- #8: Atomic – Error handling and how we use itConsistent – Foreign keys, check constraints, triggers, validation rules.Isolated – locking, isolation levelsDurable – transaction log – rolling back and forward
- #9: Every Transact-SQL statement is committed or rolled back when it completes. If a statement completes successfully, it is committed; if it encounters any error, it is rolled back.
- #11: Mention try - catch
- #13: You use locking, because you do not like the fact that someone can change the data you are reading at the momentOptimistic – versionsКонтрол на достъпа
- #14: How much memory is used – locks on server level (decommissioned)Up to 60% of the buffer pool – out of lock memory errorsIn startup (with 0 option) it allocates 2500 locks. 2%LOCK BLOCK Lock Owner block64 bytes – 32 bit system 32 bytes – 32 bit 128 bytes – 64 bit system 64 bytes – 64 bit
- #15: Обясни какво се случва – как се минава от SUXИнтент лок – нужни са, защото SQL Server може да хваща локове на много различни обекти. Ако някой процес иска да локне таблицата, все пак трябва да има начин да му се каже, че някой ъпдейтва стойности в нея, нали?
- #16: Обясни какво се случва – как се минава от SUX+ Shared <- Exclusive <- SharedSIX - convert lock. Example – shared lock on a row now has to become X lock. However, for the exclusive to occur, we ed IX in the upper levels – that is the moment when we convert our S and IS locks to X and SIX locks
- #17: Всяка една транзакция преминава от тук. Обяснение защо.
- #18: Why do we need Shared locks – to explain it to the audienceWhy do we need intend locks and what is their goal? Why are they helping us?
- #19: Този слайд трябва да е последен в секцията introduction to locksDynamic Management ViewsSystem stored proceduresSQL ProfilerPerformance Monitor countersBlocked Process ReportDeadlock events and trace flag 1222SQL Server 2008 addsPerformance Data CollectionExtended Events
- #22: Not exactly – it always decides during compilation ROW or PAGE level – tow options only (when standard DML operations, not altering object)After that it could escalate to partition/table level
- #23: Memory takes memory 60…. InternalsEscalating in case of 40% of the memory used for locks Or 5000 locks on one objectIf escalation fails because of incompatible lock – sql server waits for another 1250 locks Always to table(partitioning) – auto to partition from 2008What is the solution?
- #24: Should be VERY VERY Careful when choosing to control it manually!! Could easy cause more problems, and it is rarely used as a lock escalation solution nor as a resolution of blocking problems! Could be set in specific cases only
- #25: The READPAST query hintERROR HANDLING for error 1222!
- #26: Blocking what is that, when does it happen, reasons behind (holding locks because of long running user trans (examples), long running trans because of resource usage problems (IO waits..), lock escalations because of row by row operations (examples), how to identify/ how to isolate reasons behindКонтекст: by default – pesimistic, read committed
- #27: Threads are killedLOCK_MONITOR checks every 5 seconds for deadlock. Cycle and conversion deadlocksError 1205 should be handled Some operations cannot be chosen for deadlock victims (rollbacks)After the deadlock occurs, the engine still continues to work and check every so often (100 ms) for other deadlocks
- #33: Shortly:isolation level RU, Isolation level RRReadpast - difference between READPAST, READUNCOMMITTED,and setting LOCK_TIMEOUT to 0. All these techniques let you avoid blocking problems,but the behavior is slightly different in each caseSerializable with Table lockDiagnose and recognize:Adam Machanic'ssp_WhoIsActive @get_additional_info = 1 will give you the isolation level, and I've used that for quick diagnostics to find out who's doing dirty reads. Just good for a first pass though.+ Aaron’s script: http://www.mssqltips.com/sqlservertip/2475/sql-server-queries-with-hints/
- #36: RCSI, snapshot isolation, update conflicts
- #37: Snapshot transaction metadatasys.dm_tran_ version_store; sys.dm_tran_transactions_snapshot; sys.dm_tran_active_snapshot_database_transactions
- #40: Magi to add more if neededRecap of what we discussed in the previous slide?