Database concurrency control & recovery (1)
- 1. Concurrency Control & Recovery Database Consistency Multi User Environment(Data Sharing) Transactions interference System Crash Hardware failure Software failure Concurrency Control Safeguard against transaction interference Database Recovery Restore database to earlier consistent state
- 2. The concept of Transaction Action(s) by user or program to read/write in the database Logical unit of work against a database either done entirely or not even a bit of it Consist of SQL query and/or programming instructions
- 4. The concept of Transaction States of Transaction
- 5. The concept of Transaction ACID Properties of Transaction Atomicity : All or Nothing Consistency : Transform database from one consistent state to next Isolation : Independent of each other Durability : Permanent effects
- 6. Concurrency Control Database: Shared Data Multiple transactions, concurrent access, potential interference Multiple reads, No problem Multiple reads, at least one write: Potential interference Concurrency: Managing concurrent access to avoid interference
- 7. Transactional Interference: Potential Problems Lost Update
- 8. Transactional Interference: Potential Problems Uncommitted Dependency(Dirty Read)
- 9. Transactional Interference: Potential Problems Inconsistent Analysis
- 10. Serializability Schedule:A sequence of the operations by a set of concurrent transactions that preserves the order of the operations in each of the individual transactions. Serial Schedule:A schedule where the operations of each transaction are executed consecutively without any interleaved operations from other transactions. Nonserial Schedule:A schedule where the operations from a set of concurrent transactions are interleaved.
- 11. Serializability Two transactions—same data item—only read (No Problem) Two transaction—different data items—read/write (No problem) Two transactions—same data items—either of them write (Potential Problem, the order matters) Serializable Schedule: If the interleaved operations of the two concurrent transactions produce the same results, are called seriazable schedule.
- 12. Concurrency Control: Serializability (a) And (b) are two equivalent serializable schedules (c) Is the serial schedule
- 13. Concurrency Control: Recoverability Serializability ● Serialiazable schedules maitain consistency ● Assumption: No failure ● Potential Problem: Irrecoverable Schedule Irrecoverable Schedule
- 14. Concurrency Control: Recoverability Recoverable Schedule If ITEM(a) was updated by Transaction Ti and latter on read by Tj, then Ti should commit prior to Tj. Concurrency Control Techniques • Locking • Time Stamping • Optimistic Techniques
- 15. Concurrency Control Techniques: Locking An item accessed/updated by one transaction may be denied access by another transaction. Locking • Shared Lock(read): can only read • Exclusive Lock(write): can do both read and write • System support for upgrade/downgrade of locks
- 16. Two-Phase 2PL Locking Protocol All the locking operation in a transaction must precede the first unlock Growing Phase:A lock is required as soon as a data item is accessed. May be read or write.All locks are secured. No Unlock Shrinking Phase: No new lock could be acquired after first unlock. Locks are only released. Upgrade allowed only in growing phase Downgrade allowed only in shrinking phase
- 17. Two-Phase 2PL Locking Protocol Preventing Lost Update Problem
- 18. Two-Phase 2PL Locking Protocol Preventing Uncommitted Dependence Problem
- 19. Two-Phase 2PL Locking Protocol Preventing Inconsistent Analysis Problem
- 20. Two-Phase 2PL Locking Protocol Creating Cascade rollback Problem
- 21. Two-Phase 2PL Locking Protocol Possible Solution to cascade rollback Rigorous 2PL: Release all unlock at the end Strict 2PL: hold only exclusive unlocks till end
- 22. Deadlock:A locking problem When two(or more) transactions wait for each other to release their corresponding locks. Problem: Deadlock
- 23. Deadlock:A locking problem Solution: Rollback certain transaction(s) and restart User should be unaware of deadlock and solution Solution: Timing Deadlock Prevention Deadlock detection and recovery
- 24. Deadlock: Solutions Timing • System defined time slice • If transaction timed out, aborted and restarted automatically • transaction may not necessarily be in deadlock • Simple protocol, used by many commercial DBMSs
- 25. Deadlock: Solutions Deadlock Prevention • Two solutions by proposed by Rosenkrants et. al. (1978) • Timestamp assigned to each transaction • Wait-Die: older transaction wait for younger • If younger request lock hold by older, younger aborted(die), restarted with same timestamp (eventually gets oldest) • Wound-Wait: younger wait for older • If older request a lock hold by younger, younger is aborted(wounded) Conservative 2PL • Acquire and release all locks at once • Advantage if lock contention is heavy: No blocking, no wait • Low Contention: Locks are held longer • High lock setting overheads: Must release all locks even if single of them not granted. • Not practical: Advanced knowledge of locks required
- 26. Deadlock: Solutions Deadlock detection and Recovery •TiTj shows Ti is dependent on Tj •Shows Tj hold a resource required by Ti •Deadlock exists if WFG contains circle TiTjTk •Frequency of deadlock detection • Too large: deadlock undetected • Too small: time waisted • Dynamic approach Wait-for-graph
- 27. Deadlock: Solutions Recovery • Choice of deadlock victim • Transaction that has been running the long • How many dataitems have been updated • How many dataitems to update • How far to rallback • Avoiding starvation • The same transaction is the victim repeatidly • Use a counter to count number of time a transaction rollbacked • If reach upper limit, use different protocol
- 28. Timestamping (Another concurrency control protocol) Timestamp: A unique identifier, represent the relative starting time/order of the transactions -- System clock or logical counter is used Timestamping: Older transactions get priority incase of conflict A read and write by a transaction on a data item is allowed only if the preceding update to the data item was made by older transaction
- 29. Timestamping ( Timestamping continues…) A transact T has timestamp ts(T) A dataitem x has read timestamp as read_timestamp(x) and write timestamp as write_timestamp(x) Transaction T wants read x Allowed only if ts(T)>write_timestamp(x) otherwise an older(earlier) transaction is trying to read a value updated by younger(newer) transaction Older transaction is too late, rollbacked and restarted with new time stamp Set read_timestamp=max(ts(T), read_timestamp(x)
- 30. Timestamping ( Timestamping continues…) Transaction T wants write x Allowed only if ts(T)>read_timestamp(x) and ts(T)>write_timestamp(x) If ts(T)<read_timestamp(x) then younger (newer) transaction has already read it and is using it and older is late in updating Similarly if ts(T)<write_timestamp(x) then T is trying to update x to an obsolete value In both cases Restart T with later timestamp Otherwise the transaction can proceed Timestamping is serializable, but not recoverable
- 31. Timestamping(Thomas’s Write Rule) Transaction T wants write x Allowed only if ts(T)>read_timestamp(x) and ts(T)>write_timestamp(x) If ts(T)<read_timestamp(x) then younger (newer) transaction has already read it and is using it and older is late in updating Similarly if ts(T)<write_timestamp(x) then T is trying to update x to an obsolete value In first case Restart T with later timestamp, as before In the second case simply ignore update, called Ignore Obsolete Write Rule Otherwise the transaction can proceed
- 32. Optimistic Techniques Conflict (interaction between transactions) is rare, is the basic premise No conflict checking, No delays Efficient policy where conflicts are less frequent Before commit, check for conflict, rollback if found Very efficient: No locks, no concurrency checks According to premise less transaction rollback Intolerable in environment where conflicts are frequent Choose another concurrency control
- 33. Optimistic Techniques Three phases in OT Read Phase Extends from start to commit All values are read and stored in locally Changes are made to local variables Validation Phase Checks serializibilty not violated, database remain consistent Restart transaction if conflict occurred, restart Write Phase If update transaction, apply changes to database stored locally
- 34. Optimistic Techniques Assign timestamps start(T), validation(T), finish(T) to each transaction T Validation is passed only if All earlier should finish before T i.e. finish(E)<start(T) If finish(E)>start(T) then Data items written by E are diff than read by T (Writes done serially) Start(T)<finish(E)<validation(T)
- 35. Granularity for Dataitems The size of data item used as unit of protection
- 36. Granularity of Data item The size of data item used as unit of protection Granularity has greater performance implications on concurrency control algorith There is a tradeof between coarse vs fine granularity E.g. Granularity is not the same for updating a single record vs 80% records of a table Coarse granularity, low degree of concurrency, low locking information maintenance Fine granularity, High degree of concurrency, more locking information maintenance A better approach, mixed granularity, upgrade and downgrade of locks
- 37. Database Recovery Restore database to correct state incase of failure DBMS is resilient if it is fault tolerant
- 38. Storage MediaTypes Volatile Primary Memory, random access, fast, but expensive Non-volatile online secondary memory disk storage, random access Other non volatile offline secondary storages: MagneticTape and Optical Disk Suit only backup, slow, MT sequential access
- 39. Types of Failures System crash Media failure Application software errors Natural physical disasters Carelessness Sabotage All failures involve either main memory or disk copy
- 40. Transactions and Recovery Unit of recovery is transaction Recovery manager guarantee atomicity and durability Database buffer complicate the issue Durability guarantees when database buffer flushed Committed transactions may not reach the database Buffer flushed either when full or forced written
- 41. Transactions and Recovery (continue) In the event of failure oActive transactions(incomplete) udone, i.e. rollbacked oCommitted tranactions are redone, called rollforward oPartial undo, when single transaction roll back oGlobal undo, when all active transactions rollbacked
- 42. Transactions and Recovery (continue) Example transactions rollback/rollforward
- 43. Buffer Management Pages brought in as soon as requested When buffer is full, old pages replaced with new ones Page replacement policies: FIFO, LRU Two var associated with each page: pinCount, Dirty On each request pinCount is incr, also called pinned Decr by the system when done Pinned pages can’nt be replaced Write to disk if Dirty is set, on replacement For new page Dirty is set to 0
- 44. Buffer Management (Continued…) When writing pages two policies used o Steal Policy o Pinned pages could be stolen from the transaction, i.e. written to disk oAlternative is no-steal o Force Policy o Dirty pages are immediately written to disk on committ o Alternative no-force No steal and force is simple to implement With no-steal no rollback, with force no rollforward Steal and force Steal obviate the need for large buffer space no-force provide opportunity for later transaction to update and then write
- 45. Recovery Facilities o Backup Mechanism o Logging (also called journaling) o Checkpointing o Recovery meneger Backup Offline storage of data and log files Used if database is distroyed or damaged Tacken at regular interval Either complete or incremental Backup Incr is changes after last full/incr backup
- 46. Log/Journal File Only for insert and update Type of Record Only for delete and Update NextRecordof thistransaction
- 47. Log/Journal File (Continues…) Important in both recovery and performance • Log file is some times duplexed or triplexed • for performance, log stored on separate physical drive • backup log file where log data is huge • minor failures recovered from online log in short time • Major failure from offline log
- 48. Checkpoint The point of synchronization b/w data and log files Buffers are force written Force write all committed and active transactions Also a check point record is written(consist of IDs of Active transactions) During Recovery o Rollforward transactions with commit record after the last checkpoint o Rollback transactions without commit record
- 49. Recovery Techniques Two types • Major recovery if database file damaged • Restore last back • Apply changes from log file after last backup • Assumption: Log file not damaged, separate storage • Minor recovery such as after system crash • Rollforward/rollback certain transactions • Use the before and after image in log file • Two protocols deferred update and immediate update are used
- 50. Recovery Techniques(continues…) Deferred Update Don’t write until commit, no undoing if aborted Requires redoing committed transaction Log File Use 1. Write start record at start 2. During write, write log record except before image, don’t write anything to buffer or database 3. If transaction commit, write commit log record, also record changes to database buffer/database 4. If transaction abort, do nothing, just ignore log record During Recovery Only rollforward, repeated failures, write operations idempotant
- 51. Recovery Techniques(continues…) Deferred Update Immediately record every change, need undoing if aborted But still requires redoing committed transaction Log File Use 1. Write start record at start 2. During write, write log record, both before and after image 3. After writing log, now record the changes to buffer 4. Actual changes will reach database when buffer next flushed 5. If transaction commit, write commit to database buffer 6. If transaction abort, undo required, use before image log 7. Write-ahead log protocol is a must During Recovery Both rollforward, using after-image, and rollback, using before-image
- 52. Recovery Techniques(continues…) Shadow Paging o Log-less protocol o Maintains two page tables for a transaction, current page table and shadow page table o Both are the same in start o Shadow never changed, used for recovery o changes recorded to current o After transaction complete, current page becomes shadow Advantages No log no log overheads faster recovery, no undo/redo Disadvantage Data fragmentation Periodic garbage collection of inaccessible blocks
Editor's Notes
- #14: &lt;number&gt;