Recovery: Write-Ahead Logging EN 600.316/416 Instructor: Randal Burns 4 March 2009 Department of Computer Science, Johns Hopkins University
Overview Log-based recovery Undo logging Redo logging Restart recovery Checkpoints
Goals Provide high-availability (fast repair) and consistency in the presence of failures Transaction failures logical error: internal condition, bad input, data not found, resource problems, etc. Correspond to software faults. system error: deadlock or other system problem that prevents this execution of a transaction rollback: explicit call to fail this transaction by application System crash bug in OS or DB hardware problems environmental reasons: power implement the failstop property
The Database Log Whatʼs a log? A sequential file that contains a record of actions taken by an entity Entity is the DBMS, actions are data writes Whatʼs a log look like? A contiguous region of storage, preferably a whole disk Why contiguous? Why a whole disk?
The Database Log Whatʼs a log? A sequential file that contains a record of actions taken by an entity Entity is the DBMS, actions are data writes Whatʼs a log look like? A contiguous region of storage, preferably a whole disk Why contiguous? So that sequential log writes are written sequentially to storage. Writing sequentially on storage is very important, as opposed to seeking Why a whole disk? The storage is not important. The arm is the valuable resource. So that no other workload shares the disk (arm) resource and interferes with sequential writing.
Memory/Storage View of Log Updates are made in memory Log may be inconsistent with on disk log Flush the log to make consistent
Physical/Logical View of Log Logically infinite Complete history of all modifications to the DB Implemented in finite storage Tail = present Head = meaningful past
What does the log contain? Log records (Duh!): an update record describes a single database writes and consists of: Transaction ID Data-item identifier (typically physical, disk location, rather than logical, tuple) Old value New value < TID, Disk Block, Old, New > There are other types of entries Transaction begin and end, checkpoints, rollbacks
The Logging Principle Logically, the log only grows, nothing is ever deleted If you want to undo some update < TID, Block, Old, New > place an equivalent undo in the log < TID(rollback), Block, New, Old > How do we implement a log in a fixed amount of space? Transactions that are resolved (aborted or committed) can be written to the main database, shrinking the log from its head. We never delete from the tail and we remove records for unresolved transactions Problem: long running transactions can overflow the log Yup you bet.
Whatʼs What The log: non-volatile storage Writes to the log are I/O The DB: non-volatile storage Writes to the DB are I/O In-memory pages of the DB Updates (not writes) are made to an in-memory copy of blocks of the DB. Pages and blocks are the same-ish thing. Pages in memory, blocks on disk. 1-to-1 correspondence.
4215 page b Database Cache 3155 page p 4219 page q 4217 page z Log Buffer 4220 begin(t 20 ) nil 4219 write(q,t 17 ) 4217 Volatile Memory Stable Storage Stable Database 4215 page b 3155 page p 2788 page q 4158 page z (page/log) sequence numbers Stable Log 4218 commit(t 19 ) 4216 4217 write(z,t 17 ) 4215 4216 write(q,t 19 ) 4199 4215 write(b,t 17 ) From: Weikum and Vossen, Figure 13.1... 13-11 4208
Deferred Modification (Redo) Log entries allow operations to be redone <TID, Block, New> Updates are written only to the log By written, we mean I/O to the DB image. The updates are applied to an in-memory copy of the database page Once a transaction commits, i.e. it has written a commit record in the log, all of the updates may be applied to the database
Restart Recovery No volatile information, just a database and log. No idea what transactions were active. General form of recovery Analysis, take a forward pass over the log constructing lists of committed and aborted/active transaction transactions active when a failure occurred are aborted during restart recovery
Redo Recovery Log can be used to restore committed updates that did not make it to the database After a failure
Redo Recovery Log can be used to restore committed updates that did not make it to the database After a failure After recovery
Immediate Modification (Undo) The log contains update entries that allow operations to be undone <TID, Block, Old> Updates are written (I/O) into the database and log
Undo Recovery Log can be used to undone updates to DB from uncommitted transactions After a failure
Undo Recovery Log can be used to undone updates to DB from uncommitted transactions After a failure Recovery
Undo Recovery Log can be used to undone updates to DB from uncommitted transactions After a failure Recovery completed
Steal and Force Steal, the ability to write an uncommitted update to disk This is called stealing a page Frees up system resources Requires undo logging Force, the requirement that all pages dirtied by a transaction are on disk when a transaction commits No force requires redo logging
Steal No Force
Undo and Redo Redux Undo logging is steal/force Redo is no-steal/no-force What workloads are these good for?
Undo and Redo Redux Undo logging is steal/force Redo is no-steal/no-force What workloads are these good for? Redo Small memory footprint (canʼt steal pages) High txn rates (commit only to log) OLTP, E-Commerce Undo Can handle memory overflow (steal) Flush at commit expensive Long-running write-oriented transactions?
Undo/Redo The best of both worlds Steal from undo and no-force from redo At the expense of Recovery complexity and space in the log
Undo/Redo Recovery Cʼ doesnʼt need to get hardened to disk Why not?