Logging and Recovery. Logging Recovery Archiving

Transcription

1 Logging and Recovery Logging Recovery Archiving

2 Recovery: Failsafe systems Failures and correctives Erroneous data entry Constraints, triggers Media failure: e.g.,bit failures, head crash Catastrophic failure: destruction of media System failure: loss of transaction state by e.g., power loss or software errors RAID Archives Redundant online copies of DB Archives Redundant online copies of DB Logging and recovery Protect data against system failures Goal: resilience (data integrity in system failures) 12.2

3 Logging and Recovery: Logging Precondition: Logging of data in normal DB use Logging: gathering and storing data which enable a system to recover from failures Log-file or log: - sequence of log records with information about interleaving transactions - Append-only written by log manager Log types - Short term log (temporary log): for recovery from transactional failures - Archive log: for long term recovery from an archive copy 12.3

4 Logging and Recovery: Recovery Recovery: - After system crash restore consistent DB state using the log-file - Some transaction have to be performed again - Some transactions have to be undone State after recovery: - latest transaction consistent state before failure 12.4

5 Logging and Recovery: Transaction failures Transaction paradigm requests - All-or-nothing of transactions (atomicity) - Durability of successful data changes Transaction failures - Occur most frequently - Have to be recovered fast and carefully in order to guarantee transactional properties Short term Logging/Recovery techniques - Undo Logging, Redo Logging, Undo/Redo - Checkpoints 12.5

6 Logging and Recovery: Components Components involved: Query Processor Transaction Manager Log Manager Data Log Buffer Manager Recovery Manager Commands: Log record to disk: Flush-log command from log manager to buffer manager to copy all new log blocks to disk Buffer to disk: Output command from transaction manager to buffer manager to write buffered data to disk 12.6

7 Logging and Recovery: Undo Logging Rules for transactions - If transaction T modifies element x: write log-entry <T,x,v> before new values of x is written to disk - If transaction T commits:write COMMIT log-entry immediately after all changed elements are written to disk Write to disk order: 1. Log records indicating database element changes 2. Changed database element 3. Commit log record DB state old DB state new DO Log record 12.7

8 Logging and Recovery: Undo Logging Example Step Action t Copy of A in buffer Copy of B in buffer Copy of A on disk Copy of B on disk Log 1 start T 2 R(A,t) 3 t:=t*2 4 W(A,t) <T,A,> 5 R(B,t) 6 t:=t*2 7 W(B,t) <T,B,> flush log output(a) output(b) flush log Commit T 12.

9 Logging and Recovery: Undo Recovery Recovery - Divide transactions into committed and uncommitted ones - Committed: <start T> and <commit T> exists - Uncommitted: <start T> exists, no <commit T> - Recovery manager scans log file backwards Actions: 1. If T is committed do nothing 2. else: change value of X to v 3. For each incomplete DB state write <abort T> in log new 4. Flush log UNDO DB state old Log record 12.9

10 Logging and Recovery: Undo Recovery Example Step Action R(A,t) t:=t*2 W(A,t) R(B,t) t:=t*2 W(B,t) flush log output(a) output(b) flush log t A in buffer B in buffer A on disk B on disk Log start T <T,A,> <T,B,> Commit T - Crash after step 12: ignore log records for T - Crash after step 10: undo T - Crash after step : undo T - Crash prior to step : if changes reached the disk undo T 12.10

11 Logging and Recovery: Undo-Checkpoints Recovery examines entire log No delete of information about committed TAs in log - Interleaving transactions might be lost Checkpoint log periodically: 1. Stop accepting new transactions 2. Wait until all current TAs commit or abort 3. Flush log to disk 4. Write <ckpt> to log, flush again 5. Resume accepting transactions Recovery backwards until checkpoint Delete or overwrite log-file before checkpoint 12.11

12 Logging and Recovery: Redo Logging Differences to undo-logging - Ignores incomplete TAs (undo: cancels) - Repeats changes from committed TAs (undo: ignores) - Write commit before values (undo: write before commit) - Recover used new values (undo: recover with old values) Rule (write-ahead logging rule) - Before modifying element x on disk write all log entries regarding this modification to disk 12.12

13 Logging and Recovery: Redo Logging Example Step Action t Copy of A in buffer Copy of B in buffer Copy of A on disk Copy of B on disk Log 1 start T 2 R(A,t) 3 t:=t*2 4 W(A,t) <T,A,> 5 R(B,t) 6 t:=t*2 7 W(B,t) <T,B,> flush log output(a) output(b) Commit T 12.13

14 Logging and Recovery: Redo Recovery Recovery - Identify committed transactions - Recovery manager scans log file forward Actions: 1. If T is not committed do nothing 2. else: write value v of X 3. For each incomplete write <abort T> in log 4. Flush log DB state old Log record REDO DB state new 12.14

15 Logging and Recovery: Redo Recovery Example Step Action R(A,t) t:=t*2 W(A,t) R(B,t) t:=t*2 W(B,t) output(a) output(b) flush log t A in buffer B in buffer A on disk B on disk Log start T <T,A,> <T,B,> Commit T - Crash after step 9: write new A and B back - Crash after step : if commit to disk: proceed as above if commit not to disk: proceed as below - Crash prior to step : no changes to A and B, write <abort> 12.15

16 Logging and Recovery: Redo-Checkpoints With Redo limited search in log not possible Checkpoint log: 1. Write <start ckpt (T 1,..T n )> to log (T i active TAs), flush log 2. Write new elements to disk (from committed TAs) 3. Write <end ckpt> to log, flush again Recovery, crash after <end ckpt> : Committed TAs before <start ckpt> not considered Recovery of all other TAs (start with T earliest from T i ) Recovery, crash after <start ckpt> : Search back to last <end ckpt> Proceed as above 12.

17 Logging and Recovery: Undo/Redo Logging Drawbacks of Undo and Redo - Undo requires immediate data writing after transaction ends increases number of I/Os - Redo requires to keep modified blocks in buffer until commit and flush increase of #buffers required by TAs Undo/Redo-Logging Log records with four components <T,x,old_value,new_value> Undo/Redo Rule: Before modifying element x on disk first write update record <T,x,old_value,new_value> to disk 12.17

18 Logging and Recovery: Undo/Redo Logging Example Step Action 1 t Copy of A in buffer Copy of B in buffer Copy of A on disk Copy of B on disk Log start T 2 R(A,t) 3 t:=t*2 4 W(A,t) <T,A,,> 5 R(B,t) 6 t:=t*2 7 W(B,t) <T,B,,> flush log output(a) output(b) Commit T 12.1

19 Logging and Recovery: Undo/Redo Recovery Recovery - Redo all committed transactions in earliest-first order - Undo all incomplete transactions in last-first order Example: Step Action t A in buffer B in buffer A on disk B on disk R(A,t) t:=t*2 W(A,t) R(B,t) t:=t*2 W(B,t) flush log output(a) output(b) - Crash after <commit T>: write new value of A and B to disk - Crash prior to <commit T>: undo A and B, if necessary Log start T <T,A,,> <T,B,,> Commit T 12.19

20 Logging and Recovery: Influences Other components influence logging and recovery Buffer (cache) management - When to write data back (before/after EOT, any time) Implementation of writing into the DB - In-place update - Indirect write to DB Other considerations - Locking (granularity) - Group commit for better performance - Direct/transaction consistent/fuzzy checkpoints - Log file size and management 12.20

21 Logging and Recovery: Archiving Goal: Protection against media failures Logs not sufficient (grows faster than DB) Keep separate archive copy of DB Archiving levels: - Full dump copy of entire DB - Incremental dump copy of changed data - Several levels in between Problem: 24/7 applications not allow for shutdown for archiving 12.21

22 Logging and Recovery: Archiving Archiving: - Archive database state in fixed order - State changes during archiving activity - Also archive log of that time Recovery requires dump and log during archiving: Reconstruct DB from most recent dump Modify DB according to following incremental dumps Modify DB using surviving logs 12.22

23 Logging and Recovery: Short summary System Failure protection: Transaction logging Undo logging and recovery - Log before write - Log Commit after write - Recovery: change data of aborted TAs back Redo logging and recovery - Log before write - Recovery: rewrite data of committed TAs Checkpoints: - Limit necessary log entries - Limit number of changes/aborts in case of failure Media failure protection: archiving - Incremental archiving for better performance - 24/7 archiving: archives+logs needed 12.23