NVRAM-aware Logging in Transaction Systems. Jian Huang

Transcription

1 NVRAM-aware Logging in Transaction Systems Jian Huang Karsten Schwan Moinuddin K. Qureshi

2 Logging Support for Transactions 2

3 Logging Support for Transactions ARIES: Disk-Based Approach (TODS 92) Write-ahead Logging 2

4 Logging Support for Transactions ARIES: Disk-Based Approach (TODS 92) + Write-ahead Logging Rollback 2

5 Logging Support for Transactions ARIES: Disk-Based Approach (TODS 92) + + Write-ahead Logging Rollback Centralized Log Buffer 2

6 On the Logging Persistence lock TXN_A: TXN_B: commit (flushing log) unlock acquire lock wait for lock Sync Commit commit (flushing log) unlock lock 3

7 On the Logging Persistence lock TXN_A: TXN_B: commit (flushing log) unlock acquire lock wait for lock Disk I/O Is the Bottleneck! Sync Commit commit (flushing log) unlock lock 3

8 On the Logging Persistence lock TXN_A: TXN_B: commit (flushing log) unlock acquire lock wait for lock Disk I/O Is the Bottleneck! Sync Commit commit (flushing log) unlock lock TXN_A: TXN_B: lock unlock commit (flushing log) lock unlock commit (flushing log) Async Commit 3

9 On the Logging Persistence TXN_A: TXN_B: lock commit (flushing log) wait for lock acquire lock unlock Sync Commit commit (flushing log) unlock lock Disk I/O Is the Bottleneck! TXN_A: TXN_B: lock unlock commit (flushing log) lock unlock commit (flushing log) Async Commit At the risk of partial data loss and incorrect result! 3

10 Rethink the Logging with NVRAM Close to DRAM 4

11 Rethink the Logging with NVRAM + Close to DRAM Byte Addressability 4

12 Rethink the Logging with NVRAM + + Close to DRAM Byte Addressability Non-Volatility 4

13 Rethink the Logging with NVRAM + + Close to DRAM Byte Addressability Non-Volatility Disk-based logging Avoid frequent disk access Obtain sequential access pattern 4

14 Rethink the Logging with NVRAM + + Close to DRAM Byte Addressability Non-Volatility Disk-based logging Flash-based logging Avoid frequent disk access Obtain sequential access pattern Reduce log flushing time Atomic Write (OSDI 08) 4

15 Rethink the Logging with NVRAM + + Close to DRAM Byte Addressability Non-Volatility Disk-based logging Flash-based logging NVRAM-based Avoid frequent disk access Obtain sequential access pattern Reduce log flushing time Atomic Write (OSDI 08) 4

16 How to Use NVRAM in DB System? Transactions Log Buffer (DRAM) Page Cache (DRAM) Log and DB Tables (HDD/SSD) Traditional Design 5

17 How to Use NVRAM in DB System? Transactions Transactions Log Buffer (DRAM) Page Cache (DRAM) Log Buffer (DRAM) Page Cache (DRAM) Log and DB Tables (HDD/SSD) Traditional Design Log and DB Tables (NVRAM) All-in-NVRAM 5

18 How to Use NVRAM in DB System? Transactions Transactions Log Buffer (DRAM) Page Cache (DRAM) Log Buffer (DRAM) Page Cache (DRAM) Log and DB Tables (HDD/SSD) Traditional Design Log and DB Tables (NVRAM) All-in-NVRAM Transactions Log Buffer (NVRAM) Page Cache (DRAM) DB Tables (HDD/SSD) NVLogging 5

19 How to Use NVRAM in DB System? Transactions Transactions Log Buffer (DRAM) Page Cache (DRAM) Log Buffer (DRAM) Page Cache (DRAM) Log and DB Tables (HDD/SSD) Log and DB Tables (NVRAM) Traditional Design All-in-NVRAM Which one we should take? Transactions Log Buffer (NVRAM) Page Cache (DRAM) DB Tables (HDD/SSD) NVLogging 5

20 Using NVRAM in Cost-Effective Way 6

21 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram all-in-ssd db-in-ssd, log-in-nvram TPCC TATP TPCB 6

22 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram x all-in-ssd db-in-ssd, log-in-nvram TPCC TATP TPCB 6

23 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram all-in-ssd db-in-ssd, log-in-nvram 18% TPCC TATP TPCB 6

24 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram all-in-ssd db-in-ssd, log-in-nvram 18% 26% TPCC TATP TPCB 6

25 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram all-in-ssd db-in-ssd, log-in-nvram 18% 26% TPCC TATP TPCB 6

26 Normalized TPS (all-in-hdd as the baseline) Using NVRAM in Cost-Effective Way all-in-hdd all-in-nvram db-in-hdd, log-in-nvram all-in-ssd db-in-ssd, log-in-nvram 18% 26% Bridging the performance gap with less cost TPCC TATP TPCB 6

27 Percentage Bottlenecks Shifted from I/O to Software 100% Misc DB Operations Lock Manager Log Contention Log Operations 80% 60% 40% 20% 0% log-in-hdd, db-in-nvram log-in-ssd, db-in-nvram log-in-nvram, db-in-hdd log-in-nvram, db-in-ssd all-in-nvram 7

28 Percentage Bottlenecks Shifted from I/O to Software 100% Misc DB Operations Lock Manager Log Contention Log Operations 80% 60% 40% 20% 0% log-in-hdd, db-in-nvram log-in-ssd, db-in-nvram log-in-nvram, db-in-hdd log-in-nvram, db-in-ssd all-in-nvram 7

29 Percentage Bottlenecks Shifted from I/O to Software 100% Misc DB Operations Lock Manager Log Contention Log Operations 80% 60% 40% 20% 0% log-in-hdd, db-in-nvram log-in-ssd, db-in-nvram log-in-nvram, db-in-hdd log-in-nvram, db-in-ssd all-in-nvram Software-induced overhead becomes the new bottleneck! 7

30 NVLogging: Redesign the Logging with NVRAM Transactions NVLogging NVRAM Page Cache (DRAM) DB Tables HDD/SSD 1 2 Use NVRAM in Cost-Effective Way Avoid the Software Overhead 3 Minimal Code Modifications 8

31 Log Management in NVLogging DRAM NVRAM Log Entry Index 9

32 Log Management in NVLogging DRAM NVRAM Log Entry Index 9

33 Log Management in NVLogging DRAM NVRAM Log_head Log Entry Index Log_tail 9

34 Log Management in NVLogging DRAM NVRAM Log Entry Index Log_head Log_tail States Pointer Log Entry 0: DRAM 1: NVRAM LogRec object 9

35 Log Management in NVLogging LogRec DRAM NVRAM Log Entry Index Log_head Log_tail States Pointer Log Entry 0: DRAM 1: NVRAM LogRec object 9

36 Log Management in NVLogging LogRec Processor cache and DRAM are volatile. DRAM NVRAM Log Entry Index Log_head Log_tail States Pointer Log Entry 0: DRAM 1: NVRAM LogRec object 9

37 Log Management in NVLogging LogRec Log persistence: flush-on-insert & flush-on-commit DRAM NVRAM Log Entry Index Log_head Log_tail States Pointer Log Entry 0: DRAM 1: NVRAM LogRec object 9

38 Log Management in NVLogging Log persistence: flush-on-insert & flush-on-commit DRAM LogRec NVRAM Log Entry Index Log_head Log_tail States Pointer Log Entry 0: DRAM 1: NVRAM Atomic Update 9

39 Avoid Log Contention with Per-Transaction Logging DRAM NVRAM Log Entry Index States Pointer Log Entry 0: DRAM 1: NVRAM 10

40 Avoid Log Contention with Per-Transaction Logging LSN #0 LSN #2 LSN #3 TX_A LogRec LogRec LogRec TX_B LSN #1 LogRec LSN #4 LogRec DRAM NVRAM Log Entry Index States Pointer Log Entry 0: DRAM 1: NVRAM 10

41 Avoid Log Contention with Per-Transaction Logging LSN #0 LSN #2 LSN #3 TX_A LogRec LogRec LogRec TX_B LSN #1 LogRec LSN #4 LogRec DRAM No Cross-space Pointer NVRAM Log Entry Index States Pointer Log Entry 0: DRAM 1: NVRAM 10

42 Avoid Log Contention with Per-Transaction Logging LSN #0 LSN #2 LSN #3 TX_A LogRec LogRec LogRec TX_B LSN #1 LogRec LSN #4 LogRec DRAM NVRAM Log Entry Index Recovery Using back pointer to access log objects 10

43 Avoid Log Contention with Per-Transaction Logging LSN #0 LSN #2 LSN #3 TX_A LogRec LogRec LogRec TX_B LSN #1 LogRec LSN #4 LogRec DRAM NVRAM Log Entry Index Log Truncation 10

44 Implementation and Experimental Setup Shore-MT + Shore-Kits Benchmark Transactions NVLogging NVRAM Page Cache (DRAM) DB Tables HDD/SSD 11

45 Implementation and Experimental Setup Shore-MT + Shore-Kits Benchmark Transactions NVLogging NVRAM Page Cache (DRAM) DB Tables HDD/SSD Intel Persistent Memory Emulator Platform 2.6 GHz, 64 GB DRAM GB NVRAM 11

46 Throughput (K TPS) Benefits on Throughput with TPCB db-in-hdd, log-in-nvram db-in-ssd, log-in-nvram Distributed Logging NVLogging DRAM NVRAM Latency (nanoseconds) 12

47 Throughput (K TPS) Benefits on Throughput with TPCB db-in-hdd, log-in-nvram db-in-ssd, log-in-nvram Distributed Logging NVLogging 83% DRAM NVRAM Latency (nanoseconds) 12

48 Throughput (K TPS) Benefits on Throughput with TPCB db-in-hdd, log-in-nvram Distributed Logging 83% db-in-ssd, log-in-nvram NVLogging DRAM NVRAM Latency (nanoseconds) NVLogging improves TPS by 2.3x, compared to the baseline (db-in-ssd, log-in-nvram) 12

49 Normalized TPS/ Benefits on Cost Savings all-in-nvram Distributed Logging NVLogging NVRAM Cost Ratio ( ) Disk 13

50 Normalized TPS/ Benefits on Cost Savings all-in-nvram Distributed Logging NVLogging NVRAM Cost Ratio ( ) Disk 13

51 Normalized TPS/ Benefits on Cost Savings all-in-nvram Distributed Logging NVLogging NVRAM Cost Ratio ( ) Disk 13

52 Conclusion Transactions NVLogging NVRAM Page Cache (DRAM) DB Tables HDD/SSD 1 2 Cost-effective Solution for DB system Higher TPS/ Per-transaction Logging with NVRAM Exploit NVRAM s both non-volatility and byte-addressability 14

53 Thanks! Jian Huang Karsten Schwan Moinuddin K. Qureshi Q&A