Flash for Databases. September 22, 2015 Peter Zaitsev Percona

Transcription

1 Flash for Databases September 22, 2015 Peter Zaitsev Percona

2 In this Presentation Flash technology overview Review some of the available technology What does this mean for databases? Specific opportunities for MySQL 2

3 Before SSDs

4 There were HDDs Good at Sequential Read/Writes RT=Seek Time + Rotation Latency Reads/Write Similar Latency No Specific Write Limits Retain data for a long time One IO Request in Parallel Low cost per GB 4

5 RAID and SAN

6 Using Many HDDs together Caching Reads Buffering Writes (Writeback Cache) Better Sequential Read/Write speed Better throughput at high concurrency Higher IO latencies for uncached IO 6

7 Flash Revolution Use Flash chips instead of platters No moving parts No seeks 7

8 NAND Flash Cell Page/Read Block Erase Block Write but no overwrite Wears with writes (erases) 8

9 Writing to the Flash Erase Set all bits to Write Change Zero to one Set some of the bits to 0: Impossible. Do Erase, when Write 9

10 Types of NAND Flash From AnandTech: 10

11 Flash Storage Design Cache Battery/Super Capacitor Controller + Complex Firmware Built-in Parallelism 11

12 Flash Controller and Firmware Tasks Write wear leveling Garbage collection Error correction Bad block mapping Read scrubbing Read disturb management Encryption 12

13 Flash Properties Lots of IOs per device! (100K+) Less random IO penalty Writes more expensive than reads (but can be faster) Limited by amount of writes Limited retention Concurrent execution on single device Fast write acknowledgement (safe or not) Can burst writes 13

14 Flash Interface Designs DIMM PCI-E SFF-8639 SATA/SAS FC and Network 14

15 Transitioning AHCI NVMe 15

16 AHCI vs NVMe Source: AnandTech.com 16

17 Some Product Examples Products and Leaders are changing quickly 17

18 Sandisk ULLtraDIMM 18

19 HGST Virident 19

20 Sandisk FusionIO 20

21 Intel P3x00 21

22 Intel

23 Intel 730 (SATA) 23

24 msata 24

25 M.2 Interface 25

26 Violin Memory 26

27 Consumer vs Enterprise Performance Endurance Durability Retention Encryption 27

28 Not your HDD All HDDs are the same; All SSDs are different 28

29 Evaluation Performance changes over time Empty Space Matters Complex internals Watch stability carefully 29

30 How Flash Fails Clear write amount defined EOL (but often can handle a lot more) One day it s gone Power Loss Protection Internal ECC and redundancy 30

31 To RAID or not to RAID? More valuable for consumer grade Watch for good Flash support RAID controller logic may slow things down Use a redundant array of inexpensive servers instead? 31

32 Redundancy Device internal redundancy Hardware RAID Software RAID Filesystem RAID 32

33 OS Support Flash support is actively being improved TRIM Sparse Files 33

34 Flash And Databases

35 Database History Most have been designed in HDD time Optimize for sequential IO Count on cheap sequential writes RAID, BBU to improve performance 35

36 It s time for Flash Your OLTP Database should live on Flash 36

37 But What Flash? Pick a flash type that is right for your application 37

38 IO vs Memory 38

39 Warmup Much faster warmup times Even if the database fits in memory, SSD might be justified 39

40 Tolerate more IO bound load HDD 5ms Can do 20 IO/s for 100ms response time (non parallel) Flash 0.1ms Can do 1000 IO/s for 100ms response time (non parallel) 40

41 Endurance Might be a top consideration 41

42 Endurance Math HGST FlashMax III 2200GB 4400GB/day over 5 Years 1400MB/sec peak writes 66 days at peak write throughput Crucial M GB 72TB total life time writes 400MB/sec write 52 hours at peak write throughput 42

43 Databases and Flash How do we optimize databases to us Flash best? 43

44 Storage Engines Innodb TokuDB 44

45 Torn Page problem Flash can avoid this with little cost due to internal design FusionIO NVMFS (Atomic Writes) Copy-on-Write File Systems ZFS BTRFS Filesystem level data journaling less preferred data=journal for EXT4 Skip-Innodb-double-write 45

46 Fast IO Path Bypass Caching O_DIRECT Native Asynchronous IO Efficient Checksuming Innodb_checksum_algorithm=crc32 Innodb_flush_method=O_DIRECT 46

47 IO Cost Accounting Sequential vs Random IO balance IO vs CPU Balance Smaller page sizes might make sense innodb_page_size=4k 47

48 Less Pre-fetching Most pre-fetched data must be used Often best to try It out 48

49 Less merging on flushing Do not assume flushing multiple sequential dirty pages has same cost Innodb_flush_neighbors=0 49

50 Less Space on Disk Innodb Compression (2x typical) TokuDB Compression (5-10x typical) Archiving data off OLTP System 50

51 Less Writes on Flash Hybrid Flash/SSD System Transactional Logs, Other logs on the HDD with RAID and BBU Small Temporary objects on tmpfs Innodb_log_file_size=<LARGE> 51

52 Logs on RAID can be fast 52

53 Single Intel 730 Sysbench 53

54 IOPS 54

55 Consistency 55

56 Is Flash Too Fast? Multiple instances might scale better 56

57 Other Thoughts Host hardware and OS matter, especially with high end flash Virtualization has higher relative overhead Network higher relative overhead 57

58 Thank you!