The Multi-streamed Solid-State Drive

Transcription

1 The Multi-streamed Solid-State Drive Jeong-Uk Kang, Jeeseok Hyun, Hyunjoo Maeng*, and Sangyeun Cho Memory Solutions Lab. Memory Division, Samsung Electronics Co., Ltd

2 SSD as a Drop-in Replacement of HDD SSD shares a common interface with HDD The block device abstraction paved the way for wide adoption of SSDs Host Application Host Application OS File System SATA OS File System Generic Layer Logical Address Generic Layer HDD SSD

3 Great, BUT Rotating media and NAND flash memory are very different! Host Host Application OS Sector base Application OS File System File System Generic Layer Logical Address Generic Layer HDD Read_Sector() Write_Sector()? SSD Read_() Write_() Erase_() Copy_() Disk NAND Flash Memory NAND Flash Memory NAND Flash Memory

4 The Trick is FTL! Flash translation layer (FTL) Logical block mapping Bad block management Garbage Collection (GC) Host Application OS (Sector based) File System Generic Layer SSD FTL Write Erase Read NAND Flash Memory

5 Garbage Collection (GC) GC reclaims space to prepare new empty blocks NAND s erase-before-update requirement Valid page copying followed by an erase operation Has a large impact on SSD lifetime and performance A A B Valid ERASED data Valid ERASED data2 Invalid ERASED data Invalid ERASED data B Invalid ERASED data Invalid ERASED data Valid ERASED data3 Valid ERASED data4 Valid page copying Free block Valid data Valid data2 Valid data3 Valid data4

6 GC is Expensive! Cassandra Update Throughput (ops/sec) Valid s copied (ops/sec) Performance of SSD gradually decreases as time goes on Example: Cassandra update throughput.2 GC overhead 3. Throughput GC highly affects the SSD performance! Time (Minutes)

7 Our Idea: Multi-streamed SSD Host Application OS File System Co-exists with the existing block layer Generic Layer General & concrete interface New interface for SSD Multi-streaming Interface SSD FTL Host-provided stream information guides desirable data placement within SSD! NAND Flash memory

8 End Result Update Throughput (ops/sec) The multi-streamed SSD can sustain Cassandra update throughput.2 Proposed Traditional SSD Time (Minutes)

9 Contents Background Write optimization in SSD The Multi-streamed SSD Our approach Case study Evaluation Experimental setup Results Conclusion

10 Effects of Write Patterns Previous write patterns (=current state) matter LBA 7 LBA 2 LBA 0 LBA 7 LBA 2 LBA 0 LBA 0 LBA 3 LBA LBA 0 LBA 3 LBA LBA LBA 6 LBA 2 LBA LBA 6 LBA 4 LBA 4 LBA 5 LBA 3 LBA 4 LBA 5 LBA 7 Sequential LBA updates into 2 Random LBA updates into 2 Need valid page copying from 0 & Just erase 0

11 Stream SSD Write to stream Lifetime Stream Data Lifetime? Write to stream 2 Lifetime 2 Stream 2 Write to stream 3 Lifetime 3 Stream 3

12 The Multi-streamed SSD Multi-streamed SSD Mapping data with different lifetime to different streams Data Host Data2 StreamID Multi-stream Generic interface Layer Data3 Data5 Application Data4 Data0 Provide information about data lifetime Data3 Data Data3 Stream ID = SSD FTL NAND Flash Memory Data2 Data7 Data9 Stream ID = Data0 Data2 Data3 Stream ID = 3 Place data with similar lifetime into the same erase unit

13 Working Example Multi-streamed SSD High GC efficiency (Reduce GC overheads) effects on Performance! Request data Request data Without Stream Reduce valid pages to copy Multi-Stream For effective multi-streaming, proper mapping of data to streams is essential!

14 Case Study: Cassandra Flushing Cassandra employs a size-tiered compaction strategy Write Request Memtable Memory Commit Log SSTable 2 SSTable 5 K K2 SSTable 6 SSTable 7 SSTable SSTable 2 SSTable 3 SSTable 4 K K2 K K2 K

15 Summary of Cassandra s Write Patterns Write operations when Cassandra runs Memtable Memory Flushing data Commit-log Write Commit Log System data Write SSTable 2 SSTable 5 K K2 SSTable 6 SSTable 7 metadata, journal SSTable SSTable 2 Compaction data write SSTable 3 SSTable 4 System K K2 K K2 K

16 Mapping #: Just one stream ID (= conventional SSD) Memtable Memory 0 Flushing data 0 Commit-log Write Commit Log 0 System data Write SSTable 2 SSTable 5 K K2 SSTable 6 SSTable 7 metadata, journal SSTable SSTable 2 Compaction data write SSTable 3 SSTable 4 0 System K K2 K K2 K

17 Mapping #2: Multi-App Add a new stream to separately handle application writes (stream ID ) from system traffic (stream ID 0) Memtable Memory Flushing data Commit-log Write Commit Log 0 System data Write SSTable 2 SSTable 5 K K2 SSTable 6 SSTable 7 metadata, journal SSTable SSTable 2 Compaction data write SSTable 3 SSTable 4 System K K2 K K2 K

18 Mapping #3: Multi-Log Use three streams; further separate Commit Log Memtable Memory 2 Flushing data Commit-log Write Commit Log 0 System data Write SSTable 2 SSTable 5 K K2 SSTable 6 SSTable 7 metadata, journal SSTable SSTable 2 Compaction data write SSTable 3 SSTable 4 2 System K K2 K K2 K

19 Mapping #4: Multi-Data Give distinct streams to different tiers of SSTables Memtable Memory 2 Flushing data Commit-log Write Commit Log 0 System data Write SSTable 2 Compaction data write 4 SSTable 5 K K2 SSTable 6 SSTable 7 metadata, journal SSTable SSTable 2 Compaction data write SSTable 3 SSTable 4 3 System K K2 K K2 K

20 Experimental Setup Multi-stream SSD Prototype Samsung 840 Pro SSD 60 GB device capacity YCSB benchmark on Cassandra Write intensive workload K data x,000,000 record counts 00,000,000 operation counts Linux kernel 3.3 (modified) Passes the stream ID through fadvise() system call Stores in the inode of VFS Application VFS fadvise (fd, Stream ID) inode field = Stream ID Intel i GHz processor 2 GB Memory Accelerates SSD aging by increasing Cassandra s flush frequency EXT4 Device Store Stream ID In buffer head SSD

21 Results Update Throughput (ops/sec) Cassandra s normalized update throughput TRIM off (TRIM off) Time (Minutes)

22 Results Update Throughput (ops/sec) Cassandra s normalized update throughput TRIM on.2 But still far from ideal 0.8 (TRIM on) TRIM gives non-trivial improvement Time (Minutes) (TRIM off) 27/37

23 Results Update Throughput (ops/sec) Cassandra s normalized update throughput Multi-App (System data vs. Cassandra data).2 Multi-App 0.8 (TRIM on) Time (Minutes) (TRIM off)

24 Results Update Throughput (ops/sec) Cassandra s normalized update throughput Multi-Log (System data vs. Commit-Log vs. Flushed data).2 Multi-Log (TRIM on) Multi-App Time (Minutes) (TRIM off)

25 Results Update Throughput (ops/sec) Cassandra s normalized update throughput Multi-Data (System data vs. Commit-Log vs. Flushed data vs. Compaction data).2 Multi-Data Multi-Log (TRIM on) Multi-App Time (Minutes) (TRIM off)

26 Result #2 Cassandra s GC overheads TRIM off (TRIM off) Time (Minutes)

27 Results Valid page copied (ops/sec) Cassandra s GC overheads TRIM on (TRIM on) (TRIM off) Time (Minutes)

28 Results Valid page copied (ops/sec) Cassandra s GC overheads Multi-App (System data vs. Cassandra data) Multi-App (TRIM off) (TRIM on) Time (Minutes)

29 Results Valid page copied (ops/sec) Cassandra s GC overheads Multi-Log (System data vs. Commit-Log vs. Flushed data) Multi-App (TRIM off) (TRIM on) Multi-Log Time (Minutes)

30 Results Valid page copied (ops/sec) Cassandra s GC overheads Multi-Data (System data vs. Commit-Log vs. Flushed data vs. Compaction data) Multi-App The throughput is very well correlated with GC overheads 0.2 (TRIM off) (TRIM on) Multi-Log Multi-Data Time (Minutes)

31 Result #3 Cumulated distribution (%) Cassandra s cumulated latency distribution Multi-streaming improves write latency At 99.9%, Multi-Data lowers the latency by 54 % compared to Normal 00 Multi-Data Multi-Log Multi-App (TRIM on) Latency (us)

32 Conclusion Multi-streamed SSD Mapping application and system data with different lifetimes to SSD streams Higher GC efficiency, lower latency Multi-streaming can be supported on a state-of-the-art SSD and co-exist with the traditional block interface Host Multi-stream enhanced Layer Multi-stream interface can be standard for using SSD more efficiently SSD FTL NAND Flash memory

33