Flash Memory. Jian-Jia Chen (Slides are based on Yuan-Hao Chang) TU Dortmund Informatik 12 Germany 2015 年 01 月 27 日. technische universität dortmund

Transcription

1 12 Flash Memory Jian-Jia Chen (Slides are based on Yuan-Hao Chang) TU Dortmund Informatik 12 Germany 2015 年 01 月 27 日 These slides use Microsoft clip arts Microsoft copyright restrictions apply Springer, 2010

2 Why Flash Memory Diversified Application Domains Portable Storage Devices Consumer Electronics Industrial Applications Critical System Components - 2 -

3 Layout of Flash Memory 0 63 Block: basic eraseoperation unit 0 (2KB + 64 Byte) (2KB + 64 Byte) (2KB + 64 Byte) (2KB + 64 Byte) Page: basic writeoperation unit 1023 (2KB + 64 Byte) (2KB + 64 Byte) 128MB Flash Memory - 3 -

4 Characteristics of Flash Memory Write-Once No writing on the same page unless its residing block is erased! Pages are classified into valid, invalid, and free pages Bulk-Erasing Pages are erased in a block unit to recycle used but invalid pages Wear-Leveling Each block has a limited lifetime in erasing cycles Eg, 10,000 ~ 100,000 erase cycles for each block - 4 -

5 Terminology Valid data: the latest version of data stored in flash Invalid data: not the latest version of data stored in flash Live page: a page that stores valid data Dead page: a pages that stores invalid data Free page: a page that is erased and is ready to store data Free block: a block that is erased and is not allocated to store any data Hot data: frequently updated data Valid hot data might become invalid in the near future Cold data: non-frequently updated data Valid cold data might stay in the same place for a long time - 5 -

6 Management Issues Flash-Memory Example 1: Out-place Update Characteristics A B C D A B Dead pages - 6 -

7 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics L D D L D D L D L L D L L L F D L F L L L L D F F L L F L L F D This block is to be recycled (3 live pages and 5 dead pages) A live page A dead page A free page - 7 -

8 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics D D D D D D D D L L D L L L L D Live data are copied to somewhere else L F L L L L D L L L L F L L F D A live page A dead page A free page - 8 -

9 Management Issues Flash-Memory Example 2: Garbage Collection Characteristics F F F F F F F F The block is then erased L L D L L L L L F L L L L D D L Overheads: live data copying block erasing L L L F L L F D A live page A dead page - 9 -

10 Management Issues Flash-Memory Example 3: Wear-Leveling Characteristics L D D L D D L D L L D L L L F D A B Wear-leveling might interfere with the decisions of the blockrecycling policy L F L L L L D F F L L F L L F D C D A live page A dead page A free page Erase cycle counts

11 Single-Level Cell (SLC) vs Multiple-Level Cell (MLC) A limited bound on erase cycles SLC : 100,000 SLC Flash MLC x2 : 10,000 Bit error probability 64 SLC: 10-9 MB MLC: 10-6 MLC Flash NUMBER OF CELLS 1 0 a) BILEVEL (1 BIT/CELL) LOGIC STATE V T 128 MB NUMBER OF CELLS b) MULTILEVEL (2 BIT/CELL) LOGIC STATE V T

12 MLC vs SLC (Cont) Electronic Engineering Times, July

13 Price and Read/Write Performance NOR NAND SLC NAND MLC x2 Price 3455 $/GB 679 $/GB 248 $/GB Read 2384 MB/sec 1533 MB/sec 135 MB/sec Write 007 MB/sec 457 MB/sec 234 MB/sec Erase 022 MB/sec 8533 MB/sec MB/sec *NOR: Silicon Storage Technology (SST) NAND SLC: Samsung Electronics K9F1G08Q0M NAND MLCx2: ST STMicroelectronics [1,2] 1 Jian-Hong Lin, Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo, and Cheng-Chih Yang, "A NOR Emulation Strategy over NAND Flash Memory," the 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Daegu, Korea, August 21-24, Yuan-Hao Chang and Tei-Wei Kuo, A Log-based Management Scheme for Low-cost Flash-memory Storage Systems of Embedded Systems

14 Flash Memory Management Management issues Write constraints imposed by flash memory Scalability issue Garbage collection Performance considerations Reliability issues Cell error rate problem imposed by MLC flash memory Error correction coding vs wear leveling Read/write disturbance problem Data retention problem

15 Typical System Architecture Application 1 Application 2 Application n fwrite (file, data) Shared Library Virtual File System (VFS) Native Flash File System Layer (eg, JFFS2, YAFFS) File-system Management Unit Allocator Address Translation / Block Assignment File Systems (eg, DOS FAT) Flash Translation Layer (FTL) Allocator Address Translation / Block Assignment flash write (block, page) Control single Cleaner Garbage Collection Wear Leveler Cleaner Garbage Collection Memory Technology Device Layer (MTD) Flash Memory Wear Leveler

16 Flash Translation Layer (FTL) FTL adopts a page-level address translation mechanism The main problem of FTL is on large memory space requirements for storing the address translation information Access LBA = 3 Logical Block Address (array index) Physical Block Address (block,page) Address Translation Table (in main-memory) (1,3) (1,2) (2,1) (1,0) (4,7) (0,4) (0,6) (0,1) (0,3) Physical Block Address (block,page) 1,3 1,2 1,1 1,0 0,7 0,6 0,5 0,4 0,3 0,2 0,1 0,0 User data Spare data Flash memory Spare data LBA=3; ECC= ; Status= ;

17 NAND Flash Translation Layer (NFTL) A logical address under NFTL is divided into a virtual block address and a block offset eg, LBA=1019 => virtual block address (VBA) = 1019 / 8 = 127 and block offset = 1019 % 8 = 3 Write data to LBA=1019 VBA=127 NFTL Address Translation Table (in main-memory) (9,23) Block Offset=3 A Primary Block Address = 9 Free Free Free Used Free If the page Free has been Free Write to the used Free first free page Free Free Free A Replacement Block Address = 23 Used Used Used Free Free

18 FTL or NFTL FTL NFTL Memory Space Requirements Large Small Address Translation Time Short Long Garbage Collection Overhead Less More Space Utilization High Low The memory-space requirements for one 1GB NAND (2KB/Page, 4B/Table Entry, 128 Pages/Block) FTL: 2MB (= 4*(1024*1024*1024)/2K) NFTL: 32KB (= 2*4*(1024*1024*1024)/(2K * 128))

19 Size of Translation Tables 1GB 32GB 1TB 32TB FTL 2MB 64MB 2GB 64GB NFTL 32KB 1MB 32MB 1GB No matter which kind of granularity of address translation is adopted, the fast growing flash memory capacity would eventually make the translation table too large to be fitted in RAM

20 Adaptive Two-Level Mapping Mechanism The coarse-grained hash table maintains the primary block and its replacement block This will be mainly used to identify the primary and replacement blocks The fine-grained hash table has limited entries to store the excessive live pages in the replacement block When the fine-grained table is full, the replacement policy has to be considered to move some pages to the coarse-grained table Improve the space utilization The delayed recycling of any primary block lets free pages of a primary block be likely used in the future A Fine-Grain Hash Table A Coarse-Grain Hash Table AFTL (A, RPBA + 5) (B, RPBA + 7) ( δ VBA, δ PPBA, δ RPBA) cg cg δ cg δ cg cg A Primary Block ( δcgppba ) A B C D E A Replacement Block ( δ cg RPBA ) A B A B A A B B Chin-Hsien Wu and Tei-Wei Kuo, 2006, An Adaptive Two-Level Management for the Flash Translation Layer in Embedded Systems, IEEE/ACM 2006 International Conference on Computer-Aided Design (ICCAD), November 5-9,

21 New Write Constraints of MLC Flash Write constraints Pages can only be written sequentially in a block Partial page write/programming is prohibited Impact on NFTL Data can t be written to any free page of primary blocks The space utilization in primary blocks is even lower Most writes are forced to be placed in the replacement block Pages of invalid data can t be marked as dead Each read operation should scan pages of the replacement block

22 Intuitive and Practical Solution Level-paging translation tables Pages of translation tables are stored in flash, and are cached in RAM Problems: Hit ratio of cached pages Extra page reads and writes for translation information Crash recovery for translation table and lost data 9 bits Level 0 paging page Level 1 paging page 3 bits Chip 4 bytes 2nd outerpage table LBA (4 bytes) 9 bits 4 bytes PBA (4 bytes) 17 bits Block empty Outer-page table 2 9 bits bits Level 2 paging page Offset 4 bytes Page table 7 bits Page 256GB Flash memory 1 Page = 2KB 1 Block = 2 7 Pages 1 Chip = 2 17 Blocks Page (2KB)

23 Performance Considerations MLC flash has growing market share: Reason: low cost and high density Drawbacks: low speed, low endurance, and low reliability Solutions: Hardware multi-channel programming Software multi-bank or multi-channel programming

24 Wear Leveling No Wear Leveling Dynamic Wear Leveling Intuitive Perfect Static Wear Leveling Erase Cycles Physical Block Addresses (PBA)

25 Static Wear Leveling Random policy It randomly select a block to reclaim after a fixed number of block erases or write requests It doesn t track the locality of data acceses, such that it might move hot data that might turn dead in the near future

26 Static Wear Leveling (Cont) Random policy with block-erasing table Each-bit flag of the table is to indicate whether the corresponding blocks have been erased Whenever the block erases are not even enough, select blocks whose corresponding bit flag are not set Pros and Cons: Pros: it can identify the locality of data accesses Cons: the block-erasing table needs extra RAM space even it is comparatively small Yuan-Hao Chang, Jen-Wei Hsieh, Tei-Wei Kuo: Endurance Enhancement of Flash-Memory Storage, Systems: An Efficient Static Wear Leveling Design DAC 2007:

27 The Block Erasing Table (BET) A bit-array: Each bit is for 2 k consecutive blocks Small k in favor of hot-cold data separation Large k in favor of small RAM space e cnt =2 =3 =0 =1 f cnt =2 =0 =1 Flash e cnt =1 =0 =2 =3 =4 f cnt =1 =0 = BET k=0 k=2 e cnt : the total number of block erases done since the BET is reset f cnt : the number of 1 s in the BET : an index to a block that the Cleaner wants to erase : a block that has been erased in the current resetting interval

28 A SimpleStatic-Wear Leveler An unevenness level (e cnt / f cnt ) >= T à Triggering of the SW Leveler Resetting of BET when all flags are set T: A threshold, T=1000 in this example e cnt : the total number of block erases since the BET is erased e cnt =1998 =1999 =2000 =2004 =2998 =2999 =3000 =3004 =3999 =4000 =0 f cnt =2 =3 =4 = f cnt : The number of 1 s in the BET : An index that SW Leveler triggers the Cleaner to do garbage collection : An index in the selection of a block set : An index to a block that the Cleaner wants to erase : A block that has been erased in the current resetting interval k=2 The Cleaner is triggered to Reset to a randomly 1 Copy valid data selected block set free area, 4000 After a / 4 period = 1000>= Erase block in the selected of of time, selected block the (ethe block set cnt total / total f set, cnt erase >=1000) erase and , 3 count but Inform all / reaches 2 3 the flags = Allocator 1000 in (flag) 3999 BET >= 2998 to update 1000 are 1à the (E address reset cnt / f cnt >= mapping BET T) between LBA and PBA

29 Main-Memory Requirements 512MB 1GB 2GB 4GB 8GB k=0 256B 512B 1024B 2048B 4096B k=1 128B 256B 512B 1024B 2048B k=2 64B 128B 256B 512B 1024B k=3 32B 64B 128B 256B 512B MLC x2 (1 page = 2 KB, 1 block=128 pages)

30 File-System Considerations - Observations Archive: Linux-2617 source Number of files 19,535 Number of directories 1,200 Average file size 11 KB Archive size 215,666 KB FAT is the default file system for removable storage devices Write the files to a removable storage device File system NTFS FAT Number of write requests 24, ,670 Time taken (min:sec) 4:33 54:21 Read the files from a removable storage device File system NTFS FAT Number of read requests 14,568 23,528 Time taken (min:sec) 2:53 3:19 FAT file systems introduce excessive write requests! why?

31 File-System Considerations - Observations (Cont) Layout of FAT filesystem File #1 FAT #1 FAT #2 Directory Entry Content of File LBA 0 Directory Entry FAT #1 FAT #2 File #2 Directory Entry Content of File Directory Entry

32 File-System Considerations Applications Card Readers Operating System API File System Drivers Host Disk Drivers Device Drivers Filter Drivers USB Flash Drives Device Device Controller Storage Media Bus Drivers Host Device Device Controller Storage Media Flash Memory Cards (a) (b)

33 Overview When? Notifications Read and Write IoPackets USB Mass Storage Device Driver I/O Request Dispatch Unit Other IoPackets I/O Response Cohesivecaching algorithm Filter Driver Filesystem Identifier File System Layout Partition Table & Boot Sectors Cache FIFO Queue I/O Request Transport Unit Debug Unit I/O Response Trace Debug Viewer USB Bus Driver

34 The Allow/Prevent Medium Removal Operation Code in the USB Bulk-Only Transport <CBW> Prevent Medium Removal <CSW> Success FAT File System hrtimerc intermodulec itimerc kallsymsc Disk Driver 1 st FAT 2 nd FAT Directory Entry File Content Directory Entry Disk W W W W W W W W USB Mass Storage Class, Bulk-Only Transport Protocol CBW: Command Block Wrapper CSW: Command Status Wrapper USB Mass Storage Device Driver <CBW> Write10 LBA=1446 LEN=1 <DATA> <CSW> Success USB Bus An IoPacket Send a notification <CBW> Allow Medium Removal <CSW> Success

35 Worsening Reliability - Narrow Threshold Voltage Window MLC/TLC/QLC technology must squeeze the available window of threshold voltage for each logical state Higher Bit Error Rate Lower Endurance SLC MLC Source:

36 Reliability Issues The low-cost MLC flash Has lower erase cycles Has higher and higher bit error rate Ways to improve reliability Error correction coding (ECC) à passively Non-erasure code such as BCH and RS Wear leveling à positively Distribute block erases as even as possible Note: Erasure code is used in communication Non-erasure code is used in storage systems

37 Error Correction Coding Error correction code of a page is stored in its spare area The time on error correction might be affected by the location of error bits or the number of error bits The space of the ECC hardware V is increased as the number of t supported error bits increases Trend of MLC flash The page size is getting larger The bit error rate is getting higher Fast erasing bits The fast worn-out flash cells

38 Data Retention Problem The guaranteed data retention: 10 years As the cell size is getting smaller, the number of electrons in the floating gate of a flash cell is getting smaller For example: A programmed cell can store 10,000 electrons A lost of only 10% in this number can lead to a wrong read à A loss of less than 2 electrons per week can be tolerated