File Systems for Flash Memories Marcela Zuluaga Sebastian Isaza Dante Rodriguez
Outline Introduction to Flash Memories Introduction to File Systems File Systems for Flash Memories YAFFS (Yet Another Flash File System)
Introduction to Flash Memories Memories technology classification
Introduction to Flash Memories Flash Memories: main features Non-volatile: : it does not need power to maintain the information stored in the chip. Fast read access times (though not as fast as volatile DRAM) ) and better shock resistance than hard disks. Hence, suitable for data storage in embedded devices.
Introduction to Flash Memories features Unlike EEPROM,, it is erased and programmed in blocks consisting of multiple locations Costs far less than EEPROM Examples of applications include USB Flash drives, digital audio players, digital cameras and mobile phones
Introduction to Flash Memories Two types NOR Flash Random, direct access interface for readings Fast random reads Slow erase and write Suitable for code storage NAND Flash Random access is not supported, only sequential Lower cost (than NOR) Smaller size erase blocks Better performance for erase and write More endurance Mainly for data storage
Introduction to Flash Memories Disadvantages Random access for rewrite or erase not supported Finite number or erase/write cycles Higher cost per bit compared to hard disks drives
Introduction to File Systems Subsytem of an OS, whose purpose is to provide long- term storage Set of abstract data types ADT (set of data and operations) Storage hierarchical organization, manipulation, navigation, access, and retrieval of data Make use of an underlying data storage device Access to an array of fixed-size blocks (sectors, generally 512 bytes) File system software is responsible for organizing these sectors into files and directories,, and keeping track of which sectors belong to which file.
Introduction to File Systems Types of File Systems Disk file systems Storage of files on a data storage device (disk drive). Exp: FAT, NTFS, HFS, ext2, ISO 9660, ODS-5,, and UDF. Database file systems Non hierarchical structured management Identification by their characteristics, like type of file, topic, author. metadata
Introduction to File Systems PC File Systems FATFS File Allocation Table File System - Developed by Microsoft for MS-DOS - Supported by virtually all existing operating systems - Main drawback is fragmentation. HPFS High Performance File System - Efficient placements of the allocation tables and root directories - Filenames can be up to 254 chars long NTFS New Technology File System - Several improvements over FAT such as improved support for metadata - Advanced data structures to improve performance, reliability and disk space utilization Ext2 second extended file system - Native filesystem for Linux - Maximum data size of 4 terabytes - Maximum filename length of 255 characters
Introduction to File Systems FAT s File Systems FAT12 FAT16 FAT32 Developer Full Name Introduced Microsoft File Allocation Table (12-bit version) (16-bit version) (32-bit version) 1977 (Microsoft Disk BASIC) July 1988 (MS-DOS 4.0) August 1996 (Win 95 OSR2) 0x0B, 0x0C (MBR) Partition identifier 0x01 (MBR) 0x04, 0x06, 0x0E (MBR) EBD0A0A2-B9E5 B9E5-4433 Max file size 32 MiB 2 GiB -87C0-68B6B72699C7 4 GiB Max number of files 4,077 65,517 268,435,437 Max filename size 8.3 or 255 characters when using LFNs Max volume size 32 MiB 2 GiB 4 GiB with some implementations 2 TiB
Introduction to File Systems Storage Level of abstraction for the HW responsible for storing and retrieving Specified blocks of data. Abstraction given by block device drivers.
Each file contents: File contents (data) Introduction to File Systems File Contents File attributes (metadata) File size Owner, access control lists Creation time, last access time, last modification time, File name
Introduction to File Systems Data Scattering File system: A mapping problem <filename, data, metadata> <a set of blocks>
Introduction to File Systems Ext2 file system File System Example - A disk-based file system for Linux - Similar to UNIX Fast File System (FFS) - Evolved to Ext3 File system (with journaling) - Directory: pathname metadata (i-node) - Direct/indirect block pointers: i-node i data blocks
Introduction to File Systems Unix File System Superblock Size description of the FS Inode Metadata, pointers Datablock Allocation for the FS
File Systems for Flash Memories Due to the particular characteristics of Flash memories, special file systems are needed to spread writes over the media (wear-levelling levelling) and deal with the long erase times of NOR flash blocks. Two approaches for FFS (Flash File System): Layered approach Native (or cross-layer) layer) approach
File Systems for Flash Memories Layered Approach Uses the Fast Translation Layer (FTL) to fully emulate a magnetic disk and includes: Sector mapping Garbage collection Power-off off recovery Bad block management Wear-leveling leveling Error correction code (ECC) Power management
File Systems for Flash Memories Layered Approach (2) Benefits Easy to deploy because no modification is required for upper layers Some commercial Flash memories already come with FTL. Limitations Most FTLs are patented Kernel is not aware of the presence of flash memory.
File Systems for Flash Memories Native Approach Cross-layer optimization Kernel manages raw flash memory directly More opportunities to optimize the performance Kernel is involved in some FTL functionalities Sector mapping, garbage collection, wear-leveling, leveling, power-off off recovery, etc Example: Flash-aware aware file systems: JFFS/JFFS2, YAFFS Limitations Need to change the host operating system Only applicable Flash-embedded devices
YAFFS Yet Another Flash File System
YAFFS History Toby Churchill Ltd (TCL) needed a flash filing system for their devices reliable with fast boot time. First option: adding NAND support to the existing flash file systems, JFFS, JFFS2 boot time and ram consumption were problem adding NAND support wasn't trivial The company Aleph One decided to design a different filing system explicitly for NAND
YAFFS - History..Progress Decided to create YAFFS - Dec 2001 Working on RAM emulation - March 2002 Working on real NAND (Linux) - May 2002 WinCE version - Aug 2002 uclinux use - Sept 2002 Linux rootfs - Nov 2002 psos version - Feb 2003 Shipping commercially - Early 2003
YAFFS Why NAND flash? Embedded and mobile systems are increasingly using NAND flash for storage because it has various advantages over other storage technologies NOR flash is not very dense (i.e. not much storage per chip), is costly and is slow to write NAND flash is low cost, dense, and writes fast; but it has other limitations It is needed a journaling file system that is able to work around the limitations of NAND and exploit it for best performance. Has a fast erase time as compared with NOR flash. The NAND physical interface is very simple The small size and low current requirements make it very suitable for embedded systems
YAFFS Other NAND characteristics NAND is not random access, but page oriented. Thus, we do all reads & writes in pages NAND writes will only change 1 bits to 0. The only way to get 1s again is to erase the entire block. Errors are far more likely in NAND devices. Blocks of memory can be bad when the device is shipped and further blocks can become unusable over time. Therefore error detection and correction is highly desirable. YAFFS handles bad blocks and uses formats which are resistant to corruption.
YAFFS YAFFS Characteristics The first NAND-specific flash file system Special reference to embedded systems primarily for internal NAND rather than removable NAND (SM cards) reliability is more important than compatibility. Exploits low-cost NAND chips and is both fast and robust Highly portable to other operating systems Open source project
YAFFS YAFFS Characteristics It is design based in JFFS although NOR and NAND flash have very different properties A flash file system that works with NOR incorporates various mechanisms that are not required for NAND, and NAND needs extra mechanisms not required for NOR The design was greatly simplified by not including compression Robustness through journaling strategies. Significantly reduce the RAM overheads and boot times associated with JFFS. YAFFS uses a physical flash format similar to SmartMedia.
YAFFS File System Design File data stored in chunks, same size as flash pages (512 bytes) Block - Erasable set of pages (usually 32) Each file has an id - equivalent to inode. Chunks numbered 1,2,3,4 etc - 0 is header. Each flash page is marked with file id and chunk number These tags are stored in the OOB - 64bits: including file id, chunk number, write, serial number, tag ECC and bytes-in in-page-used
YAFFS File System Design On overwriting the relevant chunks are replaced by writing new pages with new data but same tags - the old page is marked discarded File headers (mode, id, length etc) get a page of their own (chunk0) Pages also have a 2-bit 2 serial number - incremented on write Allows crash-recovery recovery when two pages have same tags (because old page has not yet been marked discarded ). A block containing only discarded pages (termed a dirty block) ) is an obvious candidate for garbage collection. YAFFS maintains a tree structure in RAM memory of the physical location of these chunks.
YAFFS Page allocation Pages are allocated sequentially from the currently selected block. When all the pages in the block are filled, another clean block is selected for allocation. If there are insufficient clean blocks available, then a dirty block is erased to free it up as a clean block. If no dirty blocks are available, then the dirtiest block is selected for garbage collection.
YAFFS Garbage collection Garbage collection is performed by copying the valid data pages into new data pages thus marking all the pages in this block dirty and freeing it up for erasure. Another idea: selecting a block at random some small percentage of the time - thus reducing the chance of wear differences. Relative to NOR, NAND writes and erases very fast. Therefore garbage collection might be performed on- demand (during a write)
YAFFS YAFFS 2 The original motivation for YAFFS 2 was to add support for the new NAND with 2kB pages instead of 512-byte byte pages and strictly sequential page writing order. To achieve this, a new design is used which also presents the following benefits: zero page rewrites means faster operation. (YAFFS1 uses a single rewrite in the spare area to delete a page). ability to exploit simultaneous page programming on some chips. improves performance relative to YAFFS1 speed(write:1.5x to 5x, delete: 4x, garbage collection: 2x) lower RAM footprint (approx. 25% to 50% of YAFFS1). The main philosophical difference between YAFFS and YAFFS2 is how discarded status is tracked. Since re-writing writing cannot be done, "discarded" flags are not used. Now the strategy is including more tag information.
YAFFS File System Limits YAFFS 2^18 files (>260,000) 2^20 max file size (512MB) 1GB max file system size YAFFS2 8GB max file system size
References www.wikipedia.com www.aleph1.co.uk http://labs.google.com/papers/gfs-sosp2003.pdf sosp2003.pdf http://www.yale.edu/pclt/boot/ifs.htm http://learnlinux.tsf.org.za/courses/build/internals/ch08s04.html http://hjem.get2net.dk/rune_moeller_barnkob/filesystems/ http://linux.org.mt/article/filesystems