Technologies Supporting Evolution of SSDs By TSUCHIYA Kenji Notebook PCs equipped with solid-state drives (SSDs), featuring shock and vibration durability due to the lack of moving parts, appeared on the market around 2006. However, single-level-cell (SLC) NAND flash memories were used in SSDs at that time, and it was believed that multilevel-cell (MLC) flash memories, while they could double the storage capacity compared with SLCs, were not applicable because of a lack of reliability. In 2008, Toshiba developed the world's first (Note 1) SSD equipped with MLC NAND flash memories and realizing a low write application factor (WAF), thereby increasing the reliability of MLC SSDs. This was achieved through the analysis of actual PC users' workload in detail, as well as the development of a new NAND cache system and an appropriate NAND management algorithm. MLC NAND flash memories have now become the mainstream for SSDs used in notebook PCs, and are contributing to new values such as a thin and small form factor (SFF), very low standby current, and instant-on capability. 1. Introduction Since the development of NAND flash in 1987, the replacement of hard disk drives (HDDs) with NAND flash memories has been a perennial theme. Recently, as notebook PCs are increasingly equipped with stick-type solid-state drives (SSDs), the full-fledged replacement of HDDs has begun in earnest. In the 1990 s, SSDs were known as silicon disks and flash drives, and they were used in some industrial equipment because of their advantages with respect to shock and vibration resistance thanks to their moving parts-less disign, including motors and pickups. However, NAND flash memories could only be adopted in a limited range of equipment because of their high costs compared to HDDs. In the 2000 s, multi-level-cell (MLC) NAND flash memories (MLC NAND) were developed. This technology has significantly reduced the bit cost of applications using NAND flash memories, such as SD cards and silicon audio, which in turn has led to the rapid adoption of MLC NAND. On the other hand, due to concerns regarding reliability, MLC NAND seemed poorly suited for installation in PCs. Given such circumstances, Toshiba established a project team to further the development of SSDs in 2006. This team was challenged to develop the world s first SSD equipped with MLC NAND (MLC SSD). This required that MLC NAND reliability sufficient for PC use be realized by understanding the characteristics of the cutting-edge MLC NAND technology, as well as that devices be developed with performance exceeding that of the HDDs and those SSDs equipped with single-level-cell (SLC) NAND flash memories (SLC NAND) (SLC SSDs) on the market at that time. In 2008, Toshiba overcame these challenges and developed the world s first MLC SSDs. In this article, I will describe the technical challenges that we have overcome in the development of MLC SSDs and the technologies we adopted to do so. 2. Technological Challenges for MLC SSDs The most significant feature of MLC NAND is that it can support twice as large of a capacity as SLC NAND. In terms of SSDs, this means that the cost of 64 GB or 128 GB products the capacity considered necessary for PC use can be reduced at once to half that of SLC NAND. Although MLC NAND can support twice the capacity of SLC NAND, MLC presented the following challenges: (1) Lower read and write speeds (2) Larger number of error bits when reading data (3) Larger erase data size (4) Less number of write and erase times Of these challenges, a less number of write and erase times was believed to be the largest obstacle to using MLC SSDs in PCs. There were concerns that the number of write and erase times of the memory element, MLC NAND, might reach its limit within three years (the normal lifespan for PCs), thus leading to the loss of stored data. Thus, this was the first challenge we faced in the development of MLC SSDs. Although the number of write and erase times for SLC NAND available at that time was usually around 100,000 data write and erase times, MLC NAND (Note 1) As of February 2008, according to a survey by Toshiba 7
could guarantee only around 3,000 to 5,000, approximately one-twentieth to one-thirtieth that of SLC NAND. We needed to prove that MLC SSDs could withstand the typical use conditions for PCs even with their write and erase times range. 3. Service Life of MLC SSDs MLC SSD service life is nearly equivalent to that of the MLC NAND mounted on them. In practice, however, the amount of data written by the host system to the SSD is not identical to that written to the NAND flash memories mounted on that SSD. This is because NAND flash memories have a larger erase data size than the write data size. Specifically, when saving data smaller than the erase data size, a data write the size of the erase data is performed. This restriction on write operations is referred to as the write amplification factor (WAF), which is an index that indicates how many times the amount of data actually written to NAND flash memories is larger than that written by the host system. For example, if a 200 KB file is edited and saved to a NAND flash memory that has an erase data size of 2 MB, except in special cases, 2 MB of data will be erased and rewritten. For this reason, when the host system makes a write and erase request for 200 KB of data, in reality 2 MB of data is rewritten, which is equivalent to 10 WAF. Therefore, if an MLC SSD which is rewritable for 3,000 times is used each time at 10 WAF, the service life of that MLC SSD will end after the host system makes approximately 300 data write requests. Consequently, we needed to design SSD control algorithms and architectures for ensuring a low WAF in order to prove that MLC SSDs can be used for PCs. This required us to correctly understand the size and frequency of users actual data use as well as the amount of data saved by users per day. 4. User Model In order to design SSD control algorithms and architectures for ensuring a low WAF, we needed to model users actual usage conditions. To this end, we monitored and modeled PC usage at our company. We considered that such PCs were operated longer and wrote more data than those for home use. As a result of monitoring data access by more than 200 users for a total of two months, we found that both the amount of data written and the hibernation amount (Note 2) can be approximated by a log-normal distribution. Figure 1 shows an example of such a distribution of the amount of data written. Assuming a normal distribution for the data, we (Note 2) Hibernation refers to suspending the operating system by writing the system status to an external device, such as an SSD or HDD. The hibernation amount is the amount of data written for this purpose. obtained the following values for the average μ and standard deviation σ. (1) Amount of data written: μ 2.2 GB/day σ 0.5 GB/day (2) Hibernation amount (approx. 1.05 GB) μ 1.2 times/day σ 1 time/day Consequently, we discovered that the typical user in our company writes approximately 4 GB (2.2 GB + 1.05 GB 1.2 times = 3.46 GB) of data to his or her PC per day. 5. Random Access and Sequential Access For a 64 GB MLC SSD, writing approximately 4 GB of data per day uses one-sixteenth of its capacity each day. Even if a PC equipped with an MLC SSD is used every day for three years, the write and erase times will only amount to approximately 70 times, which is sufficiently small in consideration of the write and erase endurance of MLC NAND, which is in the range of 3,000 to 5,000 times. Further, the actual write and erase time of MLC NAND can be derived by multiplying the WAF with this value. The WAF is tightly correlated with the size of the data to be written. When writing data by random access in small units (512 bytes or 4 KB), the WAF will be a very large value such as 1024. On the other hand, even if data is written in the same small units, writing data by sequential access can make the WAF small. For this reason, knowing the size of each piece of the approximately 4 GB of data written per day as well as the access method used, both of which we obtained through monitoring our users, is essential in designing appropriate SSD control algorithms, which determine the service life of MLC SSDs. For this reason, we studied actual access by the host system. As a result of our survey, which was conducted simultaneously with our study of users, we found that the minimum write size of 4 KB, which increases the size of the WAF dra- 8
matically, is used for approximately 66% of the total amount of data written. Further, write sizes of 16 KB or less, which are significantly smaller than the erase data size, are used for 90% of all data writes. In addition, we found that random access was frequently performed. We were unable to obtain any data to facilitate the design of SSD control algorithms. Therefore, we decided to develop the new cache system described in the next chapter to reduce the WAF of MLC SSDs. 6. Cache Systems We found that the host system performs small data random access as its basic data access pattern. At the same time, we discovered that this data access demonstrated highly temporal and spatial locality. We thought it would be possible to design SSD control algorithms for reducing WAF by implementing a cache system and absorbing such small data random access. However, the spatial locality was not sufficiently small to fit into several tens of MB of DRAM, and the temporal locality extended to several days, a period exceeding the length of time for which volatile DRAM can retain cache data. We overcame this problem by implementing a dedicated internal file system in the SSD by using a conventional DRAM cache as well as the NAND itself as a secondary cache (Figure 2). reduced the WAF, which determines the service life of MLC SSDs. Through this technology, Toshiba achieved WAFs of 3 to 6. As a result, we were able to design SSD control algorithms which prevent MLC SSDs from exceeding their write and erase limits during typical use even if several tens of GB of data, rather than just 4 GB of data, is written per day. 7. SSD Performance Many consumers expect SSDs to offer high-speed reads and writes unachievable by HDDs. This feature depends upon the performance of the NAND flash memories installed in the SSDs as well as the physical configuration of such SSDs. Generally, this physical configuration is expressed as the product of the number of channels and the number of banks, such as 4 channels 4 banks, 8 channels 4 banks, etc. 7.1. Read performance The maximum read speed of an SSD is determined by the product of the NAND flash memory interface speed and the number of channels. The logical maximum read speed of our first generation SSDs was 320 MB/s (the interface speed of the NAND flash memories was 80 MB/s and the SSDs had a four-channel configuration). Recently, toggle NAND flash memories have been developed which are capable of achieving 133 MB/s and 200 MB/s NAND flash memory interface speeds, contributing significantly to the development of SATA gen3 (third generation Serial Advanced Technology Attachment) SSDs, whose interface speeds exceed 500 MB/s. 7.2. Write performance The maximum write performance is determined by the product of the NAND flash memory write speed and the number of parallel write chips. If the typical write speed of MLC NAND is 10 MB/s, and if a 16-chip parallel write operation is performed using a 4-channel 4-bank or 8-channel 2-bank configuration, the maximum write performance will be 10 MB/s 16 = 160 MB/s. If power consumption is ignored, a 32-chip parallel operation in an 8-channel 4-bank configuration can achieve a write speed of 320 MB/s, exceeding the 3 Gb/s of SATA gen2. One characteristic of the SSD s internal NAND cache system is the garbage collection feature: only data which needs to be updated is written to the erased area, as overwriting is impossible in the same manner as that performed by conventional DRAM caches and the erased area is secured by efficiently collecting only valid data in the cache. This development of a special cache system for SSDs has greatly 7.3. TRIM command To achieve the above-mentioned maximum write speed, it is necessary to write only the required data to erased blocks. However, real SSDs have few blocks which contain only erased or only valid data. To write data to such SSDs, then, the valid data from multiple blocks must be collected and moved to other blocks before making preparations for writing data by erasing the original blocks. For this reason, write performance degrades in actual use, and in extreme cases, the PC may appear as if it has temporarily frozen. 9
The TRIM command was proposed as a way to solve this problem. If a file is erased, the TRIM command notifies the SSD that the sectors previously occupied by the erased file have been invalidated. By using the TRIM command, invalid data no longer needs to be copied when written, which should have the effect of preventing write speed reductions. However, since the SSD determines the operation taken after receiving a TRIM command, this benefit does not necessarily apply for all SSDs. 8. New Advantages of SSDs By adopting MLC NAND, SSDs have achieved high performance, high reliability, and reduced costs. However, their bit cost is still ten times higher than that of HDDs. For this reason, even today SSDs are mounted in only a limited number of high-end notebook PCs. Given such circumstances, we continue to propose the following new advantages of SSDs. 8.1. Low standby power consumption and instant-on Sine HDDs are equipped with spindle motors, the standby power consumed by HDDs in an active idle state is commonly slightly less than 1 W. By contrast, the standby power consumption of SSDs can be reduced to the 0.1 W level. Although HDDs can also reduce standby power consumption to the 0.1 W level when in a sleep state, this increases their recovery times; HDDs cannot quickly return to an active state as SSDs can. Making use of this new advantage of SSDs, notebook PCs can dramatically extend their battery life, thereby allowing them to differentiate themselves from those models equipped with HDDs. In this way, SSDs allow for a close approximation of the instant-on PC, a long-demanded product. 8.2. Small and thin form factors When first developed, SSDs were enclosed in 2.5-type and 1.8-type cases to ensure compatibility with HDDs. In recent years, however, as a result of the development of large capacity MLC NAND and ball grid array (BGA) packages equipped with eight NAND flash memory chips, 128 GB, 256 GB, and other large capacity SSDs can now be found even in small and thin form factors (Figure 3). As a result, the space required for drives inside notebook PCs has been reduced, consequently enabling longer battery life by increasing battery capacity as well as facilitating the release of ultra-thin stylish notebook PCs, which are equipped with stick-type SSDs. 8.3. Even faster drives For 7,200 rpm HDDs (high-speed HDDs), the SATA gen2 read speed of 3 Gb/s is sufficient. For SSDs, however, read speeds exceeding that of SATA gen3, which is 6 Gb/s, can be realized by making NAND flash memory interfaces faster and increasing the number of parallel operations. Based on this feature, SSDs are expected to incorporate before other devices the high-speed interface standards such as serial-attached small computer system interface (SCSI), commonly known as SAS (12 Gb/s), and Peripheral Component Interconnect Express, commonly known as PCI Express 3.0 (Note 3). As soon as the appropriate interfaces are developed, they will likely be installed in SSDs, contributing to improved performance in the server and storage equipment mainly used by enterprises. 9. Toward the Further Development of SSDs There is no doubt that the SSD market will continue to expand in the future with the introduction of SSD notebook PCs, installations in enterprise servers and storage equipment, and in other forms as the era of cloud computing arrives. Technology for ensuring such development, the reliability improvement functions achieved through such as RAID (Redundant Array of Independent/Inexpensive Disks) will become important. This issue is not unique to SSDs. Systematic methods for implementing data protection functions for unexpected failures that were developed for HDD-based systems in the past are seen as essential. As the miniaturization of NAND flash memories mounted on SSDs progresses year after year, their physical processing becomes increasingly difficult, leading to an increase in frequency of a series of semiconductor-specific errors, including bit errors, errors caused by the write unit, and errors (Note 3) PCI Express is a registered trademark of PCI-SIG. 10
caused by the erase block unit. As a countermeasure against these errors, we have ensured reliability for the write unit by enhancing error check and correct (ECC) functions, such as Bose-Chaudhuri-Hocquenghem (BCH) coding. In addition, by implementing RAID-like measures within the drive, unexpected errors caused by the page, block, and chip units can be handled. This allows us to offer users lower bit costs and higher reliability SSDs equipped with cutting-edge NAND flash memories. Further advances in SSDs will lead to further advances in cloud computing. 10. Conclusion This article gave an overview of MLC NAND-based SSD technologies. SSD interfaces are not limited to SATA and PCI Express. Although the interfaces are different, SD cards used for digital cameras, etc. and the e MMC (Note 4) installed in tablet PCs and smart phones have the same challenges as SSDs (Note 4) e MMC is a trademark of the JEDEC Solid State Technology Association. regarding making full use of NAND flash memories and are based on the same basic NAND control technologies applied to the SSDs explained above. In other words, the technologies supporting the development of SSDs are the technologies for making full use of NAND flash memories, and these technologies are related to those supporting other applications using NAND flash memories. We also expect that new flash memories besides NAND will be available in the future. As described in other articles of this special report, basic research on new flash memories is already underway, and we must be ready to make full use of such new flash memories as drives. Currently, however, multiple methods for achieving new flash memories are being advocated, and there are a wide range of challenges before full use can be made of them as drives. In order to overcome such challenges and develop and achieve practical use of SD cards, e MMC, and SSDs that will support the future era of cloud computing, we will develop technologies in collaboration with our various partners, including universities. TSUCHIYA Kenji Storage Products Div. 11