Extending SSD Simulator to Support Shared Channel between Packages

Transcription

1 Extending SSD Simulator to Support Shared Channel between s Y. A. Winata Graduate Student, Department of Electronic Engineering, S. Jin Graduate Student, Department of Electronic Engineering, I. Shin* Associate Professor, Department of Electronic Engineering, Abstract Several solid state drive (SSD) researches use software simulator since real SSD hardware architecture is difficult to modify. Unfortunately, there are not many open-source SSD simulators in public domain. Moreover, the mostly used simulator which was created by Microsoft research supports only dedicated channel for each package. This is different with current commercial SSDs nowadays. Several commercial SSDs support shared channels to connect between packages. Thus, in this paper, we modify Microsoft research simulator so that it can simulate current commercial SSDs. To make sure our implementation is correct; the modified simulator is configured as dedicated channel SSD and compares the experiment results with original SSD simulator. We further expand our results to see how different SSD architecture affects performance. From the experiment results, it is proven that we have successfully modified Microsoft Research SSD Simulator to supports shared channel between packages. Keywords: shared channel, SSD, simulator, channel, package Introduction Solid State Drives (SSDs) which use NAND flash memory are becoming popular nowadays in both enterprise and workstation machines since it has low power consumption, fast I/O performance, and high shock resistance capability. To increase SSDs performance and size, manufacturers usually employ several packages connected to a channel inside an SSD [1]. Inside the package itself, there can be one or multiple dies. In other words, one channel is connected to several dies. Each die consists of one or multiple planes [2]. Since SSDs use NAND flash memory instead of spinning disk, to ensure the compatibility of SSD in the current file system, SSDs employ flash translation layer (FTL). FTL hides NAND flash memory limitations such as erase-beforewrite [3] and request distribution to packages (i. e. stripping or interleaving [4]). Both hardware (channels, packages, dies, and planes) and software (FTL) architecture is different in each SSD product which leads to diverse performance characteristics. Understanding hardware and software architecture in real SSD is difficult since SSD internals usually is proprietary. Knowing this problem, several researchers develop an open source SSD board for development [5]. However, if the research demands a configurable SSD architecture, using real SSD board is not an option. Since real hardware cannot be modified easily. For alternative, researchers use SSD simulation software instead to simulate real SSD [4, 6]. Based on our study, the simulator which is widely used [4] cannot fully represent current SSD architecture and needs to be adjusted. Moreover, there is no proper documentation of the SSD simulator and makes several researchers have different understanding on how the simulator works. Meanwhile the other simulator [6] is rarely used in research. In this paper, we study the current SSD simulator which is widely used in current researches [4]. This SSD simulator is created by Microsoft Research and integrated with Disksim 4. 0 [7]. We modify the simulator so it supports shared channel between packages. This study also gives information to other researchers that the current simulator is not sufficient for simulating current SSD architecture. Related Works Simulator which is widely used and also used in this study is the simulator by Agrawal et al [4] (which is commonly called as MSRC SSD Extension). For convenience, later in this paper, we call this simulator as SSD extension. This widely used simulator can be used for analyzing different SSD architectures. However, in their study, they simplified the channel operations and all packages have fully interconnection with the controller. This is not represents current SSD architecture because full interconnection means one packages has a dedicated channel. The other simulator which resembles current SSD architecture is research by Dirik and Jacob [6], they changes the both SSD architecture and bus bandwidth. Unfortunately, we cannot find their source code for further study and this simulator is rarely used in current researches. Different with Dirik and Jacob, in our implementation, instead of changing bus bandwidth we assume that bus bandwidth in the channel process only one request at a time. 3915

2 Background SSD Architecture We use SSD architecture from Micron [1, 2] for this study. SSD consists of multiple channels and multiple packages. To connect with host, SSDs use host interface controller such as PCI-Express or SATA. After transferred through host interface, request will arrive at SSD controller. Controller then decides which NAND flash package is used and request will be transferred through channel and way. Current SSD architecture usually uses one channel to connect several NAND flash packages as in Figure 1. Since host interface controller has limited bandwidth, adding more channel does not always increase SSD performance and only increases production cost. Thus, balancing the number of channels and number of packages inside a channel is important. Host Interface Flash Controller ch-1 ch-2 ch-3 ch-4 RAM Figure 1: Current SSD architecture Architecture architecture can be seen in Figure 2. Inside a package, there are one or multiple dies which have either dedicated data line, or shared data line to each die. In other words one channel is connected to several dies. The number of dies which is connected to a channel is different in each SSD product. If we assume channel bandwidth is as wide as page, when a channel sends a page sized request to data register, channel is considered as busy since it cannot send another request. way Inside a die, there are several planes, each of which has data register for read and write operations. A plane consists of several blocks and a block consists of several pages. When same type of requests are issued to a die (i. e. writes only or reads only), planes can work simultaneously. This is called n- plane mode [8]. Since there are several independent units in an SSD, various techniques were studied to increase SSD performance. One of it is superblock [9]; Agrawal et al implement this one as ganging [4]. Superblock and n-plane mode is not our focus and we will leave it for future study. Flash Translation Layer Unlike Hard Disk Drives (HDDs), SSDs uses NAND which has different request unit. If HDDs use unit of a sector for all operations (read/write/erase), SSDs use page as read/write unit, and block as erase unit. Moreover, HDDs allow overwrite while SSDs do not allow overwrite. In SSDs a page needs to be erased before it can be written. There are several other differences between HDDs and SSDs. Hence, to implement SSDs without changing current file system, SSDs implement an intermediate-software-layer called flash translation layer (FTL). To solve the overwrite problem, FTL uses out-of-place update [3] for overwrite request. Instead of erase and write in the same location, FTL writes request to another clean page and mark the old location as invalid. The clean pages allocation algorithm is also different in each SSD product. Clean pages can be allocated in channel level, die level, or plane level [10]. After several out-of-place update, when SSD is lack of clean pages, garbage collection is triggered to obtain clean pages by removing invalid pages. Since unit of erase is a block, FTL selects victim block with the most invalid pages. When there are valid pages inside the victim block, this pages need to be moved to another block before this block can be erased. Because garbage collection consists of reads, writes and erases operation, it usually has high latency. FTL also handles package selection for write request. We call the package selection process as binding algorithm. There are several binding algorithms such as basic address based [4], write order based [6], or modified binding such as state based [11] binding is known. For this paper, we will only use basic address based in our experiment. Plane 0 Plane 1 Plane 0 Plane 1 Die Die Data Register Block 1 Block 3... Block 4095 Plane Figure 2: architecture Page 0 Page 1... Page CH1 CH Figure 3: Stripping and interleaving 3916

3 To optimized parallelism, FTL chopped a big write request into chunks in size of a page. These chunks then distributed to all packages. This process is known as stripping [4]. We also use stripping for this experiment. The other way to improve parallelism is using interleaving [4, 6]. Since several dies might be connected to a channel, previous request must wait until channel is idle before it can be transferred. Stripping and interleaving process is illustrated as in figure 3. Shared Channel Implementation SSD extension does not support shared channel between packages implementation. If packages are connected to a channel the simulation will be similar with superblock architecture (ganging [4]). However, in ganging, if one package is busy, all packages are also busy. This is not true in since each package has its own ready/busy signal. Furthermore, when garbage collection is triggered inside a channel, all packages inside the channel will also busy. In terms of clean page allocation, FTL uses several ways. When request arrive, FTL can simply decide only its channel location, or it can be as detailed as plane location (decide all channel, package, die, and plane location). Current SSD extension can be used to simulate most of it (channel, package, and plane). However, if we allocate write request in a channel, current SSD extension uses ganging instead of pure channel allocation. Figure 4 and Figure 5 shows time calculation when a channel becomes idle (assumes 4 packages in channel architecture are used). When use ganging, original SSD extension assumes that channel can transfer to all packages connected at the same time just as in Figure 4. In other words, channel bandwidth is as wide as a page multiply by number of dies inside a channel. Then this channel becomes busy at the same time. When all requests are processed inside the channel, all packages inside the channel become idle again. This is also not correct behavior. To fix this, we create two new events to handle channel busy and channel idle. When channel becomes busy, start_channel_busy event is triggered. This channel cannot be used until channel_idle event is triggered just as in Figure 5. By doing this, channel can only process one request at a time. We also correct the garbage collection so if one packages is busy because of garbage collection process, the other packages can process other requests. Channel is busy Channel is idle At the same time Figure 4: Original ganging implementation in SSD extension start_channel_busy Channel is busy channel_idle Figure 5: Shared channel implementation If ganging is not used, a package in SSD extension has full interconnection with a channel. To simulate the shared channel architecture, SSD extension supports die interleaving instead. This is somehow similar as shared channel implementations. When several requests are stacked inside element queue, SSD extension collects those requests and distributes it to all dies. Each request will be added transfer time cost which is similar with shared channel architecture. Unfortunately, SSD extension only supports until four die and its process is simplified only by using calculation instead of event. Thus, using the current code, implementing n-plane mode [8] will be difficult. Experiment Results and Analysis We use Micron SLC [2] configuration as shown in Table 1. First, to make sure our implementation is correct, we set the parameter so that it represents dedicated channel architecture. This is the same as original SSD extension architecture when ganging is not used. We then run both original and modified simulators and make sure that there is no difference in the simulation results to see whether our implementation is correct or not. From the experiment results, it is confirmed that there is no difference between both codes. It means that there is no mistake in our shared channel implementation. Table 1: Simulator configuration Parameters Value transfer latency 52. 8us per page Page read latency 25us Page write latency 230us Block erase latency 700us Planes per package 2 Blocks per plane 2048 Flash package elements 32GB SSD Page size 4KB Pages per block 512KB Blocks per element 2GB Binding Address based Cleaning in background Disabled Copy back Enabled 3917

4 Table 2: Traces characteristics Server # of Request Write Ratio (%) Request/sec hm_ hm_ proj_ prxy_ rsrch_ src1_ src2_ stg_ ts_ usr_ wdev_ fin To further expand our research results, we vary the number of packages inside a channel. To do this, we use workloads which are gathered from the enterprise data center in Microsoft Research Cambridge (MSRC) [12]. The trace details for the MSRC traces used can be seen in Table 2. % response time against baseline hm_0 proj_0 rsrch_0 src2_0 ts_0 wdev_0 Response Time Average (vary #ways) hm_1 prxy_0 src1_2 stg_0 usr_0 fin_2 2 packages 4 packages 8 packages 16 packages number of ways Figure 6: Percent response time average increase against baseline varying number of packages We then vary the number of packages inside a channel and do the experiment in the modified SSD extension code. The experiment results are shown in Figure 6. We remove dedicated channel experiment since we use it as a baseline. As we can see, increasing the number of packages inside a channel makes the response time average increase despite of trace s write request ratio. However, other traces characteristic such as requests per second does affect the SSD performance. For example, when busy workload is used, there is no big performance reduction when adding additional package inside a channel. This is due to the interleaving effect. We can see this behavior in fin_2 trace experiment result. In fin_2 experiment, response time is increased only around 6-12% while in idle trace such as hm_1 it is increased from % when additional package is added. This happens because of the interleaving effect between packages. When SSD has busy workload, a lot of requests need to wait in the queue despite of the number of packages inside the channel. When a lot of requests are stacked in queue, interleaving between packages is more optimal and channel wait latency can be hidden. Conclusion We succeed modifying SSD extension code, a simulator which frequently used in several researches, so that it supports shared channel between packages architecture. To make sure our implementation is correct, we set the SSD so it uses dedicated channel. There is no difference in the experiment results, thus we assume our experiment is already correct. We then vary number of packages inside a channel. As expected, increasing number of packages makes the response time slower despite of trace write request ratio. However, trace characteristics does affect the performance of general architecture SSD. When SSD has a busy workload, increasing number of channel does not reduce the overall SSD performance. This happens due to interleaving between packages can hide the channel wait latency. For future works, we are planning to implement several superblocks [9] algorithm and n-plane mode [8]. Acknowledgements This work was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (NRF-2013R1A1A ). References [1] Micron Technology Inc. M500DC 1. 8-Inch SATA SSD Features, [2] Micron Technology Inc. Micron MT29FXXXG08A Flash Memory Features, [3] A. Ban Flash file system, United States Patent. No. 5, 404, 485, April [4] N. Agrawal, V. Prabhakaran, T. Wobber, J. D. Davis, M. S. Manasse, and R. Panigrahy Design tradeoffs for SSD performance, USENIX Annual Technical Conference, pp , June [5] The OpenSSD Project, openssd-project. org/ wiki/the_openssd_project, January 2015 [6] C. Dirik and B. Jacob The performance of PC solidstate disks (SSDs) as a function of bandwidth, concurrency, device architecture, and system organization, ACM SIGARCH Computer Architecture News, vol. 37, no. 3, pp , June [7] J. S. Bucy The disksim simulation environment version 4. 0 reference manual (cmu-pdl ), Parallel Data Laboratory, May [8] Micron Technology Inc. Improving Performance Using Two-Plane Command Enabled Micron Devices, Technical Note TN-29-25,

5 [9] Micron Technology Inc. Memory Management in Arrays, Technical Note TN-29-25, [10] J. Y. Shin, Z. L. Xia, N. Y. Xu, R. Gao, X. F. Cai, S. Maeng, and F. H. Hsu FTL design exploration in reconfigurable high-performance SSD for server applications, ACM In Proceedings of the 23rd international conference on Supercomputing, pp , June [11] Y. A. Winata, K. Sanghoon, and I. Shin Enhancing internal parallelism of solid-state drives while balancing write loads across dies, Electronics Letters, vol. 51, no. 24, pp , November [12] D. Narayanan, A. Donnelly, and A. Rowstron Write off-loading: practical power management for enterprise storage, ACM Transaction of Storage, vol. 4, issue. 3, pp. 10:1-10:23, November