Empirical Analysis of Solid State Disk Data Retention when used with Contemporary Operating Systems

Transcription

1 DIGITAL FORENSIC RESEARCH CONFERENCE Empirical Analysis of Solid State Disk Data Retention when used with Contemporary Operating Systems By Christopher King and Timothy Vidas Presented At The Digital Forensic Research Conference DFRWS 2011 USA New Orleans, LA (Aug 1 st - 3 rd ) DFRWS is dedicated to the sharing of knowledge and ideas about digital forensics research. Ever since it organized the first open workshop devoted to digital forensics in 2001, DFRWS continues to bring academics and practitioners together in an informal environment. As a non-profit, volunteer organization, DFRWS sponsors technical working groups, annual conferences and challenges to help drive the direction of research and development.

2 Empirical Analysis of Solid State Disk Data Retention when used with Contemporary Operating Systems Christopher King (CERT) Tim Vidas (CMU ECE) DFRWS 2011 New Orleans, LA 2011 Carnegie Mellon University

3 Notices 2011 Carnegie Mellon University Except for the U.S. government purposes described below, this material SHALL NOT be reproduced or used in any other manner without requesting formal permission from the Software Engineering Institute at This material was created in the performance of Federal Government Contract Number FA C-0003 with Carnegie Mellon University for the operation of the Software Engineering Institute, a federally funded research and development center. The U.S. Government's rights to use, modify, reproduce, release, perform, display, or disclose this material are restricted by the Rights in Technical Data-Noncommercial Items clauses (DFAR and DFAR Alternate I) contained in the above identified contract. Any reproduction of this material or portions thereof marked with this legend must also reproduce the disclaimers contained on this slide. Although the rights granted by contract do not require course attendance to use this material for U.S. Government purposes, the SEI recommends attendance to ensure proper understanding. THE MATERIAL IS PROVIDED ON AN AS IS BASIS, AND CARNEGIE MELLON DISCLAIMS ANY AND ALL WARRANTIES, IMPLIED OR OTHERWISE (INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR A PARTICULAR PURPOSE, RESULTS OBTAINED FROM USE OF THE MATERIAL, MERCHANTABILITY, AND/OR NON-INFRINGEMENT). 2

4 Outline 3 Introduction Solid State Disk Background Forensic Challenges Testing Methodology Results Conclusion 3

5 Introduction Solid State Disks are the future of data storage. 15M units projected to be sold in 2011, 118% increase over last year. 1 Large increases in Read/Write speed over hard disk drives 20x+ faster read 2 ~ MB/s sequential read ~18-77MB/s random read 10x+ faster write 2 ~40-250MB/s sequential write ~5-51MB/s random write Use less energy No moving parts

6 Problem: Data Recovery Solid State Disk Hard Disk Drive Credit: Micron Technology, Inc. 5

7 Problem: Inconsistent Research Findings Early research in SSDs has resulted in conflicting or inconsistent results. Data deleted over time (Bell and Boddington 2010) Data stays remnant in the system (Wei et al. 2011) Increased need for reproducible results Bringing science to digital forensics with standardized forensic corpora Simson Garfinkel, Paul Farrella, Vassil Roussevc, George Dinolta 6

8 Prior Work Bell and Boddington - Solid State Drives: The Beginning of the End for Current Practice in Digital Forensic Recovery? Showed SSD garbage collection was wiping unallocated space after a period of time. Wei et. al. - Reliably erasing data from flash-based solid state drives Some data still resident on SSDs due to overprovisioning. 7

9 SSDs: A Background 8 SSDs are Flash memory based disks, much like thumbdrives. They use NAND Flash in their operation. Credit: John Fader, Creative Commons License 8

10 NAND Flash Cell A cell is the simplest structure in Flash memory. When a charge is applied, the electrons tunnel into the cell through the dielectric barrier. When the charge is stopped the electrons are trapped in the cell. The resulting positive or negative charge can then be measured. 9 [2] Credit: Anandtech.com 9

11 Single Layer Cell vs. Multi Layer Cell 10 Credit: Micron 10

12 NAND Flash page 11 Credit: Cyferz, Wikipedia Creative Commons License Series of floating gates and cells all connected. A write occurs starting from the source, and writing down the entire page. A read requires a measurement of the sink, which totals up the values of the floating gates. 11

13 SSD Design 12 12

14 Example: 4GB MLC Flash 13 Cell - 2 bits Page 4KB (8,192 cells) Block - 256KB (64 pages) Plane 524MB (2048 blocks) Chip/Die 2GB (4 planes) Drive 4GB (2 chips/dies) 13

15 Implementation Challenges 14 Flash Cell Wear Maximum of 100,000 writes per cell. Delete-before-write Flash deletes in blocks of 128KB. Expensive operation Write amplification Legacy compatibility Drives must be compatible with major OS s that do not understand Flash file systems. 14

16 Implementation Challenges - Solutions 15 Flash Cell Wear Wear leveling Delete-before-write Garbage collection Overprovisioning TRIM Legacy compatibility Flash Transition Layer 15

17 Review: Data Deletion on HDDs 16 The OS stores files in clusters, tracking the location of these clusters using a table with a pointer. ie) Master File Table (MFT) in NTFS, inodes in EXT2/3 When a file is deleted, the OS deletes the pointer in the table, marking that space available. Until the OS uses those blocks again, the data remains on the disk. This data can be recovered using a number of tools. 16

18 Data Deletion on SSDs 17 When a file is deleted, the pointer in the MFT is deleted and the OS sees that area as free. The SSD marks the data as invalid. The data stays where it is until that block needs to be overwritten or if the controller needs more blocks. Garbage Collection. The SSD keeps a number of active blocks available for writes (allocation pool). When this number falls below a certain threshold, the drive initiates block cleaning (or garbage collection). The SSD chooses the block with the highest ratio of invalid pages to total pages. The SSD then copies over any non-deleted data, deletes the pages, and puts the now clean block back into the pool. A delete in NAND flash sets all the bits to 1. 17

19 Data Deletion on SSDs: TRIM 18 TRIM Command Added to ATA8 specification to solve performance problems associated with garbage collection. OS tells disk what blocks are free. TRIM operation File is deleted The OS sends a TRIM command to the SSD with the LBA location of the deleted blocks. The SSD maps this to its own FTL blocks. It then copies any valid pages to memory and deletes the block. The block is rewritten with the valid data. This can happen immediately after delete instead of when blocks are needed. The drive controller software decides when to erase the blocks it knows are now free. 18

20 Forensic Challenges 19 The sector the OS reports is not the physical location of the data. Data deleted is gone forever (in some cases). Garbage collection implementation varies among manufacturers. 19

21 Research Questions 20 How will forensics investigators know if data can be recovered? Are these disks even similar enough that data CAN be recovered? Are some SSDs better than others at data retention? Performance varies greatly between controller manufacturers does this apply to data retention? How does TRIM affect data recovery? This new ATA command seems to effect deletion of data. How does the Operating System affect data recovery? Windows 7 was optimized for SSDs what does that mean? 20

22 Test Methodology 21 Test Cases: Test 1: Data recovery from low-usage disk Test 2: Data recovery from high-usage disk Test 3: Data recovery from OS formatted disk Each test was tested using: Windows 7 Enterprise Windows XP Professional Ubuntu Linux Desktop 9.04 Data recovery was done using Backtrack 4, Sleuthkit, and custom scripts to compare individual blocks to reference files. 21

23 Test Methodology To ensure proper comparisons over many tests: Each drive was secure wiped before installing OS The OS was installed on to the disk Two reference files, a small 1 MB text file and a large, 650MB binary file were copied to the disk In the high usage test, the disk was filled with random files to simulate a filled disk. The starting and ending block locations of each file were extracted using Sleuthkit. The files were deleted normally and recovery attempted using a custom script to byte-wise compare the blocks. 22

24 Test Methodology (cont.) 23 Drive statistics 16 SSDs tested (1 HDD control) 10 different manufacturers 5 drives supporting TRIM 144 tests conducted 23

25 Results: Drives without TRIM Percent Blocks Recovered on Drives without TRIM 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Control Intel1 Intel2 Intel3 OCZ1 Corsair1 Crucial1 Large File, Low Usage, WinXP 99.99% % % % % % % Large File, High Usage, WinXP 99.99% % % % % % % Large File, Format, WinXP 99.99% % 0.00% 0.13% % 0.00% 0.00% Small File, Low Usage, WinXP 96.57% 96.57% 96.57% 96.57% 96.57% 96.57% 96.57% Small File, High Usage, WinXP 96.57% 96.57% 96.57% 96.57% 96.57% 96.57% 96.57% Small File, Format, WinXP 96.57% 59.84% 86.20% 46.76% 49.03% 24.69% 0.00% 24

26 Results: Drives with TRIM Percent Blocks Recovered on Drives using TRIM 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Control Intel1 (TRIM) Intel2 (TRIM) Intel3 (TRIM) OCZ1 (TRIM) Corsair1 (TRIM) Crucial1 (TRIM) Large File, Low Usage, Win % 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% Large File, High Usage, Win % 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% Large File, Format, Win % 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% Small File, Low Usage, Win % 0.00% 0.00% 0.00% 25.53% 25.53% 27.54% Small File, High Usage, Win % 0.00% 0.00% 0.00% 25.53% 27.54% 26.28% Small File, Format, Win7 0.00% 0.00% 0.00% 0.00% 0.00% 24.46% 0.00% 25

27 Percent blocks recovered in Windows XP across all tests % 90.00% 80.00% 70.00% Large File, Low Usage, WinXP 60.00% Large File, High Usage, WinXP 50.00% Large File, Format, WinXP 40.00% 30.00% Small File, Low Usage, WinXP 20.00% Small File, High Usage, WinXP 10.00% Small File, Format, WinXP 0.00% Intel1 (TRIM) Intel2 (TRIM) Intel3 (TRIM) Corsair1 (TRIM) Crucial1 (TRIM) OCZ1 (TRIM) PQI1 Patriot2 Kingston1 Imation1 OCZ2 Patriot1 RiData1 Intel4 Transcend1 Control 26

28 Percent blocks recovered in Windows 7 across all tests % 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% Large File, Low Usage, Win7 Large File, High Usage, Win7 Large File, Format, Win7 Small File, Low Usage, Win % Small File, High Usage, Win % Small File, Format, Win % 0.00% Intel1 (TRIM) Intel2 (TRIM) Intel3 (TRIM) Corsair1 (TRIM) Crucial1 (TRIM) OCZ1 (TRIM) PQI1 Patriot2 Kingston1 Imation1 OCZ2 Patriot1 RiData1 Intel4 Transcend1 Control 27

29 Findings 28 Data deleted on a SSD that supports TRIM, on a TRIMenabled OS, is seemingly impossible to recover. High usage disks have a greater chance of data retention over low usage disks. Windows 7: up to 17% more than low usage, 100% more with small files Formatted disks using default settings result in partial data loss for all OS s and full data loss for Windows 7 with TRIM. Manufacturer variance in garbage collection seems to change the recoverability of files on the drive. 28

30 Manual TRIM Test SSDs can be used for anti-forensics. A single click can tell the drive to TRIM itself, destroying evidence. A modified hdparm script caused two test SSDs to perform the equivalent of a secure wipe. Roughly 10 seconds to perform zeroing of drive. Further modifications to the code could reduce the time even further. Running drive was also TRIMed drive remained operational until reboot, in which drive was fully zeroed. 29

31 Implications 30 SSDs should not be considered the same as an HDD when encountered in the field. The manufacturer and model of a drive is an important consideration. Most new SSDs are being built with TRIM support. Investigators should not expect substantial results with data recovery on drives supporting TRIM and using TRIM-enabled OS s. The OS type is very important for making a decision on whether to perform static or volatile analysis. Drive manufacturer and firmware does matter for recoverability. 30

32 Future Research 31 When do different OS s send TRIM commands? Data recovery from Linux ext4 and TRIM-supported kernels. Data recovery from SSDs in OSX. Analysis of newer firmware and drives (Sandforce) More comprehensive temporal analysis 31

33 32 Questions? 2011 Carnegie Mellon University