Objectives 00:13 CSC 486/586 RAID Rebuilding In your previous module, you learned about RAID technology, including hardware and software RAIDs. In this module you will learn about the issues you need to be aware of to properly perform the acquisition and rebuilding of data stored on a RAID array, for subsequent analysis. RAID Imaging RAID Attributes RAID rebuilding 101 Rebuilding Tools X-Ways Forensics/WinHex (Specialist or Forensic license) Encase SMART 2 01:06 01:46 What the forensic examiner sees (physically). What the OS sees a 273GB primary disk and two 2,235 GB Disks 3 4 Multi-RAID levels??? 03:19 05:36 Notice the X: drive is a 4471 GB Windows Server 2003 striped volume made up of two 2235 GB physical disks which are actually each made up of 7 400GB IDE disks set up as hardware RAID volumes. (a software RAID 0 striped across two hardware volumes.) The physical drives that are actually present 3-136GB array disks and 1-136Gb hot spare, plus 14 400GB IDE disks in an Apple X-Serve RAID (not shown in screenshot). 5 6
07:37 10:16 What your imaging tool might see What your imaging tool might see * The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not 7 necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable 8 * The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable 10:51 11:49 What your imaging tool might see What your imaging tool might see 9 * The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable * The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not 10 necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable RAID Attributes 12:50 RAID Attributes 14:28 Disk Order Stripe Size RAID Header Parity Dedicated vs. Distributed Parity Type/Rotation Parity Delay Disk Order The order of the disks that make up the array This may seem like a very simple one, but when pulling individual drives from a RAID, it is easy to get them out of order or mislabel the image names for each disk image. Always double check yourself, especially when putting the disks back into the server to ensure they are in the correct order. 11 12
RAID Attributes 16:04 RAID Attributes 18:13 Stripe Size How much data is written to each disk before moving to the next disk to write the next block of data. Typical stripe sizes: 8,16, 32, 64, and 128 kilobytes per stripe you may occasionally see other sizes 13 RAID Header Static block of data at the beginning of each array May be identical (or nearly identical), making you initially think it s a mirror Usually has a byte that identifies the disk # for the array, which gives you your Disk Order Header size and disk # usually found by performing a comparison of the disks. Compaq/HP servers usually = 1088 sector header size 14 RAID Attributes 20:49 RAID Levels 23:48 Parity Rebuilding information created by XOR ing together bytes from each disk containing RAID data, the result of which get s stored as a parity value on the parity disk. The drive on which this calculated parity data is stored will depend on the type of Parity Type/Rotation used. Parity Rotation described in more detail later in presentation RAID4 = Dedicated parity disk RAID5 = Distributed parity disk 15 RAID 0 (Striping) RAID 1 (Mirroring/Duplexing) (Striping w/ Distributed Parity) Multi-RAID levels RAID 1+0 (a stripe of mirrors) RAID 0+1 (a mirror or stripes) RAID 1+5, 5+1, 0+5, 5+0, etc. Other non-raid multi-disk setups: Disk Spanning JBOD (Just a Bunch Of Disks) 16 RAID 0 26:00 RAID 1 27:46 No fault tolerance Single disk failure = array failure Fastest performance Capacity of array = total capacity of individual disks combined Items needed for rebuilding: Disk Order Stripe Size RAID header size* 17 * Not all RAIDs have a RAID header Fault tolerance (via data replication) Increased read performance, same write performance as writing to single disk 50% of disk capacity used for data redundancy Items needed for rebuilding: Typically no rebuilding necessary unless RAID header exists* 18 * Not all RAIDs have a RAID header
29:06 30:55 Fault tolerance (via parity data) Increased read and write performance 1/Nth reduction in disk capacity, used for parity, where N = # of array disks. Minimum of 3 array disks needed for any RAID level with parity Rebuilding components: Disk order Stripe size RAID header size* Parity rotation Parity delay** * Not all RAIDs have a RAID header ** Only used in Backward Delayed Parity 19 20 32:53 34:02 Parity Rotation Backward Delayed Parity (Compaq/HP)* Parity Rotation Backward Dynamic Parity (AMI) A.k.a Left Symmetric Probably the most common type * Example shown using a parity rotation delay of 4, meaning parity stays on it s current disk for 4 stripes, then moves for the next 4 stripes and so on. 21 22 34:57 35:47 Parity Rotation Forward Dynamic Parity A.k.a. Right Symmetric (Some Linux software RAIDs) Parity Rotation Backward Parity A.k.a. Left Asymmetric (Adaptec) Forward Parity A.k.a. Right Asymmetric 23 24
25 RAID Rebuilding 101 The goal in RAID rebuilding it to put back together the data that has been spread out across multiple disks and may include parity information. This is done by re-pasting the striped data back together into one disk/image and removing the parity as you go. Individual RAID disks/images Disk 0 Disk 1 Disk 2 Disk 3 Disk 4 Stripe1 T H I S Parity Stripe2 A S Parity W Stripe3 R A Parity A Stripe4! Parity I D! RAID rebuilt into single disk Disk 0 36:46 THIS WAS A RAID!! 26 RAID Rebuilding 101 The more you document about the RAID onsite, the less you have to manually try to figure out later! Boot RAID server into RAID Controller BIOS configuration utility during Power On Self Test (POST) View array configuration and write down the RAID level, disk order, stripe size, disk & array configuration, controller type, etc!!! 39:03 RAID Rebuilding 101 40:22 RAID Rebuilding 101 40:47 27 28 29 RAID Rebuilding 101 41:36 Any of the information you are unable to determine onsite during the imaging of the RAID disks will have to be either manually determined or possibly via some guesswork. Manual interpretation of the striped data on RAID disks is not difficult if you have an in-depth understanding of how data structures are laid out on a non-raid disk, including: MBR and Partition Table Boot Sectors/Records FAT tables, Root Dirs, etc. MFT records, INDX entries, etc. Unfortunately, it is not possible to cover manual data interpretation in this one hour presentation. 30 RAID Rebuilding Tools (Runtime Software) http://www.runtime.org/raid.htm X-Ways Forensics/WinHex (X-Ways Software Technology AG) http://www.x-ways.net/forensics/index-m.html Encase (Guidance Software) http://www.guidancesoftware.com/products/ef_index.aspx SMART (ASRData) http://www.asrdata2.com/ ***There are a few other RAID rebuilding tools out there but as of the writing of this presentation, the above tools were the only ones I had available to include. 43:06
43:34 Step #1 chose RAID type, number of drives, add drives images (in correct order), select block size and parity rotation. 31 44:54 Step #2 analyze data to attempt to determine correct RAID parameters. 32 46:12 Step #3 - write out a new rebuilt single image from the multiple images. 33 46:42 Pros Tests numerous combinations of RAID parameters to try and Guess settings using entropy testing. Useful when you don t know the parameters. Works with up to 14 RAID disks for. Will rebuild RAID with one missing disk/image. Cons Can only do a 2-disk RAID 0 Doesn t do Backward Delayed Parity RAIDs Requires you to actually rebuild a new image before you can check to see if you actually have the correct settings. Only after the rebuild can you open the new image in your forensic tools. Does not recognize.e01 or other image formats, must convert images to raw bit. 34 X-Ways Forensics/WinHex X-Ways Forensics/WinHex Step #1 Open each individual disk image and Interpret Image File as Disk from the Specialist menu. Step #2 Select Assemble RAID system from the Specialist menu. Open each disk component in the correct order, enter the header size, select the parity rotation type and stripe size and click OK. 48:12 48:30 35 36
37 X-Ways Forensics/WinHex If you entered the correct RAID parameters, the RAID volume is virtually reconstructed, allowing you to map out the file system. 49:23 38 X-Ways Forensics/WinHex Pros 49:41 Performs a virtual rebuild in RAM to allow you to see the results right away. File system mapping errors indicate if you have the wrong parameters. Works with up to 16 RAID disks*. Will rebuild RAID, from parity, with one missing disk/image. The only tool that does Backward Delayed Parity (Compaq/HP) and Forward Dynamic Parity* (some Linux software RAIDs). Reads.e01 or raw bit images. Cons Does not do any guesswork. * Version 14.5 and later EnCase (Software RAID) 41:46 EnCase (Software RAID) 52:28 39 40 41 EnCase (Hardware RAID) 52:37 42 EnCase 53:57 Pros Can be used to virtually reconstruct Windows Software RAIDs and some hardware RAIDs. Reads.e01 and raw bit images. Can rebuild, from parity, with a missing image. Cons Only rebuilds Right or Left handed stripe RAIDS. (Not sure what Parity rotation types these refer to, but they are not in line with the correct industry terminology used by other vendors.) Lacks features for RAID headers and Delayed and Dynamic Parity.
SMART 54:50 SMART 55:40 1 2 3 4 1 43 3 4 2 1 44 2 3 45 SMART 55:58 Pros Can be used to virtually reconstruct RAIDs. The only tool that does RAID4. Allows removal of RAID header when importing images (prior to RAID rebuilding steps). Reads.e01 and raw bit images. Cons Only rebuilds Right Symmetric or Left Symmetric parity RAID5. Relies on Linux OS it is running on for driver support (i.e. MD raid driver). Device detection may be more complex and require more user interaction or configuration. Linux drivers are not available for all controller cards. Requires Linux knowledge/familiarity. 46 Questions Use the discussion board, as usual 57:39