Alternate Data Streams in Forensic Investigations of File Systems Backups

Transcription

1 Alternate Data Streams in Forensic Investigations of File Systems Backups Derek Bem and Ewa Z. Huebner School of Computing and Mathematics University of Western Sydney and Abstract Backup utilities for the Windows environment are designed to work with the NTFS file format, but they typically provide only partial compatibility with Alternate Data Streams (ADSs) *. In particular, computer forensics tools are typically capable of discovering ADSs in the file system under investigation, but not necessarily in the backups of such file systems. We examined a number of commonly used backup utilities, and initially classified them into two broad categories: non-ads aware (ADS lost during backup), and ADS aware. Further, we discovered that within the "ADS aware" category different tools behave differently, provide varying amounts of information about ADSs during backup/restore process, and often lose data. We propose a new classification of backup software based on the treatment of ADSs during backup and restore operations, and discuss its implications for forensic investigation of file system backups. Keywords: computer forensics, Alternate Data Streams, NTFS, file systems, backup 1. Introduction Computer forensics is an emerging discipline that focuses on the gathering of evidence (often as part of a criminal investigation) from computers, computer networks, and in general from electronic media [5]. Alternate Data Streams (ADSs) are a unique feature of NTFS file systems introduced with Windows NT 3.1 in the early 1990s to provide compatibility between Windows NT servers and Macintosh clients which use Hierarchical File System (HFS). HFS uses streams named resource fork and data fork. Both streams (or forks) are linked to one name in the Macintosh file system. Resource forks are used to * Microsoft also uses the same acronym ADS in relation to Automated Deployment Services used in the Windows Server products range no connection to Alternate Data Streams. 1

2 store application metadata (icons, sounds, fonts, etc.). NTFS ADSs can provide additional descriptions for folders or files (creator, keywords, thumbnail preview, etc.), and can also be used to attach independent named data streams to an NTFS file or folder. Figure 1 shows a main unnamed stream has2ads.txt with two files cf.pdf and infoday.doc attached to the main file as two Alternate Data Streams. Figure 1 ADS visibility in NTFS and non-ntfs environments. Despite being a feature of the NTFS, ADSs are poorly documented by Microsoft. This may be the result of a conscious decision, as one can find statements in various Microsoft sources stating that ADSs may not be supported in future systems [3]. This statement of possible withdrawal of ADS support is meaningless for anyone working with a system using NTFS file format. NTFS went through many modifications and versions since ADSs were first introduced, and ADSs are still present in all current Windows Server editions, in Windows XP and in pre-release version of Windows Vista (to be released by the end of 2006). The volume of computer systems using NTFS is staggering, and it is a case of great concern for system administrators, computer forensic investigators, and indeed end users, that no standard graphic tools can show the NTFS streams. ADSs are also poorly supported by command line tools, standard file manipulation tools, and also backup utilities, both proprietary and third party. At the time this paper was written we were able to locate fewer than ten tools able to show ADSs, sometimes to add or extract an ADS. Some of them offer simple GUI interface, while some are command line based. Their description is beyond the scope of this paper. Most professional computer forensics tools are fully capable of investigating ADSs, but not in a fast, efficient way which could be compared to, say, copying a file using Windows Explorer. This paper concentrates on backup software capability to save and restore ADSs. We found that backup software provides very poor messages and log entries in relation to ADSs, and in many cases loses alternate streams completely without any warning. In other cases error or information messages are cryptic and difficult to understand. We propose a new classification of such software into five classes: Class 0 to Class 4. To determine which class a particular backup utility belongs to, we created a simple and reliable test environment. Our testing methodology is also presented in order to allow anyone to repeat the tests with any backup software, to determine its suitability to a specific task at hand, or to check what possible data loss has occurred already when the backup was done before the backup files reached the investigator. It should be stressed that the only aspect tested here is how ADSs are handled, as this is the main point of interest in the computer forensics field. The purpose of this paper is not to classify backup utilities according to their overall quality depending 2

3 on the specific environments different criteria may be more relevant. However, awareness of how ADSs are treated by specific backup software is very important in any environment, as formalizing backup/restore procedure may overcome peculiarities of the specific tool used, and minimize the potential impact of the tool imperfections. No brand names of backup utilities tested are mentioned here because of legal reasons. 2. Testing Methodology We provide a detailed description of the test environment to reinforce the understanding of the ADSs mechanism, and to allow anyone to replicate our experiments. When an operating system opens a file containing an NTFS alternate stream, by default it executes only the main stream [2]. However if a command supports ADSs, an alternate stream can be accessed using the following argument: file_name:stream_name Referring to Figure 1, the argument to access the document infoday.doc contained in the second alternate stream would be (note the : separator): has2ads.txt:infoday.doc To ensure that no contamination is introduced by additional tools, the environment was created using Windows Command prompt as shown in Table 1. Initially (Table 1a) three test files are placed on a freshly formatted diskette. A directory ad was created on an NTFS formatted hard disk C (Table 1b). Next (Table 1c) we created the first Alternate Data Stream: the file small.txt from diskette A: is attached as a ADS to folder C:\ads. Note that the command type used here is ADS aware, but the directory listing command dir is not; it does not show that the folder C:\ads has the alternate stream small.tx. In the next step (Table 1d, command echo) we created two one line text files: clean.txt and has2ads.txt Finally (Table 1e) we attach two test files cf.pdf and infoday.doc as Alternate Data Streams to the file has2ads.txt. Note that the directory listing command dir shows the same directory contents in step 1d and in step 1e, however total bytes free disk capacity shown in step 1e is decreased by the size of files cf.pdf and infoday.doc copied from the diskette. We completed creating a simple Alternate Data Streams test environment on NTFS disk C, which consists of: directory (folder) C:\ads with one ADS attached to the folder, inside this folder we have two files: one without ADS, and the other one with two ADSs attached. 3

4 Table 1 Creating test environment (a) C:\>dir a: Volume in drive A has no label. Volume Serial Number is 1CBE-EF8B Directory of A:\ 04/01/ :09 PM 49,744 cf.pdf 03/01/ :37 PM 51,712 infoday.doc 09/01/ :48 AM 18 small.txt 3 File(s) 101,474 bytes 0 Dir(s) 1,355,264 bytes free (b) C:\>mkdir ads C:\>cd ads (c) (d) C:\ads>dir Volume in drive C is SYSTEM Volume Serial Number is 7845-E3F6 Directory of C:\ads 09/01/ :46 PM <DIR>. 09/01/ :46 PM <DIR>.. 0 File(s) 0 bytes 2 Dir(s) 16,491,720,704 bytes free C:\ads>type a:\small.txt > :small.txt C:\ads>dir Volume in drive C is SYSTEM Volume Serial Number is 7845-E3F6 Directory of C:\ads 09/01/ :47 PM <DIR>. 09/01/ :47 PM <DIR>.. 0 File(s) 0 bytes 2 Dir(s) 16,491,720,704 bytes free C:\ads>echo This file is clean (no ADS) > clean.txt C:\ads>echo this file has two ADS (cf.pdf and infoday.doc) > has2ads.txt (e) C:\ads>dir Volume in drive C is SYSTEM Volume Serial Number is 7845-E3F6 Directory of C:\ads 09/01/ :49 PM <DIR>. 09/01/ :49 PM <DIR>.. 09/01/ :48 PM 30 clean.txt 09/01/ :49 PM 49 has2ads.txt 2 File(s) 79 bytes 2 Dir(s) 16,491,720,704 bytes free C:\ads>type a:\cf.pdf > has2ads.txt:cf.pdf C:\ads>type a:\infoday.doc > has2ads.txt:infoday.doc C:\ads>dir Volume in drive C is SYSTEM Volume Serial Number is 7845-E3F6 Directory of C:\ads 09/01/ :49 PM <DIR>. 09/01/ :49 PM <DIR>.. 09/01/ :48 PM 30 clean.txt 09/01/ :52 PM 49 has2ads.txt 2 File(s) 79 bytes 2 Dir(s) 16,491,614,208 bytes free 4

5 Each backup software tool was tested in NTFS file format environment and non-ntfs (FAT32) environment as follows (refer to Figure 2): folder C:\ads and its contents were backed up to NTFS disk (path 1), o the backup was restored to NTFS disk (path 2), o the backup was restored to FAT32 disk (path 3). folder C:\ads and its contents were backed up to FAT32 disk (path 4), o the backup was restored to FAT32 disk (path 5), o the backup was restored to NTFS disk (path 6). A case when the original resides on a non-ntfs media is not tested here, as it is trivial: ADS can not originate from non-ntfs file format media. Again, only one aspect of backup software is of interest here: retention or loss of ADSs, and messages relating to ADSs. Other aspects of backup tools like compression rate, network scalability, speed, error correction, etc. etc., are not relevant to this paper. We propose classification of backup tools into four classes, as detailed below. 3. Classification of ADS Handling Capabilities of Backup Software We examined backup sets created in NTFS and non-ntfs (FAT32) environments (see Figure 2). Not all data contained in an ADS is retained when a backup set is created, and not all data contained in a ADS is recreated when a backup is restored. For a computer forensics investigator it is crucial to select a method of restoring which does not cause any data loss, and to know what can be expected when the contents of a backup created by a specific software is investigated. We classified backup software into five groups depending on the level of ADS-awareness and the handling of alternate streams. Figure 2 Backing up and restoring data Class 0 (Figure 3): non-ads aware software, ADSs are ignored, and not backed up. This is a trivial case, and will not be further analysed here. Around 50% of tested tools belong to this group, including some commercial products. 5

6 Class 1 (Figure 4): ADS-aware software, which handles ADSs properly only within NTFS environment. Around 30% of tested backup tools belong to this group. Class 2 (Figure 5): ADS-aware software, which provides good compatibility between NTFS and non-ntfs environments. It offers the functionality of Class 1, but additionally it can backup intact ADSs from NTFS to non-ntfs file system environment, and restore them to NTFS environment (Figure 5, paths 4-6). Around 20% of tested backup tools belong to Class 2. One tool was unable to backup ADSs attached to a folder (it correctly backed up ADSs attached to files). Class 3 ADS-aware software, which can be seen as an unfinished implementation of Class 4, and it does not warrant closer investigation. It has all the capabilities of Class 4 (see below) with the exception of one: o it is unable to restore ADSs from a backup created on NTFS file system to non-ntfs media (Figure 2 path 3), or: o it is unable to restore ADSs from a backup created on non-ntfs media to NTFS media (Figure 2 path 6). Class 4 (Figure 2): ADS-aware software, which has complete ADS awareness in any environment. It can backup and restore the ADS part to NTFS or non-ntfs media. No software was found which could be classified as Class 3 or Class 4. Figure 3 ADS handling by Class 0 backup software (non-ads aware) Figure 4 ADS handling by Class 1 backup software 6

7 Figure 5 ADS handling by Class 2 backup software 4. Partly ADS-aware Backup Software (Class 1 and Class 2) A Class 1 tool is able to backup and restore ADSs if the operation is within NTFS environment (Figure 4, path 1-2). It fails to backup ADSs from NTFS to non-ntfs environment (Figure 4, paths 3 and 4). It is crucial for a computer forensics specialist to notice that Class 1 software is perfectly able to restore data to a non-ntfs disk, but no messages warning about the loss of data contained in ADS data would be generated. This is a practical observation, not a theoretical restriction. It would be possible for Class 1 software to warn that backup and restore environments differ, and warn against possible data loss. However this means that such a tool would check for the presence of alternate streams before generating a warning message thus implementing Class 2 (or even Class 4) behavior as explained below would be easy. Class 2 backup software uses the old block format, originally created for tape magnetic media [1]. Data read from a disk is stored as a set of logically sequential blocks. Terminology can vary depending on a specific implementation. The older, generic terms for major backup components are: tape header, data sets, on tape catalog information, end of media. Modern tape backup specification allows the use any common media backup, for example hard disks, removable cartridges, flash drives, etc., but basic tape style logic of storing data is retained. Class 2 backup software retains ADS data, but is unable to restore it to non-ntfs media (see Figure 5, paths 3 and 5). It is particularly dangerous to attempt to restore a backup which contains ADSs to a non-ntfs media. For example a backup from a NTFS disk to a FAT32 disk creates one of the following situations: backup software does not show any warning messages, and ADS data is lost, backup shows a warning message, but ADS data is still lost. 7

8 Figure 6 Restoring to FAT32 media: various warning message When attempting to restore a backup containing alternate NTFS streams to a FAT32 disk most tools generated unclear warning messages, or no warning messages. Figure 6 shows three typical examples of such warning messages when attempting to restore a backup containing alternate NTFS streams to a FAT32 disk. The first example (Figure 6a) shows a message which is completely insufficient and confusing; it says filename, directory name or volume label syntax is incorrect. It does not explain that the real problem is potential loss of alternate data streams. When the Ignore option is selected, restore operation proceeds, and the restore process does not generate any additional warnings, losing ADS in the process. Thus a computer forensics investigator could incorrectly assume that the restore succeeded, and that no data was lost. Figure 6b shows an example of a proper warning message generated by software during an attempt to restore a backup which contains alternate NTFS streams to a FAT32 disk. The message only states that the target file system does not support all features of the original file system, thus some data may not be restored. It may still not be clear to a person doing the restore what are those some unsupported features, and it may not be obvious that proceeding with the restore will cause ADS loss in restored files and folders. A software tool used in this example was not designed to scan, discover and list ADS data, it only sensed that the backup created in NTFS environment is being restored to a different environment, and thus produces a generic warning. 8

9 Figure 6c shows an example of an informative warning message generated during an attempt to restore a backup containing alternate NTFS streams to FAT32 disk. While the term ADS is not used, the message clearly lists alternate data streams found in the backup set, the list uses correct syntax, and warns specifically which streams will be lost if the restore process continues. 5. Fully ADS-aware Backup Software (Class 3 and Class 4) Class 3 software can be seen as an unfinished or logically incomplete implementation of Class 4 software, with one path missing (Figure 2, path 3 or path 5 is not implemented). We did not find any Class 3 software; it appears likely that it could exist only as an intermediate development stage leading to a full Class 4 implementation. Class 4 software (for path references see Figure 2) is fully ADS-aware, and it should be able to: Backup files and folders with ADSs to NTFS environment (path 1), Restore the backup and all ADSs to both NTFS and non-ntfs environments (path 2 and path 3), Backup files and folders with ADSs to non-ntfs environment (path 4), Restore the backup and all ADSs to both NTFS and non-ntfs environments (path 5 and path 6). We did not find any Class 3 or Class 4 software. It would be relatively easy to implement Class 4, as all data structures required to handle ADSs are already internally defined by software which belongs to Class 2. Class 2 internals show definitions of blocks and headers needed to handle structures unique to NTFS file system, like sparse data, Alternate Data Streams, etc. As an example, Seagate Software tape format specification [4] provides definitions for Windows NT Stream structures to be stored within its data set, and allocates a label within a stream header named NTFS_ALT_STREAM. Despite that there are no backup utilities capable of restoring ADSs outside the NTFS file system environment. 6. Recommended Forensic Methodology The role of backup is different for the end user and for a computer forensics investigator. The end user, or a person responsible for an organisation system data backup policy, sees a backup as a mechanism to save, and later to restore selected files and folders. The process is completed once the data has been restored [7]. For a computer forensics investigator on the other hand, restoring data is only the first step in the process of analysing what the files and folders contain [6]. It is not immediately known what software was used to create a backup being analysed; this 9

10 has to be discovered by looking at the backup contents with forensic tools. Typically the procedure used to create the backup is also not known. This information, however, is typically not hidden and is easy to discover even without knowing the internal structure of software: for example checking the backup file contents with a HEX editor shows readable sections which include the backup software name and version. Once the name and version of the backup software used is known, an investigator can find more information about its internal data structure. Using the same software an investigator can also run tests confirming which class of software it is, what are its peculiarities, and how to restore the backup without loosing ADSs. While in theory it would be possible to develop tools to read the contents of a backup set without restoring it, the amount of work and cost involved in developing such a tool could not be justified. There are too many different software packages with different backing up and compression algorithms. Moreover there are often considerable differences between different versions of the same software. Thus, in a process of forensic discovery where a backup has to be restored, the restored data has to be scanned to determine the presence of ADSs, and ADSs need to be extracted. Various tools can be used to achieve this, though their description and extraction process is not described here. The only safe procedure when using Class 1 or Class 2 software is to restore into NTFS file system target. If a backup set is restored to non- NTFS media, ADSs will be lost, and the whole computer forensics process invalid. Class 4 backup software would speed up the process of ADS extraction, and make it more reliable. Restoring a backup which includes ADSs to non-ntfs environment would allow for the extraction of all ADSs in a single restore step into specified folders. This would eliminate the need to use separate steps and tools for ADS discovery and extraction. If the same software which had been used to originally create the backup is used to restore, and also to extract ADSs, the ADS extraction is formally more trustworthy. There is no need to prove in a court of law that correct additional scanning and extracting tools were selected. 7. Conclusions and Future Work ADSs are widely used, but poorly documented and poorly supported by existing tools. ADS-awareness in all types of software utilities is generally very low. NTFS alternate streams create a perfect mechanism to hide huge amount of information; the main, visible stream can be a small file, but ADSs attached to it can be of any format and size limited only by other system factors. It is also possible to hide executables and malicious code in alternate streams. Unexpected behaviour of backup software when handling ADSs is of particular relevance in the computer forensics field, where in order to analyse files contained in a backup image an investigator needs to perform a restore. Backup software typically uses data compression, often proprietary, and it is usually not practical to analyse the contents of the original backup file without restoring it first. We demonstrated that if an incorrect approach is used, restored data might differ considerably from the original; thus crucial potential evidence may be lost. We presented a new 10

11 classification of backup software into five classes: Class 0, Class 1, Class 2, Class 3 and Class 4, and described a straightforward test which allows any person to repeat our tests and to allocate a class to any backup software using our environment. Finally, a safe methodology was presented for handling backups in order to avoid information loss, which is relevant in both forensics cases and in file systems maintenance in general. Future work in this area is required to better formalize requirements for Class 4 backup utilities. Currently (beginning of 2006) we were unable to locate any software which would pass Class 4 criteria. As new software products come to the market we expect to be able to refine our classification and forensic methodology. We also intend to broaden the scope of our research to include the impact of data compression on handling ADSs. References [1] Microsoft Knowledge Base, Description of Microsoft Tape Format (MTF), Article ID: , Last Modified: October 29, 2003, Accessed: January 19, 2006, [2] Microsoft TechNet, Windows Server 2003 Technical Reference, How NTFS Works, Last Modified: March 28, 2003, Accessed: December 10, 2004, chref/8cc5891d-bf8e d-dac5418c5948.mspx [3] Microsoft Knowledge Base, How To Use NTFS Alternate Data Streams, Article ID: , Last Modified: July 13, 2004, Accessed: November 11, 2004, [4] Microsoft Tape Format Specification, Version 1.00a - document rev. 1.8, Seagate Software, Inc., [5] E. CASEY, Digital Evidence and Computer Crime, Elsevier Academic Press, London, UK, 2004, ISBN [6] K. MANDIA, C. PROSIE and M. PEPE, Incident Response & Computer Forensics, Second Edition, McGraw-Hill/Osborne, Emeryville, CA, 2003, ISBN X. [7] W. C. PRESTON, Unix Backup and Recovery, O'Reilly, Sebastopol, CA, 1999, ISBN