How To Measure The Quality Of A File Carving

Transcription

1 National Conference on Emerging Trends in Computing and Communication (ETCC-2008) ANALSIS FRAMEWORK FOR QUALITY MEASURMENT OF CARVING TECHNIQUES Prof. Deepak Singh Tomar, Mr. Om Malviya, Mr. Rakesh Verma Department of CSE Maulana Azad National Institute of Technology(MANIT) Bhopal(M.P) Abstract File Carving is the method of investigating raw disk image based on file format specific characteristics present in that disk image. The tools that perform file carving, implement techniques like Header footer carving, Header maximum size, Header embedded length, File structure based and Block based. To measure the quality of carving techniques a framework is proposed which is based on the result produces by the tools from the disk image. It uses three quality aspects (Carving recall for Data Relevancy, Carving Precision for Correctness and Supported Recall for Reliability) of the file carving technique that has been utilize for the measurement of quality. The framework is also implemented on existing carving tools (Scalpel, Foremost) for quality appraise. Keywords Digital forensic, File carving, Digital Disk image, File carving Tools, Carving recall, Carving precision I. INTRODUCTION A. Digital Forensic Digital or computer forensics is the practice of identifying, preserving, extracting, analysing and presenting legally sound evidence from digital media such as computer hard drives. [1] B. File Carving During a digital forensic investigation different pieces of data are preserved for investigation, of which bit-copy images of hard drives are the most common. These images contain the data allocated to files as well as the unallocated data. The unallocated data may still contain information that is relevant to an investigation, in the form of (parts of) intentionally deleted files or automatically removed temporary files. Unfortunately, this data is not always easily accessible: a string search on the raw data might recover (parts of) interesting text documents, but it won't help to get to information present in for example images or compressed files. The downside of this approach is that these techniques become much less effective if the file system information is corrupted or overwritten. In these cases techniques is required that works independent of the file system information, by identifying the deleted files and file parts directly in the raw data and extracting them in a verifiable manner, these techniques are called file carving. [2] C. Current state of Carving and Motivation Carving is a general term for extracting files out of raw data, based on file format specific characteristics present in that data.. D. Problem Description If a tool produces good results, then valuable information might be uncovered. However tool results can also have a negative impact on the usability of the available information. First of all information that is not recovered by a tool is information that will most likely be ignored, since datasets under investigation are much too large for manual inspection. The biggest problem is that results are a combination of both the tool and the data being examined Another problem is that some quality aspects are simply unrelated to the carving results, like the amount of human intervention needed to process a dataset or the speed of a carver. II. DATASET AND FILE CARVING TECHNIQUES Some carving technique is used by current tools discuss below. For each technique a description is given of how the difficulties that may be present in a dataset are handled and what impact this has on the results. A. Structure of Datasets The datasets that are examined in a digital forensic investigation are usually bit-copies of full hard drives or individual partitions. The data present on these original drives continually changed over time. files were added, deleted, copied or moved, a partition may have been defragmenter, formatted or even resized, etc. 1). Fragmented Files A fragmented file is a file that has been split into multiple parts and where all parts may be placed on different locations in a dataset. Fragmented files can be divided into two categories: i). Files with linear fragmentation ii). Files with nonlinear fragmentation i). Linear fragmentation occurs when a file has been split into two or more parts, but the parts are present in the dataset in their original order. 421

2 National Institute of Technology Hamirpur (HP) ii). No guarantee that fragmentation is always linear, it is also possible that the different parts exist in the dataset in a different order than in the original file 2). Partial Files Partially overwritten or partial files can almost never be fully recovered1, in some cases partial files can be repaired, but this is beyond the scope of theproject but may still contain useful information. B). Carving Techniques This section describes the different carving techniques that are used by the open-source tools tested in section V and/or were used in the 2007 DFRWS carving challenge [3]. 1) Header footer carving 2) Header maximum size file carving 3) File structure based carving 4) Block based carving 1). Header-Footer Carving Header-footer carving is the most basic carving technique. It works by searching the dataset for the patterns that mark the beginning of a file (header), like x89pngx0dx0ax1ax0a2 for PNG files. 2). Header Maximum Size Files Carving Even though most file types have a unique header, not all file types have a fixed footer. In the case of header maximum file size carving a maximum file size is defined for these file types. If a header is found, then a piece of data is carved of maximum file size length. 3). File Structure Based Carving File structure based carving uses the internal layout of a file to determine which data is part of which file. To understand file structure based carving, it is best to first get a basic idea of the type of structures that can be present in a file. 4). Block Content Based Carving One technique that can be useful in detecting fragmentation in those cases where file structure based carving is unsuccessful, is block content based carving. Block content based carving works by calculating Meta information like character counts or statistical information over the bytes in a block. III. CARVING TOOL QUALITY MEASURING METHOD A). File Marking Of Carved File As the previous section discussion, the combination of complete, partial and fragmented files and different carving techniques can lead to four different result types. To recapitulate: 1). Positive A file that is correctly carved from the dataset is called a Positive.. 2). False Positives and Known False Positives False positive: A carving result which is not a Positive. Known false positive: A carving result of which the carver knows that it is not fully correct, and which it has marked as such false positive. 3). False Negative A file that is present in the dataset, but which was not carved. The False negative definition needs more elaboration, since in carving the recovery of a file is not necessarily all or nothing. B). Quality Criteria Measures How can these result types be translated into measurable quality criteria?" Originally a quality system was created based on the number of Positives, Unknown false positives and false negatives. It simply calculated the following two scores: a. The main score was calculated by giving points for each Positive and subtracting points for the false negatives. The higher this score was, the better a tool performed. b. A second score was calculated by counting the number of Unknown false positives, which should be as low as possible. C). Quality Criteria Measures Method Precision and Recall are two widely used measures for evaluating the quality of results in domains such as Information Retrieval and statistical classification.[3] Precision can be seen as a measure of exactness or fidelity, whereas Recall is a measure of completeness. In Information Retrieval, a perfect Precision score of 1.0 means that every result retrieved by a search was relevant (but says nothing about whether all relevant documents were retrieved) whereas a perfect Recall score of 1.0 means that all relevant documents were retrieved by the search (but says nothing about how many irrelevant documents were also retrieved).[3] Recall is defined as the proportion of the target items that the system selected: 422

3 National Conference on Emerging Trends in Computing and Communication (ETCC-2008) Precision is defined as a measure of the proportion of selected items that the system got right: Positive Precision = Positive false negative Together these two measures can be combined into a single measure of overall system performance, called the F measure. Fmeasure = 1 (α /Rl) + ((1-α)/P) Where P is precision, R is recall and α is a factor between 0 and 1 which can be used to determine the weighting of precision and recall. These equations form the basis for a quality measurement system whose goal it is to answer the following three carving quality questions: 1) What proportion of the available files was recovered? 2) What proportion of the recovered files was correct? 3) How reliable is the tool? If it claims to support a set of file types, then what proportion of these files does it recover? The first quality question can be answered using a modified version of the recall equation. This leads to the following recall equation for carving: all Where all is the number of files in the dataset. The second quality question can be answered using a modified version of the precision equation. This leads to the following precision equation for carving Carving precision = Positive Positive +unknown false positive +(1/β)known false positive Where β is a factor, which can be used to determine he relative weight of Unknown false positives compared to Known false positives. A carving specific variation on the F measure, named the Cperformance, can now be used to give an overall score for a tool, using the updated carving recall' and carving precision" scores: Cperformance = Recall = Carving recall = (α /carving recall) + ((1-α)/carving precision) Where α is a factor which determines the weighting of carving precision and carving recall. 1 Positive Positive + false negative all false negative This leaves the third quality measure, the reliability of a tool. Reliability does not state how well a tool works on all the files in a dataset, but only how successful it is at recovering the file types it claims to support. This lead to a modified version of the carving recall" measure, in which only supported files are taken into account: Supported recall = supported false negative Supported Where supported is the number of supported files in the dataset and sfn is the amount of Supported false negatives. D). Datasets First of all, these measures only work if the layout of a dataset is known, since the tool results have to be compared to the files present in that set. In 2005 Nick Mikus released datasets, based on an EXT2 file system, which are meant to test carving tools[6]. E). Testing Procedures Each tool is tested by running it on the datasets and comparing the results to the layout provided for that set. F). Score Interpretation Using the results of these comparisons, the quality scores can be created for each combination of a tool and a dataset. These scores each give a different insight into the quality and improvement possibilities of the tools. IV. PROPOSED ARCHITECTURE FOR QUALITY MEASUREMENT OF CARVING TOOL A). Proposed Architecture Use tool on raw disk image Extractio n of file Read the file for validation Mark each object file for validation 1. Compare files MD5 With disk image file 2. Read file in Hex code Resultant performan ce of file carving tools 1. Calculating carving precision 2. Calculating carving recall Performance measuring method implementation Visualiz ation PHASE (1) PHASE (2) PHASE (3) Fig.1 Proposed architecture for the quality measurement of carving tool. 423

4 National Institute of Technology Hamirpur (HP) B). Major Components 1). Extraction of file and marking Each tool is tested by running it on the datasets and comparing the results to the layout provided for that set. The marking of file after the extraction from the image file using the tool on the following bases: i. The MD5 sums are calculated over the carving results and compared to the MD5 sums of the files in the image, if provided in the layout description. Matching files are marked as true Positives. ii. The remaining carved files are checked against the remaining image files by comparing header and footer with the known file string of header and footer. Files with exact matching wit the header and/ or footer are marked as known false Positives. iii. The remaining files, which have not been marked as Known false positives are marked as unknown false positives. iv. The file that is not carved by the carver is known as false negatives. 2). Performance Measurement Phase The quality method is implemented on files carved by the file carver. To measure the quality of tool quality measurement system is given that discussed in section C of part III. 3). Result Analysis These results are analyse for the purpose of improvement of the future carving tools. The result give the insight view of the technique use by the tool that is been used for measuring the quality. Overall analysis provides measure for the improvement of future carving tools. This phase gives measure for analysing the carving tool. The improvement goals stated for the new carving framework can be divided into two categories: improve carving recall and improve carving precision. C).Proposed Procedure for Quality Measurement of Carving Techniques Input: Directory, documents Outputs: table of carving performance result Other variable: file carving recall, file caving precision 1. Read the contents of the caving result output folder and file 2. Read the MD5 of files given with image file 3. Read the MD5 of the carved file 4. Int a= read the no of carved file 5. Int b= (no of file in disk image) a 6. mark(b no of file are false negative) 7. while( carving result folder not empty) { 8. get a file a // get a file from carving result folder 9. for(int i=0;i<(no of file in image);i++){ 10. if(compare (MD5 of file given with image file to carved image file)) 11. mark file as positive 12. break } // mark and break 13. read file a hex code 14. if(compare its header and footer with given headers and footer ){ 15. mark file as known false positive break}else // mark and break 16. mark file as unknown false positive} 17. } //end of while loop 18. calculate file carving recall 19. calculate file carving precision 20. calculate Cperformance for tool 21. End V. RESULTS AND IMPLEMENTATION DETAIL A). Hardware and Software Requirements The experiments were setup in both windows & Linux environment.the Programs were developed in c for MD5 conversions and reading HEXCODE FILE B). Tested Tool and their Specification The carving tool that use for quality measurement by this architecture are: 1). Scalpel Created by Golden G. Richard III [4] Scalpel which is file system independent is used to carve files from FATx, NTFS, ext2/3, or raw partitions used to reads a database of header and footer definitions and extracts matching files from a set of image files or raw device files 2). Foremost Created by Nick Mikus [5] Foremost is used to recover files based on their headers, footers, and internal data structures. Supports file structure based carving for avi, bmp, doc, gif, hmlt, jpg, mov, pdf, png, rar, wav and zip files. C). Implementation Details Extraction and classification of file extracted by tool from raw disk image. The first phase of architecture is to extract of data using the tool that are using for testing in this architecture. The test image that is using is taken from the extraction of results from tool this test image is use as data set for the tools fig 2 and show the working and result of foremost tool and its output folder for carved file 424

5 National Conference on Emerging Trends in Computing and Communication (ETCC-2008) Fig 2 Working of Foremost carving tool Fig. 5 MD5 of image file Fig 3 Output summary result from foremost carving tool The results produced are classified into four following classifications based on tools need in experiment: -True Positive -Unknown false positive -Known false positive -False negative To calculate the MD5 of file HashMyfile Version 1.30 is used: Fig. 6 MD5 of carved file Fig 4 HASHMYFILE tool for MD5 calculation After calculating the md5 of carved file these md5 are matched with md5 of given image file the three snap shot show in Fig. 5,6,7 the result of comparison. Fig. 7 Result after compression The Fig. 8 show for the checking of partial files to check partial file first calculate the hex dump of each file this snap shot showing extraction of hex dump of a PDF file. 425

6 National Institute of Technology Hamirpur (HP) Fig 11 Grouping of carved file for Scalpel The Fig.12 show that provide comparison of the Cperformance of the tool two that been implemented on this framework Fig. 8 HEX code calculation of carved file D). Results The comparison of the carved file for the two tools (Foremost and scalpel) show in Fig. 9 Fig. 12 Cperformance of tested carving tools V. CONCLUSION Fig. 9 Number of file carved by different tool The Fig. 10 show that provide the graphical view of grouping of carved file for the Foremost carving tool and the next fig provide the grouping of file for the Scalpel A). Conclusion The results from raw disk image using the carving tools may have significant impact on the information available to an investigator; if tool produces accurate results then valuable information may be unveil. The quality and performance of carving tools is very crucial in digital forensic. In this work a quality measuring method in analysed for carving tools. Evaluations of tools in this method cover the aspects that is use to determine the overall quality of carving tools. B). Future work Area for future work in this quality analysis method is to improvement in the dependency on datasets with known unknown layouts and ameliorates in validation method or special purpose validators can be created which produce more accurate results. REFERENCES Fig. 10 Grouping of carved file for Foremost [1] [2] Carving Taxonomy URL: [3] [4] [5] [6] (Digital Forensics Tool Testing Images) 426