Black box Fault Finder for Dynamic Web Application Using Dynamic Test Generation

Transcription

1 Black box Fault Finder for Dynamic Web Application Using Dynamic Test Generation Anup P.Dabhade Dept. name of CSE Auroras Scientific Technological and Research Academy ASTRA Hyderabad, India V.Rajakumar. Dept. name of CSE Auroras Scientific Technological and Research Academy ASTRA Hyderabad, India ABSTRACT Web applications are the most widely used software in the internet. A web application is developed and deployed in the real environment, it is very severe if any bug found by the attacker or the customer or the owner of the web application. It is the very important to do the proper pre-analysis testing before the release. It is very costly thing if the proper testing of web application is not done at the development location and any bug found at the customer location. For web application testing the existing systems such as DART, Cute and EXE are available. These tools generate test cases by executing the web application on concrete user inputs. These tools are best suitable for testing static web sites and are not suitable for dynamic web applications. The existing systems needs user inputs for generating the test cases. It is most difficult thing for the human being to provide dynamic inputs for all the possible cases. Existing system: The tool called Apollo which can work on dynamic web applications. It can generate dynamic test cases for the dynamic web applications (PHP).The approach focus on Server-Side PHP-Code and some of client-side through web forms. The approach aims to identify two kinds of failures of web applications like Execution failures and HTML failures with the three modules called executor, bug tracker, and input generator. The algorithm used for to solve the problem is failure detection algorithm. Input for this algorithm is a server side php script with dynamically generated input. It produces a bug report of execution failures and html failures. Proposed System: The proposed approach is black box web application testing tool. Because of the tool is with black box nature, there is no need of web application server side code validation. It prepares the dynamic urls or test cases by putting the null, large, small and negative values. The test cases were passed on to the application to find the faults. 1. Introduction Web applications can be developed using server side programming languages(jsp,php,asp etc,.) or client side programming languages (Javascript, VBscript etc,.) or with static web pages (HTML) and with combination of the languages. In the dynamic web sites, developed with server side programming languages accepts inputs from the remote users. The remote user can pass web application expected inputs or can pass malicious inputs. If malicious inputs are passed to the web application, the behavior of the web application may change. The web application may crash at some instance. The web application response may be an unexpected that causes the browser to crash. The web application should be thoroughly tested for the functionality and in the security aspect

2 But When Ever we are going to deal with server-side programming, It givers an Dynamic response on basis of request to web application. Some time it should be happen that there should be error in dynamic response or we can say software bugs.such types bugs which some time crash the web browser or else the some part of imported information may be not displayed. In such case relevant information can t be display user. So for this purpose the sever-side coding must be check thoroughly. Here In this paper, we going to resolve problem associated previous paper, early exploration was detecting fault in opensource PHP application, in that case dynamic open-source PHP code as given the input to Apollo tool, the Apollo tool is specially implemented various faultlocalization techniques and test-generation strategies. But some organization don t want to expose coding of their organization due to fact that they have some privacy Statement.if in such case they want to test their web application without interacting with the sever-side coding is really an challenge to us. So as reach to the problem we have several technologies have being study on basis of that one imported software tool is available which work absolute URL pattern called as Black Box Testing without having knowledge of internal working of web Application. We are going to performing a Black Box testing, a tester will interact with the user system s interface by providing input to web Application and then examine outputs knowing how and where the inputs are work upon. There are some main important aspect of Black Box tool because code Access not required as well as efficient for large code segments. As large number of moderately skilled tester can test the application with no knowledge of implementation,programming language or operating. In previous work there main intension to determining where in the source code changes to be find failures in order to fix problem generally called as fault localization. 2.Definitions, acronyms, and abbreviations a. Software engineering (SE) is the application of a systematic, disciplined, quantifiable approach to the design, development, operation, and maintenance of software, and the study of these approaches; that is, the application of engineering to software. B.Software Bug: A software bug is an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. Most bugs arise from mistakes and errors made by people in either a program's source code or its design, and a few are caused by compilers producing incorrect code. C.Software Testing Methods: There are basically two software testing methods in Software engineering White Box Testing: White box testing is the detailed investigation of internal logic and structure of the code. White box testing is also called glass testing or open box testing. In order to perform white box testing on an application, the tester needs to possess knowledge of the internal working of the code. Black Box Testing: The technique of testing without having any knowledge of the interior workings of the application is Black Box testing. The tester is oblivious to the system architecture and does not have access to the source code. Typically, when performing a black box test, a tester will interact with the system's user interface by providing inputs and examining outputs without knowing how and where the inputs are worked upon. 3. Web Testing Procedures a. Functionality Testing: Test for all the links in web pages, database connection, forms used in the 255

3 web pages for submitting or getting information from user, Cookie testing. Check all the links: Test the outgoing links from all the pages from specific domain under test. Test all internal links. Test links jumping on the same pages. Test links used to send the to admin or other users from web pages. Test to check if there are any orphan pages. Lastly in link checking, check for broken links in all abovementioned links. Test forms in all pages: Forms are the integral part of any web site. Forms are used to get information from users and to keep interaction with them. So what should be checked on these forms? First check all the validations on each field. Check for the default values of fields. Wrong inputs to the fields in the forms. Options to create forms if any, form delete, view or modify the forms. Let s take example of the search engine project currently I am working on, In this project we have advertiser and affiliate signup steps. Each sign up step is different but dependent on other steps. So sign up flow should get executed correctly. There are different field validations like Ids, User financial info validations. All these validations should get checked in manual or automated web testing. b. Cookies testing: Cookies are small files stored on user machine. These are basically used to maintain the session mainly login sessions. Test the application by enabling or disabling the cookies in your browser options. Test if the cookies are encrypted before writing to user machine. If you are testing the session cookies (i.e. cookies expire after the sessions ends) check for login sessions and user stats after session end. Check effect on application security by deleting the cookies. (I will soon write separate article on cookie testing) c. Validate your HTML/CSS: If you are optimizing your site for Search engines then HTML/CSS validation is very important. Mainly validate the site for HTML syntax errors. Check if site is crawlable to different search engines. d. Database testing: Data consistency is very important in web application. Check for data integrity and errors while you edit, delete, modify the forms or do any DB related functionality. Check if all the database queries are executing correctly, data is retrieved correctly and also updated correctly. More on database testing could be load on DB, we will address this in web load or performance testing below. e. Usability Testing: Test for navigation: Navigation means how the user surfs the web pages, different controls like buttons, boxes or how user using the links on the ages to surf different pages.usability testing includes: Web site should be easy to use. Instructions should be provided clearly. Check if the provided instructions are correct means whether they satisfy purpose. Main menu should be provided on each page. It should be consistent. f. Content checking: Content should be logical and easy to understand. Check for spelling errors. Use of dark colors annoys users and should not be used in site theme. You can follow some standards that are used for web page and content building. These are common accepted standards like as I mentioned above about annoying colors, fonts, frames etc. Content should be meaningful. All the anchor text links should be working 256

4 properly. Images should be placed properly with proper sizes. These are some basic standards that should be followed in web development. Your task is to validate all for UI testing Other user information for user help: Like search option, sitemap, help files etc. Sitemap should be present with all the links in web sites with proper tree view of navigation. Check for all links on the sitemap. Search in the site option will help users to find content pages they are looking for easily and quickly. These are all optional items and if present should be validated. g. Interface Testing: The main interfaces are: a. web server and application server interface b. Application server and Database server interface. Check if all the interactions between these servers are executed properly. Errors are handled properly. If database or web server returns any error message for any query by application server then application server should catch and display these error messages appropriately to users. Check what happens if user interrupts any transaction in-between? Check what happens if connection to web server is reset in between? h. Compatibility Testing: Compatibility of your web site is very important testing aspect. See which compatibility test to be executed: Browser compatibility Operating system compatibility Mobile browsing Printing options Browser compatibility: In my web-testing career I have experienced this as most influencing part on web site testing. Some applications are very dependent on browsers. Different browsers have different configurations and settings that your web page should be compatible with. Your web site coding should be cross browser platform compatible. If you are using java scripts or AJAX calls for UI functionality, performing security checks or validations then give more stress on browser compatibility testing of your web application.test web application on different browsers like Internet explorer, Firefox, Netscape navigator, AOL, Safari, Opera browsers with different versions. i. OS compatibility: Some functionality in your web application is may not be compatible with all operating systems. All new technologies used in web development like graphics designs, interface calls like different API s may not be available in all Operating Systems. Test your web application on different operating systems like Windows, Unix, MAC, Linux, Solaris with different OS flavors. j. Mobile browsing: This is new technology age. So in future Mobile browsing will rock. Test your web pages on mobile browsers. Compatibility issues may be there on mobile. Printing options: If you are giving page-printing options then make sure fonts, page alignment, page graphics getting printed properly. Pages should be fit to paper size or as per the size mentioned in printing option. k. Performance testing: Web application should sustain to heavy load. Web performance testing should include: 1. Web Load Testing 2. Web Stress Testing Test application performance on different internet connection speed. In web load testing test if many users are accessing or requesting the same page. Can system sustain in peak load times? Site should handle many simultaneous user requests, large input data from users, Simultaneous connection to DB, heavy load on specific pages etc

5 Stress testing: Generally stress means stretching the system beyond its specification limits. Web stress testing is performed to break the site by giving stress and checked how system reacts to stress and how system recovers from crashes. Stress is generally given on input fields, login and sign up areas. In web performance testing web site functionality on different operating systems, different hardware platforms is checked for software, hardware memory leakage errors, l. Security Testing: Following are some test cases for web security testing: Test by pasting internal url directly into browser address bar without login. Internal pages should not open. If you are logged in using username and password and browsing internal pages then try changing url options directly. I.e. If you are checking some publisher site statistics with publisher site ID= 123. Try directly changing the URL site ID parameter to different site ID which is not related to logged in user. Access should denied for this user to view others stats. Try some invalid inputs in input fields like login username, password, input text boxes. Check the system reaction on all invalid inputs. Web directories or files should not be accessible directly unless given download option. Test the CAPTCHA for automates scripts logins. Test if SSL is used for security measures. If used proper message should get displayed when user switch from non-secure pages to secure pages and vice versa. All transactions, error messages, security breach attempts should get logged in log files somewhere on web server. 4. WEB APPLICATION There are two definitions of the types of web applications. The first from wiki.answers.com Client Side - Client side web application executes at client side (on browser) & all the resources of pages loads at client s side. Server Side - Server side web application executes at central server in which page life The second definition from oracle.com Presentation-oriented. A presentation-oriented Web application generates dynamic Web pages containing various types of markup language (HTML, XML, and so on) in response to requests. Service-oriented. A serviceoriented Web application implements the endpoint of a finegrained Web service. Serviceoriented Web applications are often invoked by presentation-oriented applications. Basically, these are the same. Presentation-oriented is like client side -- it is executed in the browser. Things like Google search, finding products, looking up phone numbers or zip codes. Service-oriented are like server side applications -- the main processing happens at the server end. For example, the DDP is an example of a service-oriented - server side web application ---so is a shopping cart, , etc.. The image below illustrates a clientserver web request

6 5. PATH MINIMAZATION ALGORITHM Find shorter path constraint for a given bug report which given by oracle tool W3C validator. On The basis of that Eliminates irrelevant constraints better assist programmer to detect location of the fault Solution for a shorter path constraint is among all listed bugs report often a smaller input is considered but, it should not guarantee returned path constraint is shortest that exposes failure here, in this algorithm we are possibly reduced our bugs report. suppose consider In given request there is home.html page, there are so many paths or links listed by validator showing the bugs in related page.we have to indentify last cast path to reduced the bugs report. a. Minimization Example We illustrate the minimization on the following example. The malformed HTML bugs can be triggered along different execution paths. For example, both of the following path constraints lead to inputs that expose the bugs here we take two path showing the same bugs. let s consider Path 1 <search.php?> a=10 & b=10 -> response page 1-> B1 (a) Path 2 <search.php?>a=10->response page 2-> B1 (b) Here in both the path 1 and 2 <search.php?>a=10 is common path for bug B1 so path minimization algorithm reduced path as follows Path <search.php?> a=10 & b=10-> for the response page having bug B1. 6. IMPLEMENTATION The Apollo dynamic web Application testing tool is previously work on opensource PHP Application but now we dealing with absolute URL so for this purpose we made some important changes over the previous Architecture so as to enhanced the functionality of Apollo testing tool here is some of the main requirement as follows. a. State manager. The module keeps track of all the pages to be visited If finds the absolute urls that are possible on the site under test. It maintains the state of the url (processed, to be processed, etc,.) b. Shadow Interpreter. It executes the server side program for testing.it interprets each url that are to be processed. It executes programs with, without, negative values, and large values as input arguments.it gathers the web pages produced for each interpretation. c. UI Option analyzer. It gathers all the input variables that are used in each server side program. It finds the arguments names such as name, id etc. It works based on syntactic analysis on the program. d. Symbolic driver. It consists of all the syntactic symbols and their states for identifying the input arguments. For example: GET_[ symbol ] symbol e. Constraint solver. It arranges the absolute urls with all possible combinations. Eg: login.php has id, pwd as input args. url 1: url 2: f. Input value generator. Based on the input argument type, an integer can have negative, positive values. It produces and assigns null values for the input arguments. It produces large values for integer, float, char etc. 7. Bug finder. The outputs of shadow interpreter are a web page. The page is validated against W3C validator. If W3C validator confirms with any bug, the url and the page is considered as reproducible scenario. W3C validator gives bugs report it not possible to pinpoint each bugs report because it s very large numbers. It must required to minimize the bug report for 259

7 purpose here, we are going to implement path minimization algorithm. Find shorter path constraint for a given bug report.eliminates irrelevant constraints better assist programmer to detect location of the fault.solution for a shorter path constraint is often a smaller input, Does not guarantee returned path constraint is shortest that exposes failure. Description: The systems architect establishes the basic structure of the system, defining the essential core design features and elements that provide the framework. The systems architect provides the architects view of the users' vision. In This diagram, the tool acts as an browser for user agent and accepts input from the user in form URL.once the url is placed it interact with the server and hits every pages stored them into remote location. Analyzer makes request/response table. All the response present with response table will be send to W3C validator. W3C validator validates each and every pages finds bugs. A software bug is an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. Reports detailing bugs in a program are commonly known as bug reports. This validated bugs reports will stored in bugs repository. The bugs reports we are getting is hug number that even we can t correct. Next to we apply bug minimization algorithm to minimize bugs reports. SYSTEM ARCHITECTURE Fig Shows Black box fault finder for dynamic web application using dynamic test generation. Incremental model The incremental build model is a method of software development where the model is designed, implemented and tested incrementally (a little more is added each time) until the product is finished. It involves both development and maintenance. The product is defined as finished when it sat is files all of its requirements. This model combines the elements of the waterfall model with the iterative philosophy of prototyping. Each component is delivered to the client when it is complete. This allows partial utilization of the product and avoids a long development time. It also avoids a large initial capital outlay and subsequent long waiting period. This model of development also helps ease the traumatic effect of introducing a completely new system all at once. Use of Incremental model This model is used in the following scenarios - Whenever we need early need of benefits. - Needs to get basic functionality early. - Whenever clients are not clear about their requirements. - Whenever there is a need of testing the initial requirement and have to identify the 260

8 pros and corns for the betterment of the system. So here client need an initial system and after deploying that system client is allowed to interact with the system. After using the system client will do specify modifications to this system or he will ask for the additional features.so in this scenario I would like to choose incremental model which satisfies the requirements of the client. A: Analysis 1. Existing System The tool called Apollo which can work on dynamic web application. It can generate dynamic test cases for the dynamic web application (PHP).The existing approaches are focus on Server-side PHP-code & some of client-side through web form. The existing take the input as open-source PHP only, hence it restricted to sever-side web application language. The main aims to find two kinds of failures of web application. Execution failures & HTML failures. The algorithm used for to solve the problem is failure detection algorithm. Input for this algorithm is a server side PHP script with dynamically generated input. 2. Proposed System The Proposed System take the input as URL (web address), hence it not restricted to sever-side web application language. For this, the proposed approach is black box web application tool. The technique of testing without having any knowledge of the interior working of the application. Hence, code Access not required, no knowledge of implementation, programming language or operating system. The proposed approach is black box web application testing tool. Because of the tool is with black box nature, there is no need of web application server side code validation. It prepares the dynamic URLS or test cases by putting the null, large, small and negative values. The test cases were passed on to the application to find the faults. B: Design: The design phase actually describes how the solution will work. System features are implemented in a series of short, time boxed iterations. Each iteration results in an executable release of the software. It is customary to write full text use cases during the construction phase and each one becomes the start of a new iteration. Common UML (Unified Modeling Language) diagrams used during this phase include Activity, Sequence, Collaboration, State (Transition) and Interaction Overview diagrams. 3. UML DIAGRAMS Unified Modeling Language (UML) is a standardized, general-purpose modeling language in the field of software engineering. The Unified Modeling Language includes a set of graphic notation techniques to create visual models of object-oriented softwareintensive systems. 3.1 Use Case Diagram 261

9 Definition: A use case is a description of set of sequences of actions including variants that a system performs to yield an observable result of value to an actor. Graphically a use case is rendered as an ellipse. Common Uses: We apply the use case diagram to model the static use case view of a system. We apply use case in one of the below two ways. a) To model the context of a system. b) To model the requirements of a system. In the above diagram we have a clear picture of the actors and the use cases that are related to the actors. Use of the diagram in the project: First clients are registered to enter into network. If it is valid then enter into the network. In this use case diagram which is used for the type of behavioral diagram defined by and created From a Use-case analysis. Once clients or user agent registered with the tool. It will be accept valid URL for the web-site to be validated. The tool will be authenticated with the sever by request/response combination and hits each server-side page. All The response pages will forward to the Validator which show list of bugs. Those bugs could be minimized with bug minimization algorithm. Final reduced bug report will be provided to user agent. 3.2 Class Diagram Definition: A class diagram shows a set of classes, interfaces and collaboration and their relationships. Common Uses: We can use the class diagram to model the static design view of the system. This view primarily supports the functional requirements of a system.the services the system should provide to the end users. We use the class diagram in the following ways. a) To model the vocabulary. b) To model simple collaboration. c) To model logical database schema. Use of the diagram in the project: In this class diagram, the user agents are able to place URL of the web application to be validated and that would be depends upon server response. As the class diagram which is used for the type of static structure diagram that describes the structure of a system by showing the system s classes, their attributes, operations and the relationships among the classes? Above diagram shows that how the classes are linked statically. Here local sever class diagram shows implementation of bug minimization algorithm depends on path constrain. 3.3 Sequence Diagram 262

10 Definition: Sequence diagram are used to represent or model the flow of messages, events and actions between the objects or components of a system. Time is represented in the vertical direction showing the sequence of interaction of the header elements which are displayed horizontally at the top of the diagram. Common Uses: We apply this diagram in the project in one of the following way. a) Distributed on the web based systems. b) Use case collaboration. c) Complex interaction. d) Complex Logic. Use of the diagram in the project: Sequence diagrams typically are associated with use case realizations in the Logical View Of the system under development. Here the tool accepts URL get authenticate with server to hit each and every pages that s pages are stored in remote location. Analyzer helps to build request/response table. All the responses are sending to onlinew3c validator. Validator shows list of bugs occurs in each dynamic response pages that are stored in bug repository. We apply algorithm path minimization constrain to reduce bug report. Finally reduced bugs report will send to tool. 3.4 Collaboration Diagram Collaboration Diagram Definition: Collaboration is a society of classes, interfaces and other elements that work together to provide some co operative behavior that s bigger than the sum of all its parts. Common Uses of the Diagram: We use the above diagram in one of the following ways within the project. a) To model the realization of uses cases. b) Modeling the realization of operation. c) Modeling a mechanism. Use of the diagram in the project: Processes can communicate between each other through URL accepts from user agent. On basis of accepted URL the testing tool able to hit server site pages dynamically Generating request/response table. All responses are sending to online validator that shows list of bugs. The available bags are reducing by using path constraint minimization. 3.5 State Chart Diagram: Definition: A state chart diagram shows a state machine, emphasizing the flow of control from state to state. A state machine is a behavior that specifies the sequences of states and object goes through lifetime in response to events, together with its response to its events. Common Uses: We use the state chart diagram to model the dynamic aspect of the system. The Dynamic aspect may involve the event-ordered behavior of any kind of object in any view of system Architecture including classes, interfaces, components and nodes. A state diagram is a type of diagram used is related fields to describe the behavior of Systems. State diagrams require that the system described is composed of a finite number of states. User Agent is placed URL into testing tool getting request/response from web server. All the responses coming from sever to tool are forward to online W3C validator them validator finds bugs. We implementing bug minimization constrain algorithms to minimize bugs reports

11 3.6 DEPLOYMENT DIAGRAM bug repository and minimize. Among them validator is specially used to finding bugs associated with dynamic web pages. Primary they are stored in repository and then send to bug minimization algorithm.minimizes list of bugs report. 3.7Activity Diagram Definition: A deployment diagram is a diagram that shows the configuration of a runtime processing nodes and the components that live on them. Common Uses: This diagram is used to model the static deployment view of a system. This view primarily addresses the distribution, delivery and installation of parts that make up the physical system. We use the deployment diagram in one of the three ways. a) To model the embedded system. b) To model client/server system. To model fully distributed systems. Use of the diagram in the project: Browser: Is a software application for retrieving, presenting and traversing information resources on the World Wide Web. Browsers are primarily intended to use the World Wide Web; they can also be used to access information provided by the web server in private networks or file system. In this project the browser is used by three actors interacting with the system. Web Server: The term web server can refer to either the hardware (computer) or the software (Computer application) that helps to deliver web content that can be accessed through the internet. The most common use of the web server is to host web sites but there are other uses such as gaming, data storage or running enterprise application. In this project we use this web server to run an enterprise application. Bug finder: Bug finder consists of three main component online W3C validator, Definition: Activity diagram shows the flow from activity to activity. An activity is an ongoing non atomic execution within in a state machine. Common Uses of the Diagram: Activity diagrams are commonly used in one of the following ways. a) Activity state and action states. b) Transitions. c) Objects. Use of the diagram in the project: As the diagram focuses on the transitions from activity to activity, the three different users have three activities in common i.e. I) place URL ii) gathering request/response from web server iii)finding bugs in web application. So the above activities have common joins to each other. The workflow of each actor is similar to that of the diagrams in sequence. 3.8 COMPONENT DIAGRAM Definition: A component diagram shows a set of components and their 264

12 relationships. Graphically a component diagram is a collection of vertices and arcs. Common Uses: We use the component diagram to model the static implementation view of a system. This view primarily supports the configuration view of a system. This view supports the configuration management of system parts, made up of components that can be assembled in various ways to produce the running system. We use component diagrams in one of the following four ways. a) To model the source code. b) To model executable releases. c) To model physical databases. d) To model adaptable systems. 7. Evaluation In this section, we report on experiments in which we measured the effectiveness of Apollo by implementing absolute URL as previously Apollo is effectiveness of finding bugs in PHP application. Hence it restricted only PHP language. Here we are working to black box approach since it not interacting with server-side code we are resolve the problem. 8. Conclusion We have presented a technique for finding bugs in dynamic web application that is based on absolute URL(user resource locator). We are producing the work is novel in several aspects. First, technique cannot analyze the dynamic pages, and cannot generate dynamic test cases. Here we implemented enhanced in Apollo, as earlier Apollo work on open-source PHP code. as input which not secured process because entire code is exposed. Now in proposed approach we imported black box software testing method which specially dealing with URL without having knowledge of internal working of code. Also we are uses am W3C validator as an oracle to determine situation where malformed HTML is created by the Application. Finally we minimized bugs report so time of error corrector is consumed. 9. Future Work The Apollo is suitable only for finding the errors in the document pages such as TML/JS/VS etc,. This testing approach uses the interpreters to generate the pages. Cannot be used for the PHP- security testing. So, the proposed system can be used for testing the page validation and security in future. In the security testing the main concentration will be on remotefile-inclusion attacks and local-fileinclusion attacks. 10. References [1] S. Anand, P. Godefroid, and N. Tillmann, Demand-Driven Compositional Symbolic Execution, Proc. Int l Conf. Tools and Algorithms for the Construction and Analysis of Systems, pp , [2] S. Artzi, A. Kiezun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M.D. Ernst, Finding Bugs in Dynamic Web Applications, Proc. Int l Symp. Software Testing and Analysis, pp , [3] M. Benedikt, J. Freire, and P. Godefroid, VeriWeb: Automatically Testing Dynamic Web Sites, Proc. Int l Conf. World Wide Web, [4] D. Brumley, J. Caballero, Z. Liang, J. Newsome, and D. Song, Towards Automatic Discovery of Deviations in Binary Implementations with Applications to Error Detection and Fingerprint Generation, Proc. 16th USENIX Security Symp., [5] C. Cadar, D. Dunbar, and D.R. Engler, Klee: Unassisted and Automatic Generation of High- Coverage Tests for Complex Systems Programs, Proc. USENIX Symp. Operating Systems Design and Implementation, pp , [6] C. Cadar and D.R. Engler, Execution Generated Test Cases: How to Make Systems Code Crash Itself, Proc. Int l SPIN Workshop Model Checking of Software, pp. 2-23, [7] C. Cadar, V. Ganesh, P.M. Pawlowski, D.L. Dill, and D.R. Engler, EXE: Automatically Generating Inputs of Death, Proc. Conf. Computer and Comm. Security, pp , [8] J. Clause and A. Orso, Penumbra: Automatically Identifying Failure-Relevant Inputs 265

13 Using Dynamic Tainting, Proc. Int lsymp. Software Testing and Analysis, [9] H. Cleve and A. Zeller, Locating Causes of Program Failures, Proc. Int l Conf. Software Eng., pp , [10] H. Cleve and A. Zeller, Locating Causes of Program Failures Proc. Int l Conf. Software Eng., pp , May [11] C. Csallner, N. Tillmann, and Y. Smaragdakis, DySy: Dynamic Symbolic Execution for Invariant Inference, Proc. Int l Conf. Software Eng., pp , [12] D. Dean and D. Wagner, Intrusion Detection via Static Analysis, Proc. Symp. Research in Security and Privacy, pp , May [13] C. Demartini, R. Iosif, and R. Sisto, A Deadlock Detection Tool for Concurrent Java Programs, Software Practice and Experience, vol. 29, no. 7, pp , June [14] M. Emmi, R. Majumdar, and K. Sen, Dynamic Test Input Generation for Database Applications, Proc. Int l Symp. Software Testing and Analysis, pp , [15] P. Godefroid, Compositional Dynamic Test Generation, Proc.Ann. Symp. Principles of Programming Languages, pp , [16] P. Godefroid, A. Kie_zun, and M.Y. Levin, Grammar-Based Whitebox Fuzzing, Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp , [17] P. Godefroid, N. Klarlund, and K. Sen, DART: Directed Automated Random Testing, Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp , [18] P. Godefroid, M.Y. Levin, and D. Molnar, Automated Whitebox Fuzz Testing, Proc. Network Distributed Security Symp., pp , [19] W.G. Halfond, S. Anand, and A. Orso, Precise Interface Identification to Improve Testing and Analysis of Web Applications, Proc. Int l Symp. Software Testing and Analysis,