Sample Report Security Test Plan Prepared by Security Innovation
Table of Contents 1.0 Executive Summary... 3 2.0 Introduction... 3 3.0 Strategy... 4 4.0 Deliverables... 4 5.0 Test Cases... 5 Automation... 5 Table of Attack Vectors... 7
1.0 Executive Summary Security Innovation is performing a penetration test of SIJamsAndJellies.com from March 1 st to March 15 th. This test plan outlines our strategy, the deliverables and the test cases to be run by for SI. Individual test cases are detailed, together with a description of tools to be used as well as expected results. 2.0 Introduction Security Innovation provides security testing services including test design, test management, and test execution to major application development companies. The cornerstone of SI's approach is the testing techniques and methodology developed by industry-leading security researcher Dr. James Whittaker, as outlined in his books, "How to Break Software" and "How to Break Software Security." In putting Dr. Whittaker's ideas to practice, SI has developed a balanced, methodological approach to analyzing applications and identifying areas of security concern. Our philosophy is that good testing requires good planning. However, good testing also requires a "look around" to understand the system as it really is used-- in a way that cannot be ascertained from a specification. As a result, we provide an equal mix of three different approaches to security testing: prescribed functional tests for security features (to ensure they work as they are specified), exploratory testing of the application to determine weak points, attack vectors and missing or extra functionality, and automated testing for common, high risk vulnerabilities such as buffer overruns. Security Innovation is performing security testing of SIJamsAndJellies.com. SIJamsAndJellies.com is an e-commerce web site specifically designed and built for testing needs. In this testing effort, SI is concentrating on three major areas: Common Website Attacks Security Functionality Tests Business Model Attacks This plan contains several sections. The strategy section describes, in broad terms, what our overall strategy was in developing this plan. The deliverables section discusses SI s responsibilities to the customer. The test cases section, the bulk of the document, describes the background, the automation strategy and outlines individual attack vectors for the SiJamsAndJellies.com. 3
3.0 Strategy A three step process is employed in developing and executing test cases: First, exploratory tests are performed to gain an understanding of the system that cannot be obtained through public knowledge, specification, etc. Then, individual vulnerabilities are tested, based on an understanding of the threats previously identified in the threat modeling exercise. Finally, any vulnerabilities identified are further tested to determine the risk of exploitation they represent. What we outline here are attack vectors for specific features. We refrain from developing individual test steps for two reasons: 1. Development of detailed test cases requires access to the SUT to ensure completeness. 2. We feel that development of test steps is best left to the creativity of the individual tester, and that over-specifying (especially in security testing) creates an artificial sense of thoroughness. When appropriate, attack vectors are tied to commercial or internal tools that we believe necessary to perform the tests. Further details on these tools will be given in the Test Cases section of this document. 4.0 Deliverables The deliverable for this contract is a test plan document including: Background and theory of operation Itemized attack vectors for each feature Use of test automation including tools needed In addition to the test plan, as part of its security audit, Security Innovation will Perform exploratory testing of the application to train testers on the system and develop additional test cases Perform automated testing for certain kinds of vulnerabilities that allows for automated testing, e.g. buffer overruns Perform manual tests as described in the test plan Report all vulnerabilities (and exploits, if developed) to the customer immediately upon discovery in individual problem reports Participate in regular conference calls and submit at least one written interim progress report Provide a final report documenting findings, together with all source code and executables, exploits, etc. developed during the testing 4
Certain kinds of security testing are not performed by Security Innovation, including: Physical security of the SIJamsAndJellies.com plant, servers, etc. Effectiveness of failover or redundant systems, power protection, etc. Protection from insider threats from employees or others with physical or electronic access Review of internal IT security policy Social engineering, industrial espionage, etc. Review of documentation or requirements for compliance with laws, standards or certification programs 5.0 Test Cases SI will focus on three key areas of security as part of this test engagement. Those areas are: Common Website Attacks Security Functionality Specific to SIJamsAndJellies.com Attacks Against the E-commerce business model The format of the test case table is as follows: The test case number, an internal number to the test plan and is used by SI to track test cases electronically during test execution. The attack scenario for the test case. A finer description of the test along with the necessary tool(s) to perform it. The expected result for the test case. Automation Certain kinds of security testing lend themselves more to automation than others. SI has as part of its toolset for security testing a number of proprietary programs that aid in performing a single task (such as data corruption) or in finding a certain category of vulnerabilities (such as keys left in memory.) At the very least, a basic set of tools for tasks like these is required. We would recommend: Ethereal Windows/Open Source ethereal.com From their site: "Ethereal is a free network protocol analyzer for Unix and Windows. It allows you to examine data from a live network or from a capture file on disk. You can interactively browse the capture data, viewing summary and detail information for each packet. Ethereal has several powerful features, including a rich display filter language and the ability to view the reconstructed stream of a TCP session." Ettercap Linux/Open Source ettercap.sourceforge.net "Ettercap is a multipurpose sniffer/interceptor/logger for switched LAN. It supports active and passive dissection of many protocols (even ciphered ones) and includes many features for network and host analysis." Ettercap for Linux is the only freely available packet sniffer that works as an SSL proxy. By using ettercap, SSL traffic can be captured and replayed for SSL replay attacks. 5
Nmap Windows/Open Source www.insecure.org/nmap From their site: "Nmap ("Network Mapper") is an open source utility for network exploration or security auditing. It was designed to rapidly scan large networks, although it works fine against single hosts. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (ports) they are offering, what operating system (and OS version) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics." HOLODECK/HEAT/I2 Windows/SI Proprietary These tools, which intercept function calls on the client AUT in order to perform fault injection and theft of key material attacks, presuppose the ability to run arbitrary code on the box. However, in trying to obtain a shared secret, such as a universal public key that compromises all systems if one system is compromised, these tools are useful. The highest automation return is achieved through automation of attacks that require lots of combinations such as buffer overrun testing. We would recommend a dedicated test bank of machines to perform corruption testing for file and network buffer overruns. In addition, some custom test automation or additional tools may be developed during test execution. Generally, we experience the need for additional tools at one of two times: during exploratory testing to achieve a specific task, and during testing, when a particular vulnerability is found and we feel related vulnerabilities may exist elsewhere in the product. SI Data Corruption Framework Windows/SI Proprietary SI has an in-house data corruption framework for finding file-based buffer overruns. It works by repeatedly corrupting a file, then loading that file in the application under test with a custom debugger test harness. The debugger test harness interprets (1) whether the application crashes and (2) what the likelihood of that crash being exploitable is (from "Very Low" to "Almost Certain"). This framework is useful in finding file-based buffer overruns in safe-forscripting and safe-for-data ActiveX controls such as Flash, Windows Media Player, etc. Hydra Proxy Linux/SI Proprietary A combination blind proxy and network corruption tool, Hydra is connected inline between a client AUT and server application to perform corruption testing on the client, server or both. By functioning as a normal router for most packets and performing corruption based on rulesets in loadable modules, Hydra allows for exact testing of a particular protocol implementation as well as general network and protocol corruption, without modification to the client or server system. Additionally, the same functionality can be used for in-stream modification of cookies, authentication keys and data parameters. HOLODECK WEB Windows/SI Proprietary Holodeck Web is an automatic fault injection test system for web-based applications. It incorporates a spider, which automatically discovers potentially vulnerable pages, an injector that injects code for common web vulnerabilities including buffer overflows, cross-site scripting, SQL injection, OS command injection, forceful browsing and paramater tampering, and an "oracle" that examines responses to determine the success of an attack. Holodeck web works with any HTTP or HTTPS application. 6
WEBCRACKER Windows/SI Proprietary WebCracker is an application that enforces password policies against form fields in web applications. It can apply a number of dictionary and brute force attacks, including discovery of usernames and social security numbers that correspond to a default PIN or password. Table of Attack Vectors Common Website Attacks Test Case CL1 CL2 CL3 CL4 CL5 CL6 Attack Vector Description Details and Tools Expected result Attempt SQL injection in the username and password fields on the login page. Attempt OS command injection in the username and password fields on the login page. Verify that there is server side validation of the input length so that long string injection methods cannot be used to create Denials of Service etc Study the hidden fields of the Login page and derive tests from the results Attempt cross-site scripting attacks on every input field on the Login page. Examine error messages returned by the application, to determine whether the server divulges too much information Login Page subject to SQL injection. Manual testing will be performed for complex tests. subject to OS comand injection. Manual testing will be performed for complex tests. Due to its nature, this test will be conducted during a special and scheduled test execution window The source of the page can be used to determine whether client-side security/input validation is implemented subject to cross-site scripting attacks. Manual testing will be performed for complex tests. Combinations of valid and invalid passwords and usernames should be tried. be vulnerable to SQL injection be vulnerable to OS command injection There should always be server-side input length checks. The server should not rely on hidden fields to prevent users from altering sensitive data be vulnerable to cross-site scripting Error messages should be the same for all combinations, so they don t disclosure information unnecessarily. 7
CL7 CP1 CP2 CS1 Attempt to uncover usernames/passwords using brute-force/dictionary attacks. Verify that there is server side validation of the input length so that long string injection methods cannot be used to create Denials of Service etc Study the hidden fields of the Products page and derive tests from the results Attempt SQL injection in the username and password fields on the Search page. Web Cracker can be used to determine whether it is easy to uncover valid usernames and passwords. Products Page Due to its nature, this test will be conducted during a special and scheduled test execution window The source of the page can be used to determine whether client-side security/input validation is implemented Search Page subject to SQL injection. Manual testing will be performed for complex tests. It should not be easy to uncover a password, due to password guidelines or a lockout of the account after three failed attempts. There should always be server-side input length checks. The server should not rely on hidden fields to prevent users from altering sensitive data be vulnerable to SQL injection CS2 Attempt OS command injection in the username and password fields on the Search page. subject to OS comand injection. Manual testing will be performed for complex tests. Fields on this page should not be vulnerable to OS command injection CS3 CS4 CS5 Verify that there is server side validation of the input length so that long string injection methods cannot be used to create Denials of Service etc Study the hidden fields of the Search page and derive tests from the results Attempt cross-site scripting attacks on every input field on the Search. Due to its nature, this test will be conducted during a special and scheduled test execution window The source of the page can be used to determine whether client-side security/input validation is implemented subject to cross-site scripting attacks. Manual testing will be performed for complex tests. There should always be server-side input length checks. The server should not rely on hidden fields to prevent users from altering sensitive data Fields on this page should not be vulnerable to crosssite scripting 8
CM1 CC1 Alter user input by modifying the HTML source of the My Cart page in an attempt to provoke an unexpected reply from the server Attempt SQL injection in the username and password fields on the Checkout page. My Cart Page This test includes modifying the menu items in drop-boxes, such as the Update drop-down box. Checkout Page subject to SQL injection. Manual testing will be performed for complex tests. Modifying the names of menu items should have no effect on the application (menu items should be referred to by their value rather than its name to limit such attacks) be vulnerable to SQL injection CC2 Attempt OS command injection in the username and password fields on the Checkout page. subject to OS comand injection. Manual testing will be performed for complex tests. be vulnerable to OS command injection CC3 CC4 CC5 Verify that there is server side validation of the input length so that long string injection methods cannot be used to create Denials of Service etc Study the hidden fields of the Checkout page and derive tests from the results Attempt cross-site scripting attacks on every input field on the Checkout page. Due to its nature, this test will be conducted during a special and scheduled test execution window The source of the page can be used to determine whether client-side security/input validation is implemented subject to cross-site scripting attacks. Manual testing will be performed for complex tests. There should always be server-side input length checks. The server should not rely on hidden fields to prevent users from altering sensitive data be vulnerable to cross-site scripting CI1 Study the cookie to determine whether it expires after a certain period of time. Cookie Tests This test ensures that a cookie would expire after a certain period of time (TBD). stay logged-in with the same cookie indefinitely. 9
CI2 CI3 CI4 CI5 Study the cookie to determine whether it becomes obsolete when a session ends Study the cookie to attempt to determine whether it can be predicted. Study the cookie to determine whether it can be replayed/spoofed Study the cookie to determine whether it can be stolen. This test ensures that a cookie cannot be reused indefinitely. The Hydra Proxy can be used to change values inline during a session. Observation of several cookie contents might give us an indication on whether a session ID can be predicted. The Hydra Proxy can be used to change values inline during a session and determine if a session can be replayed. Ettercap can be used to attempt capturing the cookie when it is sent during a secure session. An attacker should not be able to reuse an old cookie. An attacker should not be able to predict the content of a cookie. An attacker should not be able to replay/spoof a cookie to access data he does not have permission to. An attacker should not be able to steal a cookie. Security Functionality Tests Test Case F1 F2 Attack Vector Description Details and Tools Expected result NMap the web to determine what platform is used. Look for known vulnerabilities in Apache 1.3.29 This is a research test to determine what platform is used on the server side. This is a due diligence test to ensure that the server is not still vulnerable to old issues or variations of. N/A N/A Attacks Against the Business Model Test Case Attack Vector Description Details and Tools Expected result B1 Attempt to reach the Add to Cart or Checkout pages without authenticating. B2 B3 Attempt to modify the source of the Products page in an attempt to modify the price of the order Attempt to enter a negative quantity on the Products page. Forceful browsing past authentication checks can be used in order to determine whether knowing an attacker can access these pages without authenticating Both negative and small values would represent a risk to the SIJamsAndJellies.com business model Negative quantities of products would result in a refund instead of a charge. A malicious user should not be able to access these pages without authenticating. change the amount he/she has to pay for a service. enter a negative quantity. 10
B4 B5 B6 Attempt to enter a negative quantity on the Product Description page. Attempt to modify the source of the My Cart page in an attempt to modify the price of the order Attempt to log-in as another user without the proper credentials. Negative quantities of products would result in a refund instead of a charge. Both negative and small values would represent a risk to the SIJamsAndJellies.com business model Parameter tampering techniques can be used for this test. enter a negative quantity. change the amount he/she has to pay for a service. log in as another user without valid credentials. 11