Web Application Security Assessment and Vulnerability Mitigation Tests

White paper BMC Remedy Action Request System 7.6.04 Web Application Security Assessment and Vulnerability Mitigation Tests January 2011 www.bmc.com

Contacting BMC Software You can access the BMC Software website at http://www.bmc.com. From this website, you can obtain information about the company, its products, corporate offices, special events, and career opportunities. United States and Canada Address BMC SOFTWARE INC 2101 CITYWEST BLVD HOUSTON TX 77042-2827 USA Telephone 713 918 8800 or 800 841 2031 Outside United States and Canada Telephone (01) 713 918 8800 Fax (01) 713 918 8000 Fax 713 918 8000 If you have comments or suggestions about this documentation, contact Information Development by email at doc_feedback@bmc.com. Copyright 2010 BMC Software, Inc. BMC, BMC Software, and the BMC Software logo are the exclusive properties of BMC Software, Inc., are registered with the U.S. Patent and Trademark Office, and may be registered or pending registration in other countries. All other BMC trademarks, service marks, and logos may be registered or pending registration in the U.S. or in other countries. All other trademarks or registered trademarks are the property of their respective owners. AppScan, IBM, and Rational are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. BMC Software considers information included in this documentation to be proprietary and confidential. Your use of this information is subject to the terms and conditions of the applicable End User License Agreement for the product and the proprietary and restricted rights notices included in this documentation. Restricted Rights Legend U.S. Government Restricted Rights to Computer Software. UNPUBLISHED -- RIGHTS RESERVED UNDER THE COPYRIGHT LAWS OF THE UNITED STATES. Use, duplication, or disclosure of any data and computer software by the U.S. Government is subject to restrictions, as applicable, set forth in FAR Section 52.227-14, DFARS 252.227-7013, DFARS 252.227-7014, DFARS 252.227-7015, and DFARS 252.227-7025, as amended from time to time. Contractor/Manufacturer is BMC Software, Inc., 2101 CityWest Blvd., Houston, TX 77042-2827, USA. Any contract notices should be sent to this address.

Customer Support You can obtain technical support by using the Support page on the BMC Software website or by contacting Customer Support by telephone or email. To expedite your inquiry, please see Before Contacting BMC Software. Support website You can obtain technical support from BMC Software 24 hours a day, 7 days a week at http://www.bmc.com/support_home. From this website, you can: Read overviews about support services and programs that BMC Software offers. Find the most current information about BMC Software products. Search a database for problems similar to yours and possible solutions. Order or download product documentation. Report a problem or ask a question. Subscribe to receive email notices when new product versions are released. Find worldwide BMC Software support center locations and contact information, including email addresses, fax numbers, and telephone numbers. Support by telephone or email In the United States and Canada, if you need technical support and do not have access to the Web, call 800 537 1813 or send an email message to customer_support@bmc.com. (In the Subject line, enter SupID:<yourSupportContractID>, such as SupID:12345.) Outside the United States and Canada, contact your local support center for assistance. Before contacting BMC Software Have the following information available so that Customer Support can begin working on your issue immediately: Product information o Product name o Product version (release number) o License number and password (trial or permanent) Operating system and environment information o Machine type o Operating system type, version, and service pack o System hardware configuration o Serial numbers o Related software (database, application, and communication) including type, version, and service pack or maintenance level Sequence of events leading to the problem Commands and options that you used Messages received (and the time and date that you received them) o Product error messages o Messages from the operating system, such as file system full o Messages from related software

License key and password information If you have a question about your license key or password, contact Customer Support through one of the following methods: E-mail customer_support@bmc.com. (In the Subject line, enter SupID:<yourSupportContractID>, such as SupID:12345.) In the United States and Canada, call 800 537 1813. Outside the United States and Canada, contact your local support center for assistance. Submit a new issue at http://www.bmc.com/support_home

Contents System architecture... 7 AppScan test results... 8 OWASP Top Ten: AR System protections... 11 General guidelines... 14 Encryption... 14 Secure Socket Layer... 14 Secure Tomcat installation... 14 Session management... 15 HTTP TRACE disabled... 15 XSS filter enhanced... 16 Data Visualization module plugins... 16 Mid tier Return Back parameter... 16 Mid tier and portlet containers... 16

White paper Web Application Security Assessment and Vulnerability Mitigation Tests This paper highlights the IBM Rational AppScan automated assessment process for web application security that BMC implements for the BMC Remedy Action Request (AR) System. It also provides a list of security protections that BMC provides to mitigate against vulnerabilities outlined in the Open Web Application Security Project (OWASP) Top Ten list. Note: The IT environment and network infrastructure in which your AR System runs must be properly secured and include standard IT network security tools and systems such as firewalls and intrusion detection systems (IDS). The following AR System security-related information is available on the Customer Support website at http://www.bmc.com/support: BMC Remedy AR System 7.6.03 Encryption Security Guide BMC Remedy AR System 7.5.00 Installation Guide - Mid-tier post-installation procedures section BMC Remedy AR System 7.5.00 Configuring You Web Server and Installing BMC Remedy Mid Tier with a.war File white paper Web Application Security Assessment and Vulnerability Mitigation Tests 6

System architecture System architecture The AR System architecture is multi-tiered; it consists of a Presentation layer, a Logic layer, and a Data layer as shown in Figure 1. Figure 1. AR System security architecture diagram Presentation layer The Presentation layer consists of the web browser client connected to the mid tier with secure socket layer (SSL) encryption. You must implement SSL to secure the connection between the browser and the web server. BMC supports any SSL version that is supported by the HTTP web services vendors listed in the BMC Remedy AR System Compatibility Matrix, which is available on the Support website. Web Application Security Assessment and Vulnerability Mitigation Tests 7

White Paper Logic layer The Logic layer includes instances of a mid tier, a JavaServer Pages (JSP) engine, a web server, and the AR System server. The JSP engine and accompanying servlets provide dynamically generated HTML and XML documents in response to web client requests. The mid tier installer includes and can automatically install a bundled version of the Tomcat web server. The mid tier translates client requires, interprets responses from the AR System server, handles web service requests, and runs server-side processes that present AR System functionality to the client from the AR System server. The server executes workflow and business logic that define all AR System applications. Because all AR System clients are API-based, turning on encryption ensures that all interactions with the server are encrypted. Data layer The Data layer consists of one or more databases, which perform data storage and retrieval functions. The AR System server connects to the Data layer using database client API libraries. The server can work with the database encryption libraries used to protect data that is transmitted between the server and database. 8 Web Application Security Assessment and Vulnerability Mitigation Tests

AppScan test results AppScan test results BMC uses IBM Rational AppScan, a Web 2.0 security assessment tool, as an integrated part of the software development life cycle (SDLC). By performing a wide range of early detection testing, BMC identifies and fixes or mitigates vulnerabilities before they become security risks. AppScan provides issue severity levels and detailed descriptions as well as advisories and issue solution recommendations for potential security risks related to AR System components. BMC uses this data to investigate and proactively resolve security issues. Figure 2 shows a sample AppScan results page. Figure 2. Sample AppScan test result window Web Application Security Assessment and Vulnerability Mitigation Tests 9

White Paper Table 1 lists the AppScan version 7.8 test results. No high-severity vulnerabilities were detected in the AR System mid tier version 7.5.00 patch 7. Table 1. AppScan test results AR System Servlet AdminServlet ApplicationServlet AttachServlet Test Result False vulnerabilities were detected. This AR System servlet is implemented in the web service module. Users must provide a user name and password when the service is requested. False vulnerabilities were detected. An error page notifies users that a session is not valid. BackChannelServlet FBImageServlet Flashboard_params FormsServlet HomeServlet Imagepool ImageServlet LicenseReleaseServlet LoginServlet LogoutServlet Plugineventester ProtectedWSDLServlet ReportServlet False vulnerabilities were detected. The embedded script is not executed. It is reported as an error. In addition, an error is logged and appears in the status bar. Access is not allowed. False vulnerabilities were detected. An error is logged and appears in the status bar. Access is not allowed. False vulnerabilities were detected. The mid tier responds with an error page. 10 Web Application Security Assessment and Vulnerability Mitigation Tests

OWASP Top Ten: AR System protections AR System Servlet Report_params ResourceServlet ViewFormServlet Test Result False vulnerabilities were detected. When URL parameters are sent, BMC advises users to deploy HTTP over SSL. False vulnerabilities were detected. The embedded script is not executed. It is reported as an error. When URL parameters are sent, BMC advises users to deploy HTTP over SSL. OWASP Top Ten: AR System protections Using AppScan, BMC specifically tests for vulnerabilities identified in the Open Web Application Security Project (OWASP) Top Ten list. Security risks identified by OWASP and AR System protections are listed and described in Table 2. Table 2. AR System protections against the OWASP Top Ten Sample risk OWASP description AR System protections Injection Attackers trick a process into calling external processes of their choice by injecting control-plane data into the data plane. Command injection has two forms: An attacker changes the command that the program executes, explicitly redefining the command. An attacker changes the environment in which the command executes, implicitly redefining the command. To prevent command injection, AR System disables server-side scripting. To prevent JavaScript and SQL injection, AR System: Encloses all dates in quotes and escapes all quotes. Uses filters for escape characters. Provides strong-types and usersupplied fields. Checks for type constraints. To prevent blind SQL injection, AR System properly filters escape characters. Secures variables with strong types and validation. Sets security privileges on the database to least required. Web Application Security Assessment and Vulnerability Mitigation Tests 11

White Paper Sample risk OWASP description AR System protections Cross-Site Scripting (XSS) Broken Authentication and Session Management Insecure Direct Object References Cross-Site Request Forgery (CSRF) Security Misconfiguration Attackers can make a single request to a vulnerable server that causes the server to create two responses. The second response might be misinterpreted as a response to a different request, possibly one made by another user sharing the same TCP connection with the server. Attackers can bypass authentication mechanisms if credentials do not accompany every request. Attackers force the return of sensitive information instead of non-sensitive information that would be returned normally. Using this technique, attackers make victims perform actions that they did not intend to, such as logging out, purchasing items, or other functions provided by the vulnerable website. The victim s browser is tricked into issuing a command to a vulnerable web application. The vulnerability is caused by browsers automatically including user authentication data such as a session ID, IP address, or Microsoft Windows domain credentials with each request. This attack involves exploiting insecure configurations. All user-supplied HTML special characters are encoded into character entities, thereby preventing them from being interpreted as HTML. All requests contain credentials. The mid tier does not use cookies. It uses a cache ID in the URL and controls the user role (such as the Admin role.) AR System uses web server session management to store AR System authentication into the HTTPS session. All object references are subject to permissions enforced by the AR System server. The AR System disables web server scripting in the mid tier. In addition, logic that runs processes on the AR System server is restricted by the AR System permissions model, and processes that may be run are restricted to specific directories on the server. AR System configuration guidelines ensure secure operation. For example, AR System restricts user access to directories required for user operations, and AR System validates all user input. 12 Web Application Security Assessment and Vulnerability Mitigation Tests

OWASP Top Ten: AR System protections Sample risk OWASP description AR System protections Insecure Cryptographic Storage Failure to Restrict URL Access Insufficient Transport Layer Protection Unvalidated Redirects and Forwards The most common flaw in this area is simply not encrypting data that deserves encryption. When encryption is employed, unsafe key generation, nonrotating keys, and weak algorithm usage is common. The use of weak or unsalted hashes to protect passwords is also common. External attackers have difficulty detecting such flaws due to limited access. Attackers may access pages beyond the login page without authorization. Attackers may intercept unprotected network traffic if only SSL or TLS is used during authentication. Applications frequently redirect users to other pages, or use internal forwards in a similar manner. Sometimes the target page is specified in an unvalidated parameter, allowing attackers to choose the destination page. All sensitive data is encrypted within AR System. All communication between the web browser and the web server can be encrypted using HTTPS. All communication between the web server and the AR System server can be encrypted using API encryption. All access to all AR System pages require authorization from the AR System server. AR System uses transport layer security and digital signatures to perform end-to-end validation after a connection is made to an endpoint. FIPS-compliant Performance and Premium Encryption add-on components are provided for additional cryptographic protection among AR System components. All AR System parameters are validated and authenticated against user credentials. Web Application Security Assessment and Vulnerability Mitigation Tests 13

White Paper General guidelines This section describes general security guidelines to consider when using AR system. Encryption AR System provides BMC Remedy Encryption Performance Security and BMC Remedy Encryption Premium Security components that you can install to provide well-protected communication among AR System components. Performance Security includes a Federal Information Processing Standard (FIPS) encryption option. When this option is enabled, network traffic is encrypted using AES CBC with a 128-bit key for data encryption and a 1024-bit modulus for the RSA key exchange. It uses SHA-1 for message authentication. This option supports the minimum FIPS 140-2 encryption requirements. Premium Security includes a premium FIPS encryption option. When this option is enabled, network traffic is encrypted using AES CBC with a 256- bit key for data encryption and a 2048-bit modulus for the RSA key exchange. It uses SHA-1 for message authentication. This option supports premium FIPS 140-2 encryption requirements. Secure Socket Layer You should use secure socket layer (SSL) to encrypt the traffic between the HTTP web server and the browser client. Configuring the environment for SSL support is beyond the scope of any guidance that BMC provides. Note that enabling SSL can impact performance due to the extra overhead required to encrypt and decrypt traffic. Secure Tomcat installation Because the Tomcat JSP engine is bundled with the mid tier, the AR System installation script performs the following clean-up tasks to ensure that security issues in Tomcat are resolved: Removes the contents of the root directory from the Tomcat_installation_directory/webapps directory. Adds an index.html file to the root directory. This file appears if the administrator enters http://localhost:8080 in a browser and Tomcat is running properly. Removes the tomcat-docs directory from the Tomcat_installation_directory/webapps directory. 14 Web Application Security Assessment and Vulnerability Mitigation Tests

General guidelines Removes the host-manager and manager web default web applications from the Tomcat_installation_directory/webapps/server/webapps directory. Removes the deployment descriptors for the host-manager and manager applications from the Tomcat_installation_directory/conf/Catalina/localhost. directory. The descriptors are the host-manager.xml and manager.xml. Removes all unused ports from service (in particular, port 8080). It strips the default server.xml configuration file in the Tomcat installation directory so that the installation supports the mid tier only. These tasks make the Tomcat installation more secure; however, it can be difficult to determine if the mid tier or if the Tomcat engine failed to install properly because all extraneous services are removed. To ease this problem, an index.html page that displays when Tomcat is running is also installed. If the mid tier fails to run after installation, complete the following steps to determine whether the problem is the Tomcat installation or the mid tier installation: 1. Stop Tomcat. 2. Open the Tomcat_installation_directory/conf/server.xml file and uncomment the Connector entry at port 8080. 3. Restart Tomcat. 4. In a browser on the same computer as the Tomcat installation, go to http://localhost:8080. If the Tomcat engine is running properly, the message: Tomcat is running displays in the browser. Session management HTTP TRACE disabled If a session between the web browser and the mid tier is idle for 90 minutes or if the user closes a browser, the AR System license is released. You can configure idle time parameters in the Mid Tier Configuration tool. HTTP TRACE is a default function in many web servers, primarily used for debugging. The client sends an HTTP TRACE request with all header information including cookies, and the server simply responds with that same data. To prevent cross-site tracing (XST) attacks that use XSS and the HTTP TRACE function, the HTTP TRACE function in the mid tier is disabled by default. To disable the HTTP TRACE function completely, you must also disable HTTP TRACE on the application server hosting the mid tier. For information about how to enable the TRACE function, see HTTP tracing in the mid tier in the BMC Remedy Mid Tier Guide. Web Application Security Assessment and Vulnerability Mitigation Tests 15

White Paper XSS filter enhanced By default, the mid tier contains an XSS filter that is frequently updated with additional characters. Data Visualization module plugins By default, security is disabled for data passed through the mid tier using the data visualization model plugins. To enable mid tier security for the plugins, you must add the following option to the config.properties file: arsystem.plugin_securitycheck=true Mid tier Return Back parameter The default value of the Return Back parameter is false. You must change the value to true to prevent the mid tier from allowing a user to submit a URL containing a Return Back parameter. To change the value, add the following setting to the config.properties file and restart the mid tier: arsystem.allow.returnback.url=true If the default value is not changed, arsystem.allow.returnback.url could allow users to alter a base return URL when the URL is sent back to the browser from the web server. This behavior could make the system vulnerable to a phishing attack. Mid tier and portlet containers To prevent frame phishing vulnerabilities in the mid tier, the mid tier verifies that it is not placed inside a portlet container and/or displayed in third-party frames or iframes. If a portlet container, third-party frame, or iframe is detected, the mid tier automatically disconnects from the object and displays the content in a single window. 16 Web Application Security Assessment and Vulnerability Mitigation Tests

178629 *187116*