DATABASE AUDITING TOOLS AND STRATEGIES

Transcription

1 DATABASE AUDITING TOOLS AND STRATEGIES Authored by: Ed Chopskie, Vice President SenSage, Inc.

2 TABLE OF CONTENTS Introduction.. Page 3 Native Database Auditing.... Page 4 Vendor Native Auditing Capabilities... Page 4 Limitations of Database Management Systems... Page 5 Third-Party Database Auditing Solutions... Page 6 Trade-offs.... Page 6 Types of Third-Party Solutions.... Page 6 Software Agents and Probes... Page 8 Features of Database Auditing Agents. Page 8 Log Management Solutions for Database Auditing.. Page 10 Meeting Compliance Requirements... Page 11 Ad Hoc Queries, Reports and Alerts.. Page 11 Mini Case Study. Page 11 A Hybrid Approach... Page 12 Summary..... Page 12 About the Author... Page 13 Introduction to Database Auditing 2

3 Introduction Database management systems such as Oracle, Sybase, Microsoft SQL Server, IBM DB2 and IBM IMS all contain the ability to create audit records of all transactions that access the data stored in their databases. Additionally, these database management systems also provide the ability to audit changes to the structure and access to the database. Each product provides different levels of granularity on what can be audited and the audit features are bundled with the database at no additional cost. Audit records created by the database management systems should not be confused with re-do or archive logs that are created by the database for any data update. Those logs are used exclusively for forward database recovery and are in a format that is optimized for a recovery. They contain basic information limited to only changed data and physical location in the table where it was placed. Additionally, re-do and archive logs do not contain any record of SQL SELECT statements, as a SELECT statement does not modify data. Unfortunately, most native database auditing capabilities add overhead to database processing and the amount can be an issue in some situations. Native database audit tools also contain minimal functionality other than creating records of database access. Some native database management system audit tools such as Oracle, DB2 z/os and IMS do provide some audit record storage, reporting and alerting tools, but these tools often do not meet the segregation of duties requirements that auditors require. Microsoft SQL Server and Sybase contain minimal auditing tools and very poor ability to store and report on audit records. Finally, no database management system provides the ability to detect access and changes in real-time. For these reasons, a new set of software tools, commonly referred to as Enterprise Database Auditing and Real-Time Protection have come to market that provide lowoverhead audit collection with storage, alerting and reporting capabilities. Organizations evaluating a strategy for auditing databases should consider the trade-offs regarding OPEX (operating expense), CAPEX (capital expenditure), and accuracy. The purpose of this paper is to introduce the current options available for database auditing and trade-offs using each. Introduction to Database Auditing 3

4 Introduction to Native Database Auditing Native database auditing tools are turned off by default when a database is installed and must be enabled and managed by database administrators (DBAs). Examples of native database auditing tools provided by the leading databases are: IBM DB2 z/os and IMS: Audit traces that are provided at no cost and optionally combined with an additional product (at an additional cost), the IBM Audit Management Expert (AME) stores audit records in a DB2 table. Oracle: Fine Grained Auditing (FGA) that dumps audit records into XML records or optionally combined with an additional product (at an additional cost), the Oracle Audit Vault. Microsoft SQL Server: C2 auditing and Server-Side traces. Microsoft does not provide significant optional storage, reporting or alerting tools. Sybase: Minimal native support. Provides an additional product, Sybase Data Auditing (at an additional cost), which is a re-branded third-party product from Lumigent. Each of these native audit capabilities can be configured to be extremely granular down to the table name or in the case of Oracle FGA, down to specific SQL statements and columns. Additionally, the database management vendors continue with each release of their products to increase capabilities while reducing overhead requirements of utilizing their audit traces. Vendor Native Auditing Capabilities Some vendors are further ahead than others with Oracle arguably being the most advanced provider of native auditing capabilities and Microsoft SQL Server being the laggard according to industry analysts. For example, SQL Server C2 auditing, introduced in SQL Server 2000, does not provide the ability to specify individual tables to trace and therefore requires administrators to use server-side traces if they want to limit the tables audited in database instance (the use of placing triggers on tables will not catch SQL SELECT statements and therefore will not meet serious audit requirements). This functionality has not been significantly improved in SQL Server 2005 and while requiring DBA support to continuously maintain (the traces must be turned on manually whenever the database is restarted), it remains functional. The exact details are beyond the scope of this paper but are documented by Microsoft at TechNet. Regardless of the steps required to enable SQL-level auditing (or DL/I in the case of IMS) in the database management system, they each provide the ability to document all access to individual tables including the SQL statement issued, the userid associated with the transaction, and the time. Introduction to Database Auditing 4

5 For example, consider a user who works at a luxury department store in Chicago who has privileged (DBA) authority. This user decides to access a table (named production.transation_details ) containing credit card holder details using the following SQL command (shown in Figure 1.) from anywhere on the network: select * from production.transaction_details where first_name = Oprah and last_name = Winfrey Figure 1. In general, each database management system, with SQL auditing enabled on the production.transaction_details table will capture this event with a record that loosely resembles the following (shown in Figure 2.) Timestamp SSID USER_ID SQL_STATEMENT 4/25/ :33:15 AM PROD01 echopskie select * from production.transaction_details where first_name = Oprah and last_name = Winfrey Figure 2. Any SQL statement including statements that view and manipulate the data in the table (known as DML or Data Manipulation Language Statements) or SQL statements that modify the structure of the table or change access privileges to the table (known as DDL or Data Definition Language Statements) can be captured. For example, consider the same privileged user who grants read or update access to the same table to one of his colleagues with a user id of jpflaging, the DDL statement will look something like the statement in Figure 3. This SQL transaction can also be logged. Additionally, the command that a privileged user who turns off a trace using a DDL command is also logged, making audit traces very effective at capturing potentially fraudulent behavior by both privileged and non-privileged users. Grant all on table production.transaction_details to jpflaging Figure 3. Limitations of Database Management Systems While the database management systems allow the capture of these transactions, they provide limited capability to effectively store and query the audit records. While some of the vendors, specifically IBM and Oracle, do provide optional database repositories to store the audit logs, these solutions are often maintained by the same DBAs who manage the database management systems being audited, violating a segregation of duties audit requirement. This issue prompts many organizations into investigating using a database auditing solution, a log management solution or a combination of both to effectively capture, store and report on database log records. Introduction to Database Auditing 5

6 The advantage of using native database auditing tools is 100% accuracy as any database access will be captured and defeating the audit capability can only be accomplished by turning off the database audit trace. The disadvantages include both possible CAPEX and OPEX increases. Using the native database auditing will impact database performance, which might require more powerful servers for the database instances to run on. Fortunately, the database vendors continue to improve the performance impact of their native database auditing. Skilled database administrators can tune the audit impact by reducing the traces to the proper tables and moving audit log destination files to separate (and fast) disks away from the database files. OPEX increases are less significant and this includes the time required to ensure that traces are running. Third-Party Database Auditing Solutions Third-party database auditing solutions are currently offered by more than two dozen vendors, mostly small startup companies with few customers. Each of the leading solution providers claim that they all do not require that native database auditing be enabled and use this as the leading feature and benefit of their solution. The third-party database auditing vendors are quick to point out the significant overhead of using native tools, but in reality the amount of overhead may be acceptable for the benefits gained and cost avoidance. Trade-offs There are trade-offs, however, in the techniques used by these third-party solutions. The most significant impact is the amount of CAPEX required from the purchase of additional appliances and software. And unlike the native database auditing tools, third-party solutions can be actively defeated by knowledgeable users who are aware of the weaknesses of the products such as what types of access are not captured. Additionally, these solutions can be defeated accidentally from changes in the infrastructure, tasks failing to start or database management system level changes. Each solution offers the promise of not incurring any database overhead that occurs by using native database auditing and this claim is somewhat true. Before examining the marketing claims of the database auditing solutions, a quick overview of their approaches to database auditing, followed by the problems and trade-offs with each will face follows. Types of Third-Party Solutions There are primarily three types of approaches being offered by the leading enterprise third-party solutions: 1) Network-based appliances that sniff SQL packets on the wire as they are sent to the database management system from client connections Introduction to Database Auditing 6

7 2) Software-only solutions that are deployed with or without the use of agents that attempt to interrogate the shared memory of the database management system to gather the SQL statements 3) Some combination of the first two approaches The first approach is problematic as it attempts to read packets on the wire as they make their way to the database management system. This approach only works for remote access to the database server and not for local access. Some SQL activity might not be initiated via remote client access and may actually be initiated on the actual server containing the database management system instance. As there would be no network packets containing SQL statements to detect this access, all network sniffing approaches are defeated by this type of access. Typically DBAs have complete access to the servers running their database instances and their activity, if initiated from the server, would be undetected by a packet sniffing solution. Additionally, if an application server that accesses the database is installed on the same device as the database, a packet sniffing solution will not be able to capture the SQL statements executed on the database server from the app server. Several technical problems also exist for packet sniffing solutions including the amount of appliances required, compatibility with network devices, and the ability to inspect encrypted packets on the network. The maintenance required for deploying and maintaining the sniffing appliances at the proper physical location on the network to ensure that they are capturing all SQL packets as they approach the server can be daunting. The sniffing approach could require dozens or more appliances to be deployed and maintained in a production network, significantly adding to CAPEX as many of the appliance solutions have list prices between $25K to $50K per appliance. Consider the following diagram (Figure 4.) provided by a leading provider of database auditing solutions (Guardium) and the amount of appliances that must be deployed and maintained to actively collect and store audit records as they are collected on the network that shows the requirement of eight appliances to monitor 12 database instances. Figure 4. Introduction to Database Auditing 7

8 As previously stated, some network devices either do not support or are incapable (the CPU of the switch will be unable to process) of the duplex mode required by packet sniffing solutions and, therefore, the appliances are unable to capture and inspect the packets. This issue is a problem for all packet sniffers and not unique to database auditing tools that use packet sniffing. Finally, many sniffers used for database auditing (and data leakage tools as well) are rendered useless for inspecting any traffic on the network that is encrypted. Software Agents and Probes To overcome the problems with deploying packet sniffing appliances throughout a network, some solutions are deployed as software agents or probes. Some appliance solutions such as Guardium actually provide probes (Guardium calls theirs the S-TAP software probe) to address the weaknesses with their network sniffing approach. Software agents and probes are typically deployed on the database servers or application servers that frequently access the database management system. While installing agents on every database server is a rational approach to auditing database activity and does not have the issues that network appliances have, the approach suffers from the classic problems of having to deploy/maintain software agents, the actual server coverage, and the CPU overhead required by the agent/probe processes. Vendors admit this is around 5%, which will likely negate any possible performance gains over using native database auditing. Additional problems such as the availability of the agent/probe process, the ability to terminate the process by privileged users, and the processes ability to keep up with the database processing also exist. Features of Database Auditing Solutions While the database auditing solutions each have deficiencies over native database auditing in terms of accuracy and coverage, these tools do offer some compelling functionality that the native tools do not provide. These features specifically include: Policy creation Real-time policy violation detection Alerting Segregation of duties Policy Creation Policy creation involves creating a set of rules that alert security administrators to an event that is not allowed by the organization. The creation of policies is an iterative task and no product can provide a complete set of rules out-of-the-box. Security administrators start with a basic set such as changes to permissions and new user creations. Recall the example in Figure 3. where a DBA user granted complete access to a table to another user. Typically security administrators would review this activity to discover if it was properly documented in a change control system. By looking at the SQL statement alone, it is not possible to determine if a policy violation has occurred in Introduction to Database Auditing 8

9 most cases but the information provides a starting point to review the change to determine if the DBA was authorized to grant the user the level of permission that was granted. Database auditing tools also provide other generic policy review reports such as failed logins, new object creations, etc. Using a sophisticated policy editor, it is possible with database auditing products to create alerts and reports for more granular events such as individual table access or changes to data that should not change very often. Consider the example in Figure 1. where a user accessed the records of a customer in a table containing sensitive data. A rule that creates alerts on access to this table may or may not be useful as many users are likely to be authorized as part of their jobs to access this data. A sophisticated policy editor would allow security administrators to exclude users that are known to have access and only report on other users who do not have day to day responsibility to accessing sensitive data. Additionally, a table that should have very few updates can be audited for all update activity to insure that changes to data, such as invoice amounts, are authorized. Real-time Policy Violation Detection Real-time policy violation detection is often hyped by database auditing vendors as the holy grail of detecting security violations and is touted by vendors as the major advantage over using native database auditing. Recall that native database auditing collects events that occurred in the database instance and writes those events to a file. The file then must be processed with another tool (some database vendors provide some tools but others do not) and this processing occurs some time after the events are created. While real-time detection of events sounds reasonable, it is much harder in practice and the benefits over periodic alerts are marginal. Recalling the examples used previously, sending a real-time alert to a security administrator about the creation of a new user ID or the changes in permissions could be configured with a policy editor to be a real-time event. In most organizations, however, these types of changes happen constantly in dozens of systems and in reality, unless the data in question is a matter of national security or could involve a system that could be subject to substantial theft, reviews of these types of changes are typically batched to daily or even weekly tasks. In reality, there usually is not a single event that can be captured in real-time to alert security administrators of some policy violation or security breach. Most security incidents are detected by correlating events over time or looking for anomalies. For example, access to a database by an authorized user would not likely be considered worthy of a real-time event alert. However, if security administrators noticed that the user had accessed the data significantly more often over time than other users accessed the data, or there is a history of accessing data at off hours, it might prompt an investigation that a real-time alert would not have caught. Alerting Alerting is a feature that alerts administrators to the existence of events or a report. For example, as security administrator needs to be alerted to the existence of events on a Introduction to Database Auditing 9

10 periodic basis. Often these alerts are also forwarded to an incident management system to document that an incident ticket was opened and closed in a reasonable amount of time. Segregation of Duties Finally, segregation of duties involves removing the personnel responsible for managing the database management system from having any administrative involvement in the processing of audit trails. All of the native database auditing tools require some DBA involvement to insure that auditing is enabled. Some solutions such as the Oracle Audit Vault and the IBM Audit Management Expert (AME) further require DBAs to maintain the databases that store audit records. A major strength of database auditing solutions is that they do not require significant DBA involvement to enable and very little, if any, DBA involvement in the day to day operation of the solution. To summarize the trade-offs of third-party database, the solutions all add significant CAPEX over native database auditing tools and the approaches used by these tools contain significant technical issues that can limit the ability and accuracy of database auditing and raise OPEX to maintain. Third-party solutions do however provide functionality not provided by most native database auditing tools for policy creation, alerting and segregation of duties requirements. Log Management Solutions for Database Auditing A sophisticated log management solution, such as the SenSage Event Data Warehouse, is a viable alternative to third-party database auditing solutions in almost all cases. It adds value by incorporating additional data sources into the analysis, providing increased reporting and retention capabilities, and by not having the weaknesses of the other database auditing choices. SenSage utilizes and enhances the native database auditing capabilities by providing the ability to collect the audit records, efficiently store the records and provide the alerting, reporting and ad hoc query functionality missing from the native approaches. And because many security incidents take place over an extended period of time, SenSage provides years worth of data fully on-line and always queryable. This enables easy but precise investigations and forensic analysis over a massive amount of data. SenSage also satisfies the segregation of duties requirements by moving the database audit records into a secure repository where the audit records are parsed, compressed (10:1 compression ratio) and optionally encrypted in a proprietary data warehouse where the records cannot be modified. SenSage also provides role-based access control over users by controlling what features the user can use, what reports they can view, and even what data within those reports the user can see. When combined with native database auditing capabilities, SenSage provides more functionality than third-party database auditing tools with a significantly lower CAPEX and OPEX. Introduction to Database Auditing 10

11 Meeting Compliance Requirements SenSage provides dashboard reports mapped to specific compliance requirements for regulations such as PCI DSS, HIPAA, Sarbanes-Oxley, FISMA, and many others. Forensic and ad hoc reporting against the captured database audit records can be executed with SQL precision through easy to use investigation reports. And because each organization has its own unique reporting requirements, SenSage provides a query-building wizard that does not require any SQL knowledge. So recalling the example in Figure 2 where a user with the ID of echopskie queried a sensitive table, SenSage provides the ability to easily query all access by this user over several years of data in minutes to determine if there are excessive accesses or policy violations. Ad hoc Queries, Reports and Alerts Ad hoc queries can be stored as reports or alerts. If security administrators wanted an alert every time the user echopskie queried a specific table, the creation of the alert takes less than a minute and can execute on any determined schedule. While the database audit records are not processed in real-time as transactions are executed, the records are processed as quickly as they are loaded into the log data warehouse. These types of queries and alerts can detect slow and low attacks or potential fraud. Mini Case Study: Insider Fraud at Société Générale Managers for France s second largest bank, Société Générale, have described some of the means an authorized futures trader employed to avoid the bank's internal controls and escape detection until January 2008 for fraudulent transactions that cost the bank more than $7 billion and nearly toppled the bank. Its Executive Chairman, Daniel Bouton, describes the pattern like "a mutating virus" in which hundreds of thousands of trades were hidden behind offsetting faked hedge trades. Officials say the rogue trader was careful to close the trades in just two or three days, just before the trades' timed controls would trigger notice from the bank's internal control system, and would then shift those older positions to newly initiated trades. Had auditors at Société Générale had access to a database auditing tool capable of reporting the number of database transactions and the exact details of the transactions, the auditors would likely have noticed the excessive number of transactions compared to other traders and the suspicious nature of the details. The fraud at Société Générale is a classic example of a low and slow type of security violation that cannot be caught in real-time and requires analysis over millions (or even billions) of records spanning years. SenSage provides additional value by combining a fully functional database auditing solution with a log management solution that uses agent-less technology to collect from hundreds of sources including network infrastructure, operating systems and Introduction to Database Auditing 11

12 applications. For example, if DBA user echopskie unloaded a database containing sensitive information, that event may not be considered suspect as that task may be part of his job. However, if a report of his activity from captured log records from operating systems and data leakage tools showed that the unloaded data was moved to his PC and later copied to a USB disk device several days later, the combination of these events might alert security administrators of possible theft. Unlike third-party database auditing tools that can only capture and report on database activity only, SenSage uniquely correlates log data from an unlimited amount of sources to show a complete picture of user activities. Deployed as a clustered software-based solution, SenSage s CAPEX requirements are a fraction of deploying database audit appliances. Additionally, SenSage OPEX is low due to the self-tuning data warehouse, the elimination of DBA support, the inclusion of ETL tools for agent-less collection of database audit logs and an intuitive flexible reporting package. A Hybrid Approach For some organizations, a hybrid approach of using third-party database auditing tools combined with a log management solution that enhances the native database auditing tools is an option. If an organization decides that real-time alerting of database activity is a requirement for some of their databases and the CAPEX/OPEX investment is justified for those databases, they can choose to use native database auditing for less sensitive data. Additionally, a log management system such as SenSage can be used to collect, centralize and correlate data collected by third-party tools to augment forensic and ad hoc query capabilities beyond database activity. In this scenario, a third-party tool is used for real-time event detection only and SenSage complements by providing compliance reporting, historical ad hoc queries and correlation with additional log sources. Summary Organizations should carefully evaluate their requirements for database auditing and consider the technical issues, CAPEX and OPEX of native versus third-party solutions. While third-party database auditing solutions provide significantly more functionality than native tools, their limitations and costs might be prohibitive. While these tools market their advantages based mostly on the performance requirements of native tools, in reality their software agents/probes also add overhead to servers while their appliances add significant CAPEX. A log management solution, such as the SenSage Log Data Warehouse, provides organizations with the functionality required for security and compliance initiatives by enhancing native database auditing capabilities without the costs and technical limitations of third-party database auditing tools. Introduction to Database Auditing 12

13 About the Author Ed Chopskie is the Vice President of Marketing for SenSage. Ed s database management and security experience includes years of DBA and technical support work on DB2, IMS, CICS and RACF at CSX (NYSE:CSX), a $10 billion transportation and logistics company. Additionally, Ed held technical sales roles at BMC Software (NYSE:BMC) where he worked on BMC s Patrol product line for Oracle, SQL Server and Informix. Ed has presented technical papers at database user groups including IDUG and IOUG. Corporate Headquarters: SenSage, Inc. 55 Hawthorne Street, Suite 700 San Francisco, CA (415) Introduction to Database Auditing 13