Event Log Management

Transcription

1 Event Log Management A Guide to a Stress-free Audit written by Quest Software, Inc. White Paper

2 Copyright Quest Software, Inc All rights reserved. This guide contains proprietary information, which is protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written permission of Quest Software, Inc. WARRANTY The information contained in this document is subject to change without notice. Quest Software makes no warranty of any kind with respect to this information. QUEST SOFTWARE SPECIFICALLY DISCLAIMS THE IMPLIED WARRANTY OF THE MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. Quest Software shall not be liable for any direct, indirect, incidental, consequential, or other damage alleged in connection with the furnishing or use of this information. TRADEMARKS All trademarks and registered trademarks used in this guide are property of their respective owners. World Headquarters 5 Polaris Way Aliso Viejo, CA info@quest.com U.S. and Canada: Please refer to our Web site for regional and international office information. Updated August 29, 2006 WPW_EventLogManagement_092806_NH

3 CONTENTS INTRODUCTION...1 DEFINING THE TERMS...2 REGULATIONS AND CORPORATE COMPLIANCE...4 HEALTH INSURANCE PORTABILITY AND ACCOUNTABILITY ACT (HIPAA)... 4 GRAMM-LEACH-BLILEY ACT (GLBA)... 5 SARBANES-OXLEY ACT (SOX)... 5 INTERNAL SECURITY POLICIES... 6 SUMMARY... 7 TYPES OF EVENT LOG MANAGEMENT...8 ARCHIVING ) Collect ) Store ) Report/Alert A Case in Point ANALYSIS AND REPORTING A Case in Point REAL-TIME MONITORING Identification Response A Case in Point EVENT LOG MANAGEMENT BEST PRACTICES...15 PLANNING ) Define the Critical Reasons for Implementing an Event Log Management Solution ) Choose which Events to Collect and the Types of Event Log Management Solution to Implement ) Choose which Components of the Environment are Critical for Event Log Management ) Estimate the Volume of Logs SELECTING A SOLUTION Collect Store Report/Alert DEPLOYING A SOLUTION Performance Considerations Audit and Event Log Retention Settings SUMMARY...23 i

4 INTRUST: AUDITING AND POLICY COMPLIANCE FOR THE SECURE ENTERPRISE...24 SECURELY COLLECT EVENT DATA ENSURE LOG INTEGRITY STORE MORE DATA ONLINE REPORT INTELLIGENTLY TRACK USER ACTIVITY ALERT IN REAL-TIME MINIMIZE NETWORK IMPACT INCREASE SECURITY REDUCE COSTS AND ADMINISTRATIVE WORKLOAD ABOUT QUEST SOFTWARE, INC CONTACTING QUEST SOFTWARE CONTACTING QUEST SUPPORT NOTES...28 ii

5 White Paper INTRODUCTION Federal regulations such as the Sarbanes-Oxley Act, the Health Insurance Portability Accountability Act, and the Gramm-Leach-Bliley Act have fueled the need for businesses to know exactly what is happening in their corporate networks. As a result, IT organizations are being required to provide more frequent auditing and reporting on their networks. In fact, auditing IT data has become a standard business practice for most companies today and the emphasis on auditing will only increase in the future. This paper provides an overview of the wealth of information that event logs can provide when preparing for an audit. We will discuss external regulations, internal policies, and best practices that are driving companies to prepare for an audit. Finally, we will discuss the best practices for implementing an event log management solution and provide assistance in choosing a solution to eliminate the stress from your next audit. It should be noted that this paper is written with a focus on external regulations that impact organizations that are based in the United States or that conduct business in the United States. However, many international regulations have similar auditing requirements that also raise the need for an event log management solution. 1

6 Event Log Management: A Guide to a Stress-free Audit DEFINING THE TERMS Let s begin with a discussion of key words and phrases used throughout this paper. Event Log Every operating system, application or network device write records about their activity into one or multiple log files, referred to as event logs. The format and method for creating event logs varies for different operating systems and applications; there is no industry standard. For example, on UNIX systems, the majority of events are written to a text file called a Syslog. Windowsbased systems, on the other hand, maintain multiple binary files for various purposes, such as the security log, the system log, and application logs. Security log Records security-related events such as successful and unsuccessful logon attempts and events related to resource use, such as creating, opening, or deleting files or other objects. System log Contains events logged by the Windows system, such as a service failing to start. Application log Contains events related to application activity, such as launch or shutdown. In most cases, each event log resides locally on its respective host server or workstation. There is no native function that aggregates event logs from around the enterprise to a central location. Event Log Management Simply stated, event log management is the ability to make sense of the multiple separate event logs generated within an organization s infrastructure. A comprehensive event log management strategy includes auditing event logs from servers and, in some cases, workstations, in order to collect, store, and report on event data in support of an audit. Many companies struggle to glean meaningful information from their event logs information that can be used to support auditing efforts. In most cases, the system administrator must sift through the multitude of event log files using native operating system tools, an extremely time-consuming task performed on a reactive, as-needed basis. These native event viewers are insufficient to be used as a true event log management solution because they provide no means to: 2 Collect event data from multiple systems Generate reports in support of an audit However, effective event log management solutions do exist, and this paper will help direct organizations to evaluate and implement a solution that satisfies their needs.

7 White Paper Audit More and more, organizations are subject to audits. As defined by the Merriam- Webster dictionary i, an audit is: 1. a formal examination of an organization s or individual s accounts or financial situation 2. a methodical examination or review In most corporations, an audit conducted in support of a regulation or internal policy usually requires an examination of the overall IT infrastructure, the processes used to conduct business, and the documentation generated as a result of these activities. IT data comprises a material percentage of the data that must be tracked. Audits generally emanate from two sources; either an external regulation or an internal policy. These two drivers will be discussed next. 3

8 Event Log Management: A Guide to a Stress-free Audit REGULATIONS AND CORPORATE COMPLIANCE There are many regulations designed to govern the practices of corporations, protect individual rights to privacy, and spur the adherence to standard best practices. Most IT professionals have heard acronyms such as HIPAA, SOX, and GLBA. These are external regulations that impact not only the IT department, but the organization as a whole. The following provides a general working knowledge of some of the key regulations impacting IT departments in the United States and, to a lesser extent, international organizations doing business in the Unites States. Health Insurance Portability and Accountability Act (HIPAA) The Health Insurance Portability and Accountability Act (HIPAA) ii was signed into law on August 21, Virtually all healthcare organizations are affected by HIPAA requirements, including health insurance companies, health care clearinghouses, and health care providers. The intention of HIPAA is to enforce standards for privacy, security, and electronic interchange of health information. In particular, HIPPA requires healthcare organizations to: Ensure the confidentiality, integrity, and availability of all electronically protected health information organizations create, receive, maintain, or transmit. Regularly review records of information system activity, such as audit logs, access reports, and security incident tracking reports. Establish, document, review, and modify a user s right of access to a workstation, transaction, program, or process. Monitor login attempts and report discrepancies. Identify, respond to, and document security incidents. HIPPA dictates the use of security standards, privacy standards, electronic transaction and code sets and unique employer identifiers when managing and maintaining this critical data. While compliance is federally mandated, compliance also benefits healthcare organizations by providing patients with confidence that their sensitive personal data is safeguarded from inappropriate use. Because of the stiff penalties for non-compliance, healthcare organizations are aggressively working toward demonstrating HIPAA compliance. Meeting these challenges requires the IT department to have systems and processes in place to collect, store, and report on the events occurring within their network, thus creating the required audit trail. 4

9 White Paper Defining audit policies and managing log data have become pressing needs in regulated industries. -- Yankee Group, November 2003 iii Gramm-Leach-Bliley Act (GLBA) The Gramm-Leach-Bliley Act (GLBA) iv was signed into law in November of To comply with GLBA, all organizations in the financial services industry must implement a comprehensive written information security program specifying how their customer information is protected. In particular, these organizations must implement: Dual control procedures, segregation of duties, and employee background checks for employees with responsibilities for or access to customer information. Monitoring systems and procedures to detect actual and attempted attacks on or intrusions into customer information systems. Compliance with GLBA is regulated by federal banking agencies such as the Office of the Comptroller of the Currency and the Federal Deposit Insurance Corporation. Because many organizations have already been audited and found out of compliance, GLBA was expanded with detailed instructions for deploying an information security program. In clarifying the new guidance, the Federal Financial Institutions Examination Council (FFIEC) states: Security is an ongoing process, whereby the condition of a financial institution s controls is just one indicator of its overall security posture. Other indicators include the ability of the institution to continually assess its posture and react appropriately in the face of rapidly changing threats, technologies and business conditions. Institutions must prove their readiness by conducting regular self-audits of their enterprise and documenting the results. Sarbanes-Oxley Act (SOX) The Sarbanes-Oxley Act was passed in July of 2002 as a direct reaction by the U.S. Congress to the accounting scandals of late 2001 and early Its aim is to provide additional oversight to the audit process and eliminate conflicts of interest by creating a standard set of criteria that all publicly held corporations must adhere to in managing their financial data. SOX also has the specific goal of advancing the standards for corporate governance. 5

10 Event Log Management: A Guide to a Stress-free Audit Record retention is central to SOX. In particular, companies and their auditors are required to retain more records than before, including documents and data that form the basis of an audit or that relate to an audit. Fines and jail terms are imposed for the deliberate and willful destruction of audit-related data, and the auditor is responsible for oversight of the enterprise s internal documentation surrounding the audit. In addition, The Retention of Records Relevant to Audits and Reviews v as directed by Section 802 of the Act states that the SEC will dictate rules and regulations concerning the retention of records such as work papers, documents that form the basis of an audit or review, memoranda, correspondence, communications, other documents, and records (including electronic records) which are created, sent, or received in connection with an audit or review and contain conclusions, opinions, analyses, or financial data relating to such an audit or review. Companies with a market capitalization greater than $75 million must comply with these new rules for fiscal years ending on or after November 15, 2004; other companies have received extensions. Under the new law, the retention time for records is generally five years; however, periods of retention vary according to a number of different variables vi. IT executives need to formulate and formalize an enterprise-wide strategy to best manage such data now and into the future, so as to reduce the enterprise's legal exposure, and ensure future data integrity. Many companies recognize that they must make a substantial investment immediately to comply. According to a recent industry analyst survey, 71% of respondents were aware of the SOX budgets within their companies and said they would make a substantial investment in becoming compliant by the end of According to a recent industry analyst survey, 71% of respondents were aware of the SOX budgets within their companies and said they would make a substantial investment in becoming compliant by the end of Internal Security Policies We have discussed several external regulations that are driving corporations to perform audits and prove compliance. There are also internal controls that an organization may put into place, typically through its IT, human resources, legal, security, or compliance departments. Many companies put auditing and security policies in place in order to maintain control over their infrastructure. Some companies capture daily events such as successful logons and logoffs so that they can understand who is on their network at any given time. Another example of a practice instituted with an organization is the requirement to track the activity of privileged users or users who have been granted the rights to set up new user accounts or remove users from the enterprise. The rights granted to these users can easily be abused leaving the organization exposed to an internal security threat. 6

11 White Paper Event log management is one of the most important facets of monitoring an enterprise for situations such as those described above. Like event log management for regulations, the sheer volume of distributed event log data when tracking user activity is enormous and requires a centralized, automated solution. Summary Maintaining compliance with internal policies or assorted industry-specific regulations, whether in the United States or abroad, means, at a minimum, keeping track of all electronic documents (data files, , images, etc.) that are covered by those regulations, as well as tracking access to those documents. Upon request, organizations need to prove, through reporting, that they have established appropriate control of access to resources. With the right event log management solution, organizations can collect, store, and report on events related to this class of activity (such as account creation, group membership changes, and permission changes) and also notify the responsible personnel of events that might indicate an intrusion. 7

12 Event Log Management: A Guide to a Stress-free Audit TYPES OF EVENT LOG MANAGEMENT Event log management solutions can be categorized according to the tasks they perform: Archiving Analysis and Reporting Real-time Monitoring Each task requires the ability to manage a different percentage of all generated events. The diagram below represents the categories of event log management solutions and the corresponding percentage of events that must be collected to satisfy the requirements of the category. Diagram 1. Types of event log management solutions and percentage of events required to be tracked to address them. 8

13 White Paper Archiving Archiving means the long-term retention of all generated events in their original format. The two main drivers for implementing event log archiving are: A legislative mandate or regulation, such as SOX or GLBA, which requires organizations to store data for some period of time. The period of time can range from 30 days to seven years or more. Forensic investigation (i.e., providing evidence for the investigation of a security incident or latent audit). Archiving is often confused with simply storing events in a database; however, because Archiving requires the storage of vast amounts of information, a file repository rather than a database is appropriate. There are three steps required to successfully execute an Archiving strategy: the ability to collect, to store, and to report on events that have been collected and stored. 1) Collect The primary requirements for the collection step are automation and scalability. Organizations can generate vast amounts of event data that must be collected regularly from throughout the enterprise without impacting users or business processes. 2) Store There are two important issues to consider when planning the storage step of the process: Space Organizations need to plan for and ensure they have adequate storage resources available to hold the event data that will be collected. Integrity When data is to be used for forensic investigation and for auditing, organizations need to ensure that the data, once collected, has remained intact and was not altered. 9

14 Event Log Management: A Guide to a Stress-free Audit 3) Report/Alert The last step requires retrieving event log entries from where they have been stored for the purpose of providing reports as evidence or for forensic analysis. For this step, retrieval granularity is the key requirement. For example, without a robust event log management solution, the task of compiling and reporting on the activity of a fired employee during the last half-year from multiple backups that were made once a week might be quite difficult to complete in a timely manner. Conversely, with an application tuned for this type of request, it can be fulfilled quite simply. In summary, the features that should be included in an event log management solution intended primarily for Archiving are as follows: Scalable architecture for collecting large amounts of data Storage types optimized for space consumption (that is, for this purpose a highly-tuned file repository system is more effective than a standard relational database) Ability to store event logs in their original format Rewrite-disabled storage medium support (e.g., CD-R or DVD-R) for ensuring inalterability of logs Ability to search through archived logs and report on the necessary portions of events Ability to import events from the archive into the reporting database for analysis A Case in Point Some enterprises establish a workforce clearance policy for employees leaving the company. The policy requires generating reports on and examining all of the employee s network activity for security violations or other suspicious actions. With the help of an event log management solution, events related to the employee s activities can easily be tracked. Analysis and Reporting The main idea behind event Analysis and Reporting is to regularly provide the security team with comprehensive summaries and detailed views of network activity. As outlined earlier, this information is needed for several reasons: Proving regulation compliance for auditors Checking to ensure users conform to internal security policies Tracking administrative activity 10

15 White Paper As with Archiving, a successful event Analysis and Reporting strategy must offer the abilities to collect, store, and report. Here the primary requirement for the collection step is the ability to collect only what is necessary for analysis. The storage should be organized so as to keep stored volumes relatively small in order to maintain the speed of data analysis. And finally, the key functions of the reporting step are comprehensive and problem-oriented representation of data and report distribution to the persons concerned. Thus, a software solution for event Analysis and Reporting must include the following features: Filtering of collected events in order to collect and store only those events that are needed for reporting. Storage optimized for data analysis. For this purpose, a database is the proper solution. Reporting using pre-defined, problem-oriented reports as well as custom reports tailored to each organization s specific needs. Publishing of reports to the appropriate individuals. According to our research and testing, the percentage of events delivered to the end user usually varies from 20% to 40% of all generated events, depending on the range of reported problems. A Case in Point In order to function efficiently, organizations regularly give certain rights to many administrators. However, only a few highly-privileged administrators are ultimately responsible for the entire enterprise security. In order to sleep well at night, these few must have a way to audit what the delegated administrators are doing on the network. The example below demonstrates how an administrative function such as account management can be tracked over a period of time. 11

16 Event Log Management: A Guide to a Stress-free Audit The Group Membership Changes Report shows that the account of User3 was first removed from the Users group by John Smith. This may have been the result of an employee termination. Then, nearly two weeks later, User3 was added to the group of Domain1 administrators, again by John Smith (the domain administrator). Moreover, the additional operation was carried out during nonworking hours. This combination of facts is quite suspicious and requires further investigation on the activities of both John Smith and User3, as John Smith may be using the User3 account as a back door. Real-Time Monitoring The primary purpose of real-time event monitoring is to be able to respond quickly to security incidents that are critical to either the continuity of business operations or the organization s confidential information. This type of protection can be achieved through implementation of an event log management solution supporting two key functions: Identifying critical events Responding to those events 12

17 White Paper Identification In order to identify events, the solution must continually inspect all event logs and to focus on only those events that represent evidence of a business-threatening security incident. However, in most cases, a security incident is not a single event, but a combination of events that occur in some period of time possibly even across multiple systems. Designation of relationships between these multiple events is referred to as event correlation. Response The second function is to provide for either an automated response to a critical event or notification to those who can resolve the issue manually, or both. (A note to the wise: automatic response actions should be used with caution. For example, automatically blocking or resetting a connection to a resource can lead to unnecessary business disruption in the case of a false alarm.) According to our research and testing, the number of events that are tracked as part of a Real-time Monitoring solution is quite small normally in the 1% 5% range of all events. From a security perspective, this type of event log management solution usually looks very attractive, but effective implementation and maintenance are challenging. The main challenges are: 1. The large number of false positives. Although the event correlation algorithms are being polished, some systems still produce many false alerts. An example is multiple invalid logon attempts by service accounts with expired passwords. A real attack can easily hide among these false alerts. 2. Staff must be assigned to quickly manage the real-time alerts. 3. Complex deployment. Agents that review logs in real time for predetermined events must be configured, deployed, and continually monitored on a multitude of servers. To reduce the number of false positives, organizations can take various steps, including fine-tuning the event correlation rules as well as limiting the number of rules and the number of systems monitored. 13

18 Event Log Management: A Guide to a Stress-free Audit A Case in Point The following are examples of business critical security incidents that might occur in a typical organization: Event log cleared by unauthorized user or process. Server rebooted by unauthorized user or process. Audit Policy changed or turned off by unauthorized personnel. In the case where a company has instituted Real-time Monitoring, an Audit Policy change by an unauthorized person might result in the following actions being executed immediately: The policy change is rolled back. The account of the policy change initiator is disabled. An notification with a report showing the incident and the results of the response action is sent to responsible persons. 14

19 White Paper EVENT LOG MANAGEMENT BEST PRACTICES Now that we seen that there are three types of event log management, let s turn our attention to best practice implementation of an event log management solution. Organizations should consider the following three key steps in implementing an event log management solution: 1. Planning Determining the requirements for an event log management solution 2. Selecting Reviewing vendor and product alternatives that best meet the requirements 3. Deploying Issues to consider when rolling out the event log management solution Planning To ensure successful implementation of an event log management solution, organizations must plan carefully and pay special attention to both technical and business needs. 1) Define the Critical Reasons for Implementing an Event Log Management Solution. When making this determination, pay special attention to the business and best practice reasons discussed in the Regulations and Corporate Compliance chapter of this paper. Understanding auditing requirements early in the process will help conduct a stress-free audit when the time comes. 2) Choose which Events to Collect and the Types of Event Log Management Solution to Implement. Based on the list of reasons defined in the previous step, identify which events need to be archived and which need to be reviewed and responded to. For example, in order to be HIPAA-compliant, organizations need to archive logon attempts. For this kind of task many enterprises need to collect and store events from their servers security logs. Once these events that must be tracked have been identified, the next step is to choose the types of event log management that is required: Caching, Archiving, Analysis and Reporting, and/or Real-time Monitoring. Caching enables organizations to provide reasonable assurance that zero logs have been lost and all are tamper free. Archiving allows organizations to prove compliance with legislative regulations and prepare for forensic investigations. 15

20 Event Log Management: A Guide to a Stress-free Audit Analysis and Reporting allows organizations to track user activity. Real-time monitoring deals with events critical to business continuity. 3) Choose which Components of the Environment are Critical for Event Log Management. Based on the list of reasons for implementing an event log management solution, organizations can single out those computers involved in the corresponding processes. For example, if a reason for implementing an event log management solution is a legislative regulation imposing a requirement for tracking user access to protected data, organizations will need to monitor only those computers that contain the protected data. In other instances, organizations may need to collect event log information from all servers or even from workstations. In other words, perform a technical assessment of the parts of the environment that need monitored. Knowing this will help estimate the required scalability of the solution. For example, the regulation might require the collection of the following information about computers on the network: Computer roles: domain controllers, member servers, workstations Platforms and operating system versions installed Resources such as file shares, directory objects, and printers, for which access will be reviewed Quest Reporter can automate this step by collecting the necessary configuration information from the selected computers and presenting it in a comprehensive report view. 4) Estimate the Volume of Logs. Estimating the space required for storing events is very important for the Reporting and Archiving type of event log management solution, as quite a large amount of data needs to be stored. Estimation can be a time consuming and laborious task best served with an automated approach. Determining the number of computers and how often event logs are overwritten on them also helps determine the performance requirements of the event log management solution. 16

21 White Paper Selecting a Solution Now that the most critical types of event log management have been defined, it is necessary to choose the right solution to implement. We recommend a technical evaluation, preferably in a lab environment that closely mimics the production environment. The evaluation process includes determining specific technical criteria and prioritizing them. The most important evaluation criteria relate to the main functions a solution should perform that is, the ability to: collect, store, report, and notify. The importance of each criterion depends on the type of event log management required. Our recommendation is to evaluate each solution against the criteria presented below and choose the one with the highest scores in the most important functions depending on the required types of event log management. ARCHIVING ANALYSIS AND REPORTING REAL-TIME MONITORING Collect Collection of all event data must be performed while maintaining efficient use of network bandwidth. Collection must filter data before sending to maximize efficiency. Collection must provide filtering on key events and correlation of events. Store Repository storage is required because of the massive volume of event data to be collected. The storage must be optimized for speed of analysis. The storage must be optimized for speed of insertion into the database. Report/Alert Frequent reporting is not required except to meet specific auditing requirements. Alerting is not critical. Must provide comprehensive and problem-oriented representation of data as well as automated report distribution. Must provide facilities to allow for operatorinitiated response. Reporting is not critical. Fast notification is critical as is both automatic and operator-initiated responses. Evaluate and purchase a centralized event log management system for your enterprise because this is one of the best security practices recommended by Microsoft. --Microsoft Technet Web site, November 2003 vii Regardless of the type of event log management that is required, it is important that each function (collect, store, report, and notify) be evaluated. We recommend that organizations consider the following for each function: 17

22 Event Log Management: A Guide to a Stress-free Audit Collect Supported event sources A solution must be able to collect from a wide breadth of event logs. The ability to collect from all Microsoft Windows event logs is a must. Some large enterprises may also want to collect events from UNIX Syslogs. And there may also be requirements to collect event data from applications, firewalls, and other network devices. Scalability Scalability is defined as the ability of the product to maintain its efficacy when the environment grows in size or volume. There are several features to investigate when determining a solution s scalability Agent support The product s ability to support agents deployed on the devices from which event data will be collected. Traffic compression The ability to compress traffic between the agents and servers. Filters on data sources The ability to specify what data to collect at a granular level. Security In many instances, the security of the solution and the data it processes is also a requirement. Consider whether the solution provides the following security features: Traffic encryption Encryption of traffic between agents and servers. Agent/server authentication Authentication between agents and servers. Optional Agent-less collection The ability of the product to optionally collect data without agents (remotely). In some environments, the use of agents can be prohibited on important servers due to security issues. 18 Performance Two important criteria when evaluating performance are the number of events collected by the agent per second and the maximum number of agents supported per server. Data collection management The resources required to deploy and manage the entire data collection process. Below are criteria by which to evaluate this effort: Configuration flexibility Ease of specifying sets of computers, event filters, and schedule settings. Agent deployment In order to accelerate the deployment process, a product should be able to install agents remotely. Note: a manual installation process may also be necessary when remote access to the target computer is blocked (e.g. by a firewall). Also consider other ways of automating the deployment process, such as through Group Policy. Agent management and troubleshooting The product should provide for centralized low-cost agent management, including such important features as remote activation and deactivation of agents, automatic delivery of upgrades to agents, deployment of custom utilities on monitored computers for use by agents, and monitoring of whether the agents are functioning.

23 White Paper Store Data storage type A complete product must have two data storage types: a file repository structure for Archiving and a database system support for Analysis and Reporting. Backup technology The ability to easily back up collected event data is mandatory since some regulations require long term storage of event data. Granular restore The ability to restore the selected granular portions of event data. Consolidation technology For wide-spread networks or networks with slow links, the product must have the ability to consolidate event data from multiple data storages automatically and on a scheduled basis. Retention management The ability to delete unnecessary granular portions of event data from the data storage. Report/Alert Predefined expert knowledge The product should either contain reports to meet auditing requirements or should allow the reports that come with the product to be edited to meet the requirements. Data analysis and representation features The product should offer well-formatted reports, advanced filtering, drilldown features, charts, and OLAP. Consider that some reports will need to be text-based while auditors may want more visually appealing reports. Report distribution system The product should also permit exporting reports to commonly used formats and distribute them as needed, such as through or by publication to a web portal. Automation of the report creation and distribution is also important. Delivery of alerts to responsible persons should be fast and reliable and include a number of notification methods. In addition to notifications, a series of linked response actions should be available. Notification methods The product should be able send alerts via , pager, and net send as well as SNMP for organizations that have deployed broad monitoring solutions such as HP OpenView. Response actions The product should allow the flexibility to link in other processes in the event certain criteria are met. For example, if a policy change occurs, the application should be able to launch another application or a series of applications that is intended to address the offending event or events. 19

24 Event Log Management: A Guide to a Stress-free Audit Deploying a Solution Once the appropriate event log management solution has been selected, the final step is deployment. There are two key issues that should be considered before starting the deployment process: performance and audit and event log retention settings. Performance Considerations Armed with the comprehensive technical view of the environment, the number of event log management servers and their locations on the network must be determined in order to effectively distribute the load. Take into consideration the following issues that may affect the performance of the solution: Collector Servers For servers dedicated to collecting event log data (also known as collector servers): Performance rate This is defined as the number of events processed per second. This will help estimate how many monitored computers one collector server can handle. As a rule, such figures are provided by the event log management vendor in the software documentation. However, for best results, always lab-test the solutions with an environment as close to production as possible. Traffic load Ensure that the link between the collector server and the monitored computers is sufficient. Otherwise, network bottlenecks may impede data collection. If permitted in the organization, we recommend agent-based solutions with compression and the ability to schedule collection during non-business hours. The results of the calculations will result in a decision to deploy one collector server per N computers and run the collections every X hours (or minutes), where N and X depend on the organization s unique characteristics and needs, including the following: Growth rates of the logs Collection performance rates Periodicity of reporting Impact on traffic 20

25 White Paper Storage Servers For servers dedicated to storing event data (also known as storage servers): First, a distinction should be made between storage servers designed for Archiving purposes and those for Analysis and Reporting purposes. Archiving storage servers should be optimized for space consumption, whereas Reporting and Analysis storage servers should be optimized for fast analysis of data. For Archiving, consider using a long-term storage system rather than a database system. A database system requires several times more space and therefore has a higher total cost of ownership compared to specialized file-based repositories or native file formats (.EVT, for example) simply because of the cost associated with purchasing and maintaining more disk. For Analysis and Reporting, a database is preferred, since the ability to analyze and report is paramount. However, consider keeping the database reasonably small to provide for fast report compilation. This can be achieved through: Keeping data for a defined short period of time (two-four weeks). Having separate reporting databases for different parts of the environment. To minimize traffic and performance issues, the technique called storage consolidation should be used. Using this technique, which is illustrated in the following diagram, each collector server has its own local storage, where the frequently-collected logs reside. Collections here may occur every 1 2 hours to prevent the logs from being overwritten. Then, periodically (every night or even less often), these local storages are consolidated into a single global archive, satisfying the need to Archive data or for ongoing Analysis and Reporting. 21

26 Event Log Management: A Guide to a Stress-free Audit "Local" storage Event consolidation every night or even less often "Local" storage Event collection every 1 2 hours Global archive "Local" storage "Local" storage Diagram 2: Storage consolidation allows widely distributed networks to collect, on a regular basis, data locally and consolidate that data into a central database on a less regular basis. Audit and Event Log Retention Settings Another important step of deployment is to verify that the audit settings of the selected part of the environment are appropriate. In other words, confirm that the systems will register only those events that are needed to satisfy the requirements outlined earlier. This will help limit the amount of storage that is required by the solution. Also, in order to be sure that no events are missed, a balance must be struck between maximum log size and the number of days events are kept in the log. Collections must occur with sufficient frequency so as to ensure that no events are overwritten prior to the next collection. 22

27 White Paper SUMMARY Whether trying to meet the needs of new federal regulations such as the Sarbanes- Oxley Act, the Health Insurance Portability Accountability Act, or the Gramm-Leach- Bliley Act or internal security policies, effectively managing event logs can take the stress out of the equation. Event log management is more and more become a standard operating procedure for many it organizations. The key to successful deploying a solution resides in: Defining requirements Determine the business need and how that translated into the correct event log management type: Archiving, Analysis and Reporting, or Real-time Monitoring, or even a combination of the three. Selecting a vendor Be sure the selected solution has sufficient functionality to collect, store, report, and notify in line with business requirements. Deploying the solution efficiently Take into account performance factors for the solution and the environment. The steps outlined within this event log management evaluation framework will assuredly remove the stress from any organization s next audit. 23

28 Event Log Management: A Guide to a Stress-free Audit INTRUST: AUDITING AND POLICY COMPLIANCE FOR THE SECURE ENTERPRISE Collect InTrust Server Store SQL Server Real-Time Alerts Report/Alert Alert DB Audit DB Net Send SNMP Traps InTrust Alerting Console or Third-Party Monitoring Consoles Report Archive Notify Agent-side Caching Failover Server Repository Reports SharePoint Portal Server Reporting Portal If your organization is grappling with selecting an event log management solution, whether to support an internal security policies or an external regulation, InTrust is the answer. InTrust helps systems administrators securely collect, store, and report on event data from across the network to meet their needs. InTrust helps to: Securely Collect Event Data With its SecureCollect technology, InTrust can securely collect enterprise-wide event data. The ability to schedule the collection reduces administrator workload and allows data collection to occur during the evening, lowering daytime network utilization. Ensure Log Integrity Through agent-side caching, InTrust ensures zero log loss and tamper free audit data. The agent writes all information in its native state to a separate, secure location on the local disk. This information is compressed in order to save storage on the local server. 24

29 White Paper Store More Data Online InTrust features a unique, two-tiered storage architecture: StoreMore. The first tier of StoreMore is the repository, which offers unparalleled compression over storing the same amount of event data in a database. The second tier of StoreMore is the database, which allows for advanced Analysis and Reporting. This unparalleled storage architecture gives administrators better historical insight into their networks than was previously possible. Report Intelligently InTrust s FlexReport technology allows administrators to create and distribute the information everyone needs to support their auditing needs. With both predefined and custom reports, administrators can be sure to provide the exact information that is required. Track User Activity Through its UserTrack technology, InTrust collects information on unusual user and administrator activity, such as attempts to access files during off hours, multiple failed log on attempts followed by a successful log on, and other business-critical security events. It then works to correlate the information and automatically alert you to an unusual activity. Alert in Real-Time InTrust s NotifyNow technology ensures that you will receive real-time notifications of UserTrack alerts. InTrust can send alert notifications directly to you via or it can send notification to third-party monitoring applications such as Microsoft Operations Manager (MOM). With this capability, InTrust give you more time to respond to business-critical security and system events. Minimize Network Impact Using optional agents, InTrust minimizes network impact by gathering only new data and by compressing and encrypting data before it travels across the network. 25

30 Event Log Management: A Guide to a Stress-free Audit Increase Security By analyzing InTrust reports, administrators can identify probing attacks, new attack methods, and sophisticated attacks designed to circumvent Real-time Monitoring. Reduce Costs and Administrative Workload InTrust eliminates the need to collect and review data manually. Administrators can automate the scheduling of data consolidation, analysis, and report generation, resulting in lower costs, reduced workload, and accurate information on demand. The deployment and configuration of InTrust is easier than ever with a choice of two walk-through wizards enabling the administrator to easily deploy the product which saves resources and time. 26

31 White Paper ABOUT QUEST SOFTWARE, INC. Quest Software, Inc. delivers innovative products that help organizations get more performance and productivity from their applications, databases and Windows infrastructure. Through a deep expertise in IT operations and a continued focus on what works best, Quest helps more than 18,000 customers worldwide meet higher expectations for enterprise IT. Quest Software can be found in offices around the globe and at Contacting Quest Software Phone: Mail: Web site (United States and Canada) info@quest.com Quest Software, Inc. World Headquarters 5 Polaris Way Aliso Viejo, CA USA Please refer to our Web site for regional and international office information. Contacting Quest Support Quest Support is available to customers who have a trial version of a Quest product or who have purchased a commercial version and have a valid maintenance contract. Quest Support provides around the clock coverage with SupportLink, our web self-service. Visit SupportLink at From SupportLink, you can do the following: Quickly find thousands of solutions (Knowledgebase articles/documents). Download patches and upgrades. Seek help from a Support engineer. Log and update your case, and check its status. View the Global Support Guide for a detailed explanation of support programs, online services, contact information, and policy and procedures. The guide is available at: Support Guide.pdf 27

32 Event Log Management: A Guide to a Stress-free Audit NOTES i ii Rada, Roy (2001) IT Reference: Health Information Transactions, Privacy, and Security. Hypermedia Solutions Limited iii Yankee Group the most trusted name for Communications and Networking Research and Consulting, November 2003 iv Federal Banking Agencies v vi vii Best Practices for Enterprise Security; Monitoring and Auditing for End Systems 28