SOLUTION BRIEF. Granular Data Retention Policies

Transcription

1 SOLUTION BRIEF Granular Data Retention Policies

2 Table of Contents 3 State of Affairs 3 The Problem 4 The Solution 4 How it Works 6 Summary

3 Granular Data Retention Policies State of Affairs The number of applications and devices on corporate networks is growing at a rapid rate. Each of these sources generate a stream of log data (also known as machine data) which is a hot commodity for different business functions from security to compliance to IT operations to analytics. The applications that use machine data typically pull the information directly from the log sources with few or no limits on the log data they copy and store. Each of the uses has different requirements for which log data should be retained and for how long. Because there is no centralized management of the machine data generated by these log sources, basic questions like which systems get what machine data are difficult, if not impossible to answer. For example, credit card logs from a database may be collected by many systems, creating a PCI compliance nightmare. The uncontrolled distribution of log data also means that there are few or no guarantees that at-risk data is kept only for the minimal amount of time possible. The systems that use machine data (such as compliance applications) often are not geared toward long-term archiving, which creates the opposite problem: long-term data retention requirements may not be met. The Problem The amount of machine data being generated imposes significant storage costs. The volume also creates challenges in meeting user expectations when searching (often to address critical security issues) and performing other operations. The tools in use may not be designed for long-term retention and may modify data in order to meet performance goals or to cut storage costs, which violate the requirement to have log data in its original format for compliance and forensics. Management is at an increased risk of non-compliance and possible resulting financial penalties, legal repercussions, and loss of business. All the while, administrators struggle with how to archive the data, accommodate growing storage requirements with finite storage resources and budgets, and ensure access to this critical information. 3

4 The Solution What is needed is a solution that can address log storage needs more economically and judiciously. The solution should provide storage of critical data for the mandated time periods and of non-essential data for shorter periods, freeing up valuable disk space for additional critical data. TIBCO LogLogic Log Management Intelligence addresses this need with granular data retention. Granular data retention supplies policies specifically constructed to address these problems. The flexibility of data retention policies assist administrators in leveraging storage and keeping essential data only for the mandated time periods while assigning less critical log sources shorter retention periods. This intelligent storage capability allows more economic use of a costly and finite resource without compromising availability and access to the data. Login Attempts Financial Data Access Router Configuration Change Print Spoolers High-Speed Index Long-Term Archive 180 days 365 days 90 days 30 days 3 years 7 years 3 years 1 year Table: Business Requirements For Data Retention Policies How it Works TIBCO LogLogic comes with a pre-defined set of data retention policies for different types of log data. These data retention policies provide the ability to define both how long to keep archives of raw data and the period of time data should be indexed to improve search performance. One of the many standard data retention policies available in LogLogic is calls for archiving raw data for one year and keeping one month s worth of data indexed. Short data retention policies like this would be assigned to less critical log sources. Long-term data retention policies might be applied to critical log sources like a general ledger system. Data can be archived with TIBCO LogLogic for up to 10 years and is available for searches, reports, and alerts for the full retention period. Create custom policies For even more fine-grained control, administrators can create custom data retention policies and apply them to specific subsets of data. These granular data retention policies enable businesses to limit long-term and operational storage to only the data that is needed, making more economical and strategic use of costly disk and storage space. For example, a nuclear facility might want certain log data that is known to be useful in tracing malware like Stuxnet to be archived for the full 10 year period, but indexed for only a short period of time. This flexibility provides the optimal mix of assurance for risk management without burdening search and reporting systems with unnecessary volumes of data. Industry specific needs driven by regulatory requirements or enterprise-specific policies and operational guidelines are great examples of why granular data retention policies are so important. 4

5 Create custom groups To facilitate the assignment of data retention policies, log data can be grouped. Administrators can organize subsets of log sources for even easier assignment to specific data retention policies, ensuring intelligent and efficient disk utilization. TIBCO LogLogic data retention policies allow the most fine-grained control of log data. Log sources can be easily rearranged into groups specific to organizational or retention requirements. New groups can be created using a few mouse clicks, allowing devices to be added via pattern matching, dynamic rules, and other methods. Set data retention priorities The facility for grouping log data is quite rich and flexible, and sometimes data belongs to more than one group. For example, an administrator might define a data retention policy for CISCO PIX routers (30 days indexed/1 year archived) and a separate policy for Severe Errors (90 days indexed/3 years archived). These are not mutually exclusive groupings. When this situation occurs, TIBCO LogLogic uses the highest priority data retention policy. If the priority of the Severe Errors data retention policy is higher than the CISCO PIX policy, severe errors on CISCO PIX routers will be retained according to the 90-day/3-year policy, and all other data will be retained according to the 30-day/1- year policy. Step 1: Create and organize sets of log data Firewall Financial Systems PCI Severe Errors Step 2: Review data retention policies and create custom policies Standard: 90/1 RAW: 1-year Custom: Financial Systems RAW: 10-years Step 3: Assign data retention policies to sets of log data Standard: 90/1 RAW: 1-year Applies to Firewall Custom: Financial Systems RAW: 10-years Applies to Financial Systems 5

6 Summary Granular data retention policies give administrators the ability to create individual polices for specific data types as required by the log source, operational guidelines, or regulatory requirements. Administrators can assign critical log sources to pre-defined or custom data retention policies quickly and easily, ensuring that log data will be retained for the long term. Granular data retention policies also allow administrators to fine-tune the amount of data kept online to optimize searching, reporting, and alerting. Benefits include the ability to: Respond faster Attain significant savings on storage costs Manage critical data correctly TIBCO Software Inc. (NASDAQ: TIBX) is a global leader in infrastructure and business intelligence software. Whether it s optimizing inventory, cross-selling products, or averting crisis before it happens, TIBCO uniquely delivers the Two-Second Advantage the ability to capture the right information at the right time and act on it preemptively for a competitive advantage. With a broad mix of innovative products and services, customers around the world trust TIBCO as their strategic technology partner. Learn more about TIBCO at Global Headquarters 3307 Hillview Avenue Palo Alto, CA Tel: Fax: , TIBCO Software Inc. All rights reserved. TIBCO, the TIBCO logo, TIBCO Software, TIBCO LogLogic, and TIBCO LogLogic Log Management Intelligence are trademarks or registered trademarks of TIBCO Software Inc. or its subsidiaries in the United States and/or other countries. All other product and company names and marks in this document are the property of their respective owners and mentioned for identification purposes only. 6 exported07apr2014