Log Management, Compliance and Auditing

Transcription

1 Log Management, Compliance and Auditing KR Information Security Solutions 1

2 About KRISS Founded early 2008, by former Indian Naval Officers and Veterans with decades of experience in Information Security, Information Warfare & Network Security. An ISO/IEC 27001:2005 certified and CERT-In empanelled IT Security Auditing Organisation. Highly motivated & experienced management and consulting team with an impressive record of providing security services to major brands across globe including FMCG, Fortune 500, CMM Level 5 companies to name a few. 2

3 Agenda Current Cybercrime Scenario Introduction to Logs Benefits of Logging and Challenges Log Management Architecture Policies, Roles, Operational Process, Security Issues, Log Analysis and Long Term Storage Regulations, Mandates and Controls Real-Time Collection and Consolidation of Logs Log Monitoring, Review, Compliance and Auditing Typical Organisation Logging Scenario and Problems Faced SIEM Benefits and Features Types of Logging Tools 3

4 Current Cybercrime Scenario 4

5 The 21 st Century the age of cybercrime Year 2010 was the year of cybercrime and cyberwars. Year of Wikileaks The New York Times, Guardian, Der Spiegel, El Pais, Le Monde, CNN, BBC and more. 2010, FBI warns Congress that cybercriminals can hack any internet-linked system Gordon M. Snow Assistant director of the FBI s Cyber Division (13th of April, 2011) 5

6 Every technology is vulnerable 6

7 New threats targeted, professional, silent There are Internet shops full of credit card, bank account, privacy, business and other confidential data. Also there are available services to rent a botnet, malicious code and attack anyone. Black Community where cybercriminals are organized better than hi level military organizations Video trainings and elearning available in social media, such as YouTube 7

8 New threats targeted, professional, silent

9 Logs are like fingerprints 9

10 Introduction to Logs 10

11 What is Log? Log is a record of the events occurring within and organisation s systems and networks, used to provide data useful for troubleshooting problems, optimising performance, maintaining security compliance and investigating malicious activities. Examples: Security Software Log Operating System Log Application Log 11

12 Common Types of Logs Operating System Log System events (startup, shutdown, failure, success, error). Audit records (successful and failed authentication attempts, file accesses, security policy changes, account changes, and use of privileges). Application Log Events logged by the applications. Some applications generate their own log files, while others use the logging capabilities of the OS. 12

13 Common Types of Logs Security Software Log Log generated by network-based and host-based security software to detect malicious activity, protect systems and data, and support incident response efforts. 13

14 Standard log formats Syslog SNMP XML CSV Binary Human Readable Text Files There is no consensus in the security community as to the standard terms to be used to describe the composition of log entries and files. Binary files often use proprietary formats that are software-specific (e.g., event logs on Windows systems). 14

15 Preparedness is the Best Defense Unfortunately, that [no log data being available] happens more often than I would like If your home had been robbed, you would have to tell the police officer what was stolen and how the burglar got in. The same is also true for the network. If you simply tell us you have been broken into, and have no evidence to support it, we may be empathetic, but we can t open a case Shelagh Sayers Special Agent, FBI, San Francisco 15

16 Where to start from? Most organizations need a central solution for gathering logs and correlating them for real time intelligent visibility. Appropriate strategic policy changes need to be made for shifting organisation s focus on monitoring business processes instead of network. Organisations need to monitor identities, applications, information and their context instead of just IP addresses, OS s and devices. If you are not already doing this You are vulnerable!!! 16

17 Logs = Activity Tracking 17

18 Logs = Accountability 18

19 Log Data Overview What Logs? From Where? Audit Logs Transaction Logs Intrusion Logs Connection Logs System Performance Records User Activity Logs Firewalls/IPS/IDS Routers/Switches Servers/Desktops Applications Databases Anti-virus VPNs 19

20 Log Management Process Log Management comprises an approach to dealing with large volumes of log messages and covers log collection, centralized aggregation, long-term retention, log analysis as well as log co-relation, searching and reporting. 20

21 Benefits of Logging and Challenges 21

22 Benefits Identification of security incidents and incident response Identification of policy violations and fraudulent activities Threat protection and discovery Forensics, e-discovery and litigation support Regulatory compliance Internal policies and procedure compliance Internal and external audit support IT system and network troubleshooting IT performance management 22

23 Challenges Several potential problems with the initial log generation because of their variety and prevalence. Multiple log sources Inconsistent log content (i.e Protocol name variations (80, HTTP, WWW), Date format variations (MM-DD-YY or MMDDYY)) Inconsistent time-stamps Inconsistent log formats (i.e Human readable, XML, Binary etc.) The Confidentiality, Integrity and Availability of generated logs could be breached inadvertently or intentionally. People responsible for performing analysis are often inadequately prepared and supported. 23

24 Meeting Challenges Prioritize log management appropriately throughout the organization Establish policies and procedures for log management Create and maintain a secure log management infrastructure Provide proper training for all staff with log management responsibilities 24

25 Log Management Architecture 25

26 Log Management Architecture Three Tiers of Log Management Log Generation Hosts that generate log data. Log Analysis and Storage One or more log servers which receive log data from the hosts. Log Monitoring Consoles that may be used to monitor and review log data and the results of automated analysis. 26

27 Log Management Architecture 27

28 Stages and Functions General Log Parsing Event Filtering Event Aggregation Storage Log Rotation Log Archival Log Compression Log Reduction Log Conversion Log Normalization Integrity Checking Analysis Correlation Viewing Reporting Disposal Log Clearing 28

29 Types of Tools Used Syslog based tools SNMP based tools SIEM / SIM / SEM 29

30 Tea Break 30

31 Policies, Roles, Operational Process, Security Issues, Log Analysis and Log Term Storage 31

32 Things to Consider: Policies Things to consider: Log Generation: Hosts, services, type of data and frequency. Log Transmission: How the log data should be transferred, how frequently and measures to protect CIA of log data during transit. Log Storage and Disposal: Log rotation, CIA protection, duration, resource allocation and log disposal. Log Analysis: Frequency, roles, access details, incident identification and response and handling information disclosure through logs. 32

33 Policy Example Source: NIST

34 Things to Consider: Roles System and Network Admin Security Admin Incident Response Team Application Developers CSO CIO Auditors 34