QRadar SIEM 7.2 Windows Event Collection Overview

Transcription

1 QRadar Open Mic Webcast #3 August 26, 2014 QRadar SIEM 7.2 Windows Event Collection Overview Panelists Aaron Breen QRadar World-wide Support Leader Adam Frank Principal Solutions Architect Jonathan Pechta Support Technical Writer Jeff Rusk Team Lead, QRadar Integration Services and Maintenance Colin Hay Team Lead, QRadar Integration Services and Maintenance Andrew Merrithew QRadar Integration Team Developer Luke Dewitt QRadar Support Technical Lead Mark Wright QRadar L2 Support Manager Reminder: You must dial-in to the phone conference to listen to the panelists. The web cast does not include audio. USA: Canada: Participant passcode: Slides and additional dial in numbers: IBM Corporation

2 Goal: Provide insight on the QRadar methods for collecting Windows-based events Microsoft Windows Security Event Log (WMI) Event collection Event Collector (EC) (16xx) Configuration polling port 8413 Console Event collection Syslog events port 514 ETHx WinCollect agent Event Collector (EC) (16xx) Event collection Syslog events port 514 ETHx Event Collector (EC) (16xx) Adaptive Log Exporter 2 Note: QRadar also supports Snare, Balabit IT Security, and other third-party software options.

3 Key capabilities of WinCollect Central management from the Console and high performance. Automatic log source creation at install. Event storage to ensure no events are dropped. Capable of collecting Forwarded events from Microsoft Subscriptions. Capable of filtering events using XPath queries or exclusion filters. Supports more remote Windows sources than the Adaptive Log Exporter. Officially supports virtual machine installs. Console can send software updates to remote WinCollect agents without having to reinstall agents in your network. Capable of forwarding events on a set schedule (Store and Forward). 3

4 Key capabilities of the Adaptive Log Exporter Benefits Easy to install and configure individual agents and basic firewall restrictions. Supports plug-ins. Supports automatic log source creation at install. Supports high EPS systems through tuning. Can collect local events and remote poll for events from other Windows systems. Drawbacks The Adaptive Log Exporter is the predecessor to WinCollect and will eventually be phased out. A single Adaptive Log Exporter can only poll up to 20 other hosts for their events. Changes must be completed on the remote Windows host. Does not support bulk adding of log sources. 4

5 Key capabilities of the Microsoft Windows Security Event Log (WMI) Benefits Agentless collection of Windows events. Supports encryption and authentication. Does not require any additional maintenance or software updates. Can be managed from the Console and supports bulk log source creation. Drawbacks Supports low event rates (the event rate should not exceed 50 EPS). Not suitable for most domain controllers. Requires a low latency connection WMI can be process/bandwidth expensive as it authenticates every connection. Configuration can be difficult on some operating system versions. 5

6 Best practices for WinCollect deployments Map and plan the number of hosts you need to collect events for in your network and identify unique system requirements. Identify systems that generate high event rates (DCs, auditing) Understand the EPS rates for peak and off-peak hours. Systems in remote networks or on slow connections. Install a dedicated WinCollect agent on Domain Controllers and other high event rate assets. No more than 500 endpoints should be polled by a single agent. Add 100 log sources at a time to see how the system reacts and to test event rates. The number of agents required is directly proportional to the number of events generated and the number of endpoints you need to monitor. When bulk adding log sources to your WinCollect agent, it helps to bulk add servers of similar performance (EPS). 6

7 No single tool fits all of the collection capabilities for Windows There are many options to consider when planning to collect Windows events. Issues to consider: Event rates for specific systems (Domain Controllers vs endpoints) Number of systems that require event coverage Agent or agentless event collection Software environment (Legacy operating systems) Corporate security policy (restrictive GPO, sensitive assets, auditing) Network (NAT, Firewalls, Congested networks, WAN/remote sites) Do you need centralized management? Cost (VMs vs physical hardware, system requirements, third-party options) 7

8 System requirements Why are the system requirements so high for the agent installations? WinCollect and the Adaptive Log Exporter have high requirements because we are unsure of the environment where the agent will be install. The specifications are set in order to ensure performance regardless of the number of events that need to be processed. 8

9 What Custom is the Event Magistrate Properties component? & Security Content Packs Creating custom event properties allows you to create regular expressions to parse important data from a payload. By default, QRadar includes a number of default custom event properties. How do custom event properties help me? When you create a custom event property, it allows specific portions of the event payload to be normalizing from the log source. This allows QRadar to parse custom fields from an event payload. The end result is that this data is more visible and can be leveraged for searches and reports. What are Security Content Packs? Recently, the QRadar integration team released a new Security Content Pack for QRadar for Windows Events. The Security Content Pack includes 61 new custom event properties for Windows-based events. Are Security Content Packs part of QRadar s automatic update? No, Security Content Packs must be downloaded as an RPM and installed on the Console. 9

10 What WinCollect is the Event Magistrate Filtering: component? XPath vs Exclusion Filters What is the difference? The difference is what data is returned and where the filtering takes place. XPath only returns the data in the query. This can be beneficial as it keeps events off the wire and reduces bandwidth. XPath Exclusion filters return the entire event log and process the events. Any EventIDs or Source that matches an event is not sent to the QRadar appliance. Exclusion filter 10

11 Support tools Two new support tools shipped with WinCollect agent version These exe files are located in Program Files\IBM\WinCollect\bin. WinCollect EventLog EPS Monitor This tool prints out the current EPS rate to the screen as each minute passes. WinCollect Ping This tool verifies the existence of a PEM certificate file and attempts to contact the Configuration Server as specified in the agent configuration. 11

12 Advanced questions: part 1 The first questions addressed by the panelists will be these that were asked in advance in the QRadar Customer forum. Q1 - sxs: How do we collect events when the network environment includes a password management appliance that generates a password at runtime? Q2 - Mordecai: Is it possible to differentiate the hardware requirements for local collection and remote collection with a Wincollect agent? Q3 - brhutchi: What solution should I use for 50+ Domain Controllers? Is ALE better than WinCollect? How do we determine which to use? Q4 - Kyle: What is the best solution for bulk disabling automatic updates when the WinCollect deployment contains thousands of agents? Q5 - Kyle: What is the procedure for deleting a group of existing WinCollect agents and then adding them back with a batch file deployment? 12

13 Advanced questions: part 2 Q6 - Eric: How are XPath queries processed? For example, I want to suppress some data from security, application, and other logs, do i need to define multiple xpath query within the query list? Q7 - Eric: Can I combine XPath queries with the Standard Log Types (Application, System, Security) or Event Types (Information, Warning, Error)? Q8 - Eric: WinCollect seems to truncate UDP output, while TCP payloads are complete. Can I increase the agent to send larger packets? Q9 - brutchi: Can WinCollect agents be configured to reduce noisy events? For example, systems or service accounts where the username is $. Q10 - Wallace: What does Enable Active Directory Lookups and when do I leverage this feature? Q11: I want to have managed WinCollect agents, but I cannot use a standard port, such as Can I change the port number? 13

14 Advanced questions: part 3 Q12: When collecting log from Active Directory Domain controller, do we need collect logs from all of domain controllers? Or do we need to only collect log from the central/hq domain controller? Q13: Is it possible to do remote collection without using user with domain Admin or Admin privilege? Q14: Where can I find WinCollect plug-ins? Q15: Is it possible to create the authentication token for WinCollect agent through a CLI or script? Q16 - RoseD: Can WinCollect encrypt traffic that is remotely polled? For example, for the WinCollect method that polls for events, can the traffic which appears to be using RPC be encrypted? Q17: What is the recommended time zone setting for WinCollect Servers in a global deployment? 14

15 Questions for the panel? Now is your opportunity to ask questions of our panelists. To ask a question now: 1. Type your question into the chat window. 2. When prompted by the operator, you can press *1 to ask a question over the phone. Note: The next QRadar open mic is scheduled for September 30th, The topic is undecided at the moment, but mark your calendars! 15

16 Question 1 for the panel? Note: This slide was added as an answer to this question. Question: Is there a method for detecting non-compliant or rogue devices in QRadar? Answer: Yes, there are actually multiple methods. 1. DHCP logs provide very useful information for detecting new devices in a network. You can use reference set rules to trigger off offenses off of a known MAC address list or hostname list. Optionally, if you have hostname standards in your corporation, you can use rules to detect hostnames that differ from your company format. For example, DHCP events that do not include IBM.com or whatever your company hostname format is can be used to quickly identify rogue devices. 2. Using Vulnerability Assessment scans to scan the network and look for new devices. The scan data updates the Asset information in QRadar, which can be used to trigger offenses. 16

17 17 IBM Security Systems Question 2 for the panel? Note: This slide was added as an answer to this question. Question: Not interesting in remotely polling for events. Have local system installations been improved? Answer: Yes, we have been making continued improvements to WinCollect, including local system installations. Administrators who are not interested in remotely polling for events can install the agent on the remote Windows system and configure a log source using the Local System check box. This does not require credentials (if the agent is installed as an administrator) and forwards events over port 514. Optionally, administrators can use unmanaged mode to remove the port 8413 requirement as we released a user interface that allows WinCollect to act similar to the Adaptive Log Exporter (ALE). See: for more information. WinCollect uses more resources than ALE, however, it has more features than ALE and processes more events and handles more connections. We talked about system requirements earlier in the presentation and made mention that 8GB and the system resources are not necessarily required for low event rate systems. If the agent is installed as one agent to one operating system and forwarding local events, then 8GB RAM and 20% of the CPU would not be required as most endpoints (user workstations, not servers or domain controllers) generate less than 10 EPS on average.

18 Question 3 for the panel? Note: This slide was added as an answer to this question. Question: What is the upgrade path for WinCollect agents? Corrections: At first, I thought this was a question about how upgrades work. This slides clarifies the upgrade paths. Current QRadar version: QRadar 7.0 MR5 QRadar 7.1 MR2 Patch 1 or above Current Agent version: Step 1 Step 2 Requirements No upgrade path. WinCollect 7.0 is the only version available for QRadar appliances at QRadar 7.0 MR5. An upgraded QRadar deployment is required. No upgrade path. WinCollect requires an RPM and agent install. The Agent RPM on the Console must be installed before the administrator installs EXE files on the Windows host. QRadar 7.1 MR2 Patch 1 or above **7.2.1 ** Ensure Port 443 & 8413 is open between the Console and the agent BEFORE you download and install the agent RPM on the Console from IBM Fix Central. Ensure that Enable Automatic Updates for the agent = true. QRadar 7.1 MR2 Patch 1 or above QRadar or above **7.2.1 ** Ensure Port 443 & 8413 is open between the Console and the agent BEFORE you download and install the agent RPM on the Console from IBM Fix Central. Ensure that Enable Automatic Updates for the agent = true. QRadar 7.1 MR2 Patch 1 or above QRadar or above As WinCollect is installed, port 8413 should be open. Install the Agent RPM on the QRadar Console from IBM Fix Central and ensure Enable Automatic Updates for the agent = true. 18

19 Question 3 Continued / more information Note: This slide was added as an answer to this question. Do certain WinCollect features require a specific QRadar version? Yes. Feature Feature available in Minimum QRadar Version Automatic Log Source Creation Agent configurations through managed hosts WinCollect or above WinCollect or above QRadar version Patch 1 QRadar version Patch 3 Agent configurations through managed hosts This feature allows communication for port 8413 through appliances that have ECS components (16xx or 18xx appliances). This feature allows admins to manage larger agent deployments without having to send all connections and requests through the Console. In large agent deployments, this prevents performance issues when trying to process all of the agent requests and adds scalability improvements. To use this feature, the admin can specify the IP address of the 16xx or 18xx appliance in the Configuration Console Address field. 19

20 Question 3 Continued / more information Note: This slide was added as an answer to this question. Is logging clean-up a feature coming to WinCollect? Yes. Yes, there is an open feature request (FR) to have WinCollect agents purge the WinCollect logs in C:\Program Files\IBM\WinCollect\logs. This feature will be available in a future WinCollect Agent version. Note: The on-air answer was interpreted as not only cleaning up old logs, but to also make error messages easier to understand when issues occur. Our development team has been making improvements to how errors are logged in WinCollect. We plan to continue to improve features and we are evaluating ideas for adding QIDs and system notifications for error messages from WinCollect agents to help administrators identify specific agent issues. 20

21 Question 4 Note: This slide was added as an answer to this question. What improvements have been made to remote polling as WinCollect has progressed? The WinCollect update from 7.1.x to 7.2.x included a number of performance improvements to how many remote hosts a WinCollect agent can poll and the overall EPS. WinCollect agent supports tuning as mentioned in the audio, but WinCollect default installations support more default log sources and higher EPS rates in version 7.2.x. For example, let s compare the documented EPS rates from WinCollect version 7.1, 7.2.0, and

22 Question 4 continued / more information Note: This slide was added as an answer to this question. In WinCollect 7.1.x, we identified a maximum of 1,000 EPS per agent for remote event collection. The tables listed below have the published EPS rates from the latest to WinCollect releases. WinCollect tested EPS rates Installation Type Tuning EPS Log Sources Total EPS Local Collection Default Local Collection Tuned 2, ,000 Remote Collection Default ,000 Remote Collection Tuned Varies Varies 1,000+ WinCollect tested EPS rates Installation Type Tuning EPS Log Sources Total EPS Local Collection Default Local Collection Tuned 5, ,000 Remote Collection Default ,500 Remote Collection Tuned Varies Varies 2,

23 Where do I get more information? If you were unable to attend this webcast or have more questions, you can ask a question anytime in our QRadar Customer Forum: Resources: Article : Configuring DCOM and WMI to Remotely Retrieve Windows 7 Events ( Article : WinCollect Event Filtering ( Article : WinCollect Error Code 0x0005 Access Denied ( Article : WinCollect troubleshooting: The RPC server is unavailable. Error code 0x06BA ( Useful links : Getting Support for IBM Security QRadar products ( Follow us: IBM Support Portal Open a Service Request Update your PMR Escalate your PMR 23

24 Statement of Good Security Practices: IT system security involves protecting systems and information through prevention, detection and response to IBM Security Systems improper access from within and outside your enterprise. Improper access can result in information being altered, destroyed or misappropriated or can result in damage to or misuse of your systems, including to attack others. No IT system or product should be considered completely secure and no single product or security measure can be completely effective in preventing improper access. IBM systems and products are designed to be part of a comprehensive security approach, which will necessarily involve additional operational procedures, and may require other systems, products or services to be most effective. IBM DOES NOT WARRANT THAT SYSTEMS AND PRODUCTS ARE IMMUNE FROM THE MALICIOUS OR ILLEGAL CONDUCT OF ANY PARTY. Copyright IBM Corporation All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 24