TIBCO LogLogic. Universal Collector (UC) User Guide. Software Release: November 2012

Transcription

1 TIBCO LogLogic Universal Collector (UC) User Guide Software Release: November 2012

2 Important Information SOME TIBCO SOFTWARE EMBEDS OR BUNDLES OTHER TIBCO SOFTWARE. USE OF SUCH EMBEDDED OR BUNDLED TIBCO SOFTWARE IS SOLELY TO ENABLE THE FUNCTIONALITY (OR PROVIDE LIMITED ADD-ON FUNCTIONALITY) OF THE LICENSED TIBCO SOFTWARE. THE EMBEDDED OR BUNDLED SOFTWARE IS NOT LICENSED TO BE USED OR ACCESSED BY ANY OTHER TIBCO SOFTWARE OR FOR ANY OTHER PURPOSE. USE OF TIBCO SOFTWARE AND THIS DOCUMENT IS SUBJECT TO THE TERMS AND CONDITIONS OF A LICENSE AGREEMENT FOUND IN EITHER A SEPARATELY EXECUTED SOFTWARE LICENSE AGREEMENT, OR, IF THERE IS NO SUCH SEPARATE AGREEMENT, THE CLICKWRAP END USER LICENSE AGREEMENT WHICH IS DISPLAYED DURING DOWNLOAD OR INSTALLATION OF THE SOFTWARE (AND WHICH IS DUPLICATED IN THE LICENSE FILE) OR IF THERE IS NO SUCH SOFTWARE LICENSE AGREEMENT OR CLICKWRAP END USER LICENSE AGREEMENT, THE LICENSE(S) LOCATED IN THE LICENSE FILE(S) OF THE SOFTWARE. USE OF THIS DOCUMENT IS SUBJECT TO THOSE TERMS AND CONDITIONS, AND YOUR USE HEREOF SHALL CONSTITUTE ACCEPTANCE OF AND AN AGREEMENT TO BE BOUND BY THE SAME. This document contains confidential information that is subject to U.S. and international copyright laws and treaties. No part of this document may be reproduced in any form without the written authorization of TIBCO Software Inc. TIBCO and LogLogic are either registered trademarks or trademarks of TIBCO Software Inc. and/or subsidiaries of TIBCO Software Inc. in the United States and/or other countries. All other product and company names and marks mentioned in this document are the property of their respective owners and are mentioned for identification purposes only. THIS SOFTWARE MAY BE AVAILABLE ON MULTIPLE OPERATING SYSTEMS. HOWEVER, NOT ALL OPERATING SYSTEM PLATFORMS FOR A SPECIFIC SOFTWARE VERSION ARE RELEASED AT THE SAME TIME. PLEASE SEE THE README.TXT FILE FOR THE AVAILABILITY OF THIS SOFTWARE VERSION ON A SPECIFIC OPERATING SYSTEM PLATFORM. THIS DOCUMENT IS PROVIDED AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. THIS DOCUMENT COULD INCLUDE TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE PERIODICALLY ADDED TO THE INFORMATION HEREIN; THESE CHANGES WILL BE INCORPORATED IN NEW EDITIONS OF THIS DOCUMENT. TIBCO SOFTWARE INC. MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCT(S) AND/OR THE PROGRAM(S) DESCRIBED IN THIS DOCUMENT AT ANY TIME. THE CONTENTS OF THIS DOCUMENT MAY BE MODIFIED AND/OR QUALIFIED, DIRECTLY OR INDIRECTLY, BY OTHER DOCUMENTATION WHICH ACCOMPANIES THIS SOFTWARE, INCLUDING BUT NOT LIMITED TO ANY RELEASE NOTES AND "READ ME" FILES. Copyright TIBCO Software Inc. ALL RIGHTS RESERVED. TIBCO Software Inc. Confidential Information

3 Contents Contents 3 List of Figures 7 List of Tables 9 Preface About This Guide 11 Audience Technical Support Information Documentation Support Information Contact Information Conventions Chapter 1 Introduction and Requirements 13 Introduction Aims and Assets of Universal Collector Requirements General Security Considerations Supported Platforms Minimum Hardware Configuration Ports Limitations JRE Documentation Chapter 2 Install UC 19 Recommendations Install Universal Collector in Graphical Mode Start Graphical Mode Install Universal Collector Uninstall Universal Collector Install Universal Collector in Console Mode Start Console Mode Install Universal Collector Uninstall Universal Collector Install Universal Collector in Silent Mode Start Silent Mode Install Universal Collector Uninstall Universal Collector Universal Collector User Guide 3

4 Chapter 3 Collect Logs 29 About the Log Sources Real-Time File Logs Windows Event Logs Syslog Logs Remote Files UC Internal Logs Create and Configure Log Sources Create a Log Source Create Several Log Sources at a Time Create a Complete Configuration Edit Log Sources Edit a Real-Time File Log Source Edit Several Real Time File Log Sources Edit a Windows Event Log Source Edit Several Windows Event Log Sources Edit a Syslog Log Source Edit Several Syslog Log Sources Edit a Remote File Log Source Edit Different Log Sources at a Time Sorting Log Sources Create Tags Sort Log Sources Chapter 4 Forward Logs 69 Create a TCP (Syslog) or UDP (Syslog) Connection Create an LMI Connection Create a Connection in Authentication and/or Encryption Mode Step 1 - Get a Root Certificate Authority from your PKI Step 2 - Create a Certificate Signing Request Step 3 - Create a valid UC certificate using a CA and OpenSSL Step 4 - Import the Certificate into *.ks or *.p Step 5 - Configure the Forwarding Process Step 6 - Enable Secure Connection Manage the list of Forwardings Graphical User Interface Overview Copy a Forwarding Delete a Forwarding Chapter 5 Monitor UC Activities 83 Start UCMon Tool Summary Screen Status Screen Log Source Forwarding Connection Universal Collector User Guide

5 Metrics Screen Log Source Forwarding Connection Trends Screen Log Source Forwarding Connection RealTime Screen Log Sources Forwarding Connection Chapter 6 Command Line Interface 95 Overview cert_mgt: Manage the Security Certificates uc_checkconf: Check the Current Configuration uc_createlogsources: Import and Create Several Log Sources at a Time uc_decodepwd: Decode Passwords for Windows Files uc_encryptpwd: Encrypt Passwords for Windows Files uc_monitor: UCMon Tool uc_reload: Reload Configuration uc_saveactiveconfas: Save an Active Configuration uc_switchto: Make Configuration Active Switching from One Configuration to Another Checking the Impacted Processes Limitations Appendix 103 Sample Configuration Files [UC Configuration] uc.xml [LMI Connection] uldp-samplecommented.uldp.xml [LMI Connection] uldp-samplecommentedauthjks.uldp.xml [LMI Connection] uldp-samplecommentedauthpem.uldp.xml [LMI Connection] uldp-samplecommentedauthpks12.uldp.xml [Log Sources] file-samplecommented.ls.xml [Log Sources] syslog-samplecommented.ls.xml [Log Sources] wmi-samplecommented.ls.xml Regular Expressions Index 1 Universal Collector User Guide 5

6 6 Universal Collector User Guide

7 List of Figures Figure 1: Universal Collector Principle Figure 2: TCP port required for Windows log collection Figure 3: Launch and Activation Permission Figure 4: Windows Event Logs Control Properties Figure 5: Windows Event Logs Control Properties Figure 6: RT File Edition tab Figure 7: Windows Event Log Edition tab Figure 8: Syslog Edition tab Figure 9: All tab Figure 10: Two filters are applied Figure 11: Two values are applied Figure 12: Overview of the creation of a UC certificate Figure 13: Generation of a Root CA Figure 14: Creation of a valid certificate Figure 15: The three supported formats Figure 16: LMI Connection Universal Collector User Guide 7

8 8 Universal Collector User Guide

9 List of Tables Table 1: Supported platforms Table 2: Minimum configuration Table 3: Ports Table 4: MC Agent Ports Table 5: Types of installation recommended Table 6: Silent mode - Values to modify Table 7: Multi-line log sources supported by default Table 8: Audit Policy Table 9: Protocols Table 10: CSV fields Table 11: CSV Example Table 12: General Settings Edition Table 13: Real-Time File edition parameters - General Table 14: Real-Time File edition parameters - Forwarding Connection Table 15: Real-Time File edition parameters - Message Filtering Table 16: Real-Time File edition parameters - Collection Table 17: Windows Event Log edition parameters - General Table 18: Windows Event Log edition parameters - Forwarding Connection Table 19: Windows Event Log edition parameters - Message Filtering Table 20: Windows Event Log edition parameters - Collection Table 21: Syslog edition parameters - Description Table 22: Syslog edition parameters - Forwarding Connection Table 23: Syslog edition parameters - Collection Table 24: Syslog edition parameters - Message Filtering Table 25: Remote File edition parameters - Description Table 26: Remote File edition parameters - Forwarding Connection Table 27: Remote File edition parameters - Collection Table 28: All parameters - General Table 29: All parameters - Forwarding Connection Table 30: Filters Table 31: Forwarding Connection edition - Security Table 32: Forwarding GUI Table 33: UCMon - Summary screen Table 34: UCMon - Log Source Status Table 35: UCMon - Forwarding Status Table 36: UCMon - Log Source Metrics Table 37: UCMon - Forwarding Connection Metrics Table 38: UCMon - Log Source Trends Table 39: UCMon - Forwarding Connection Trends Table 40: UCMon - RealTime Table 41: UCMon - Forwarding Connection RealTime Table 42: Starting the CLI Table 43: cert_mgt Table 44: uc_checkconf Universal Collector User Guide 9

10 Table 45: uc_createlogsources Table 46: Command Line Interface to decode passwords Table 47: Command Line Interface to encrypt passwords Table 48: Command Line Interface Table 49: uc_reload Table 50: uc_saveactiveconfas Table 51: uc_switchto Universal Collector User Guide

11 Preface About This Guide This User Guide provides the information needed to quickly install, configure, and uninstall the Universal Collector (UC), a lightweight but powerful software agent that collects event logs locally or remotely by acting as a Syslog listener or file collector. Audience The guide is intended for Security Network Administrators who are responsible for installing and maintaining network security software. Technical Support Information LogLogic is committed to the success of our customers and to ensuring our products improve customers' ability to maintain secure, reliable networks. Although LogLogic products are easy to use and maintain, occasional assistance might be necessary. LogLogic provides timely and comprehensive customer support and technical assistance from highly knowledgeable, experienced engineers who can help you maximize the performance of your LogLogic Compliance Suites. To reach LogLogic Customer Support: Telephone: Toll Free LOGS Local EMEA [email protected] You can also visit the LogLogic Support website at: When contacting the Support, be prepared to provide the following information: Your name, address, phone number, and fax number Your company name and company address Your machine type and release version A description of the problem and the content of pertinent error messages (if any) Universal Collector User Guide 11

12 Documentation Support Information Documentation Support Information The LogLogic documentation includes Portable Document Format (PDF) files. To read the PDF documentation, you need a PDF file viewer such as Adobe Acrobat Reader. You can download the Adobe Acrobat Reader at Contact Information Your feedback on the LogLogic documentation is important to us. If you have questions or comments, send to [email protected]. In your message, please indicate the software name and version you are using, as well as the title and document release date of your documentation. Your comments will be reviewed and addressed by the LogLogic Technical Publications team. Conventions The LogLogic documentation uses the following conventions to distinguish text and information that might require special attention. Caution: Highlights important situations that could potentially damage data or cause system failure. IMPORTANT! Highlights key considerations to keep in mind. Note: Provides additional information that is useful but not always essential or highlights guidelines and helpful hints. This guide also uses the following typographic conventions to highlight code and command line elements: Monospace is used for programming elements (such as code fragments, objects, methods, parameters, and HTML tags) and system elements (such as file names, directories, paths, and URLs). Monospace bold is used to distinguish system prompts or screen output from user responses, as in this example: username: system home directory: home\app Monospace italic is used for placeholders, which are general names that you replace with names specific to your site, as in this example: LogLogic_home_directory\upgrade\ Straight brackets signal options in command line syntax. ls [-AabCcdFfgiLlmnopqRrstux1] [-X attr] [path...] 12 Universal Collector User Guide

13 CHAPTER 1 Introduction and Requirements Contents General Security Considerations Supported Platforms Minimum Hardware Configuration Ports Limitations Documentation Introduction The Universal Collector is a software agent that collects logs from any device or log source and forwards them to an LMI Appliance or a Syslog server by using UDP, TCP, or the Universal Log Data Protocol proprietary protocol. UC collects the information from four types of log sources: Syslog, Windows event logs, Real-Time File pull, or Remote File pull. Several UC agents can be deployed on a dedicated/shared appliance or physical/virtual hardware to remotely collect hundreds of log sources located at the same site. Aims and Assets of Universal Collector Collecting Logs: UC allows you to gather data from several types of log sources while ensuring confidentiality of the logs. You can easily collect event logs from local or remote instances of MS Windows, including time-stamped or rotated files. The UC agent works as a Syslog listener. Forwarding Logs: UC forwards secure and authenticated data to an LMI server via a ULDP protocol without the need for a dedicated appliance. UC 2.3 also forwards to Syslog server with either UDP or TCP protocols. Monitoring Activity: A UCMon tool is also available to monitor the internal process of the UC and therefore make sure that your collection and forwarding processes are correctly responding. Universal Collector User Guide 13

14 Introduction Easy and Wise Configuration: A UC configuration is composed of Log Sources, Forwarding connections and UC general parameters. UC configuration must be created and updated via the GUI or the Command Line Interface. You can create, save and store a configuration. A stored configuration is useful: - to create a configuration and then activate it whenever you want, even if an active configuration is open, i.e. another configuration is running on the system - to create several configurations and deploy them rapidly on other UCs. Easy Management: Multiple UCs can be remotely managed using LogLogic Management Center (MC) and MC Agent configured and running on each UC Asset. MC is a software solution that allows you to manage Assets, schedule batch upgrade for Assets, monitor system health check, and backup and restore Asset data. Adaptability: UC is a software program with a small footprint and low memory usage on your Domain Controllers, or application services. It is highly adaptable and can be customized easily. Its lightweight although reliable configuration helps the user to manage changes according to his particular needs. Figure 1 Universal Collector Principle 14 Universal Collector User Guide

15 CHAPTER 1 Introduction and Requirements Requirements General Security Considerations The following section provides security recommendations that need to be followed to install the Universal Collector in a secure manner. Full security The administrator must ensure that the machines on which Universal Collectors are installed are fully secured. These machines should be located in a physically secure environment in which only trusted personnel have access. Password The administrator must ensure that the default admin password to connect to the machine is changed. A good password has a combination of alphabetic and numeric characters and comprises at least eight characters in length. It should be known by a very restricted number of people. Updates All operating systems and software installed on the machine must be correctly updated to avoid any security breach which could expose admin rights. Supported Platforms The machine where the Universal Collector is installed must be safe and secured by a reliable administrator. It must also be synchronized according to a NTP server (for reliable time management). Table 1 Supported platforms Type Microsoft Windows Linux/Unix Operating Systems Microsoft Windows 2003 R2 32-bit and 64-bit Microsoft Windows bit and 64-bit Microsoft Windows 2008 R2 64-bit Microsoft Windows 7 32-bit and 64-bit Redhat Enterprise Linux (RHEL) v5 32-bit and 64-bit Redhat Enterprise Linux (RHEL) v6 32-bit and 64-bit SUSE Linux Enterprise Server (SLES) v11 32-bit and 64-bit Solaris 10 x86 64-bit Solaris 10 Sparc 64-bit Solaris 11 x86 64-bit Solaris 11 Sparc 64-bit Note: With Solaris x86_64, UC will run only in 32 bit mode, even under 64 bit jvm. Universal Collector User Guide 15

16 Requirements Minimum Hardware Configuration Table 2 Minimum configuration Type Multi-core CPU Disk Space RAM Minimum Configuration 2GHz min. 5 GB 2 GB for local collection (Agent mode) or 4 GB for remote Windows collection (Collector mode) Note: Once UC Console is installed, you can check the memory used by UC service compared to the maximum memory size allocated to UC in the About screen (Help > About) as well as the version and revision number. Ports The following section describes the ports used by UC and the MC Agent. UC To use the Universal Collector along with the LMI, you will need to open specific TCP ports according to the versions of LMI currently in use. Table 3 Ports Ports Use 5514 For non-authenticated and non-encrypted connection with LMI 5.0 and For authenticated and/or encrypted Secure connection with LMI 5.0 or later (although default can be configured) 5516 For non-authenticated and non-encrypted connection with LMI 5.2 or later Make sure that the UC - LMI network connection is not blocked by a firewall. MC Agent The MC Agent communicates with Management Center about the Asset's features. To use the Universal Collector along with the MC, you will need to open specific TCP ports that are used by the MC Agent. Table 4 Ports MC Agent Ports Use Port number used by MC Agent to notify MC about changes RMI Server Port number. The default value is This must be a number between 1024 to RMI Registry Port number. The default value is This must be a number between 1024 to Make sure that the MC Agent - LMI network connection is not blocked by a firewall. 16 Universal Collector User Guide

17 CHAPTER 1 Introduction and Requirements Limitations UC 2.3 will not provide large scale enabled management software for the standalone UCs as it is assumed that in large scale enterprise environments IT has already implemented scalable solutions for deploying and monitoring software applications. The minimum resolution to display UC Console is 1024x768. The installer requires JRE 7 to be installed on the system, and also the java executable must be in the user's PATH. In Linux it is recommended to install the JRE using an RPM (SuSE, RedHat) so that java is automatically put in the user's path. JRE Before installing UC 2.3.0, you must have JRE 7u5 or later installed on your machine. Older versions of JRE 7 are not supported. On Windows: At startup, UC will attempt to discover JRE from the system PATH. If you install the JRE on Windows using the JRE installer, then you should not experience any problems as long as you install it using the installer defaults and do not change the location where the JRE is to be installed. On a Linux system: In Linux it is recommended to install the JRE using an RPM (SuSE, RedHat). Note: If you make any changes to the location where you install the JRE, you should restart the service(s). On Windows you should run UC_HOME\bin\uninstall-service.bat and then UC_HOME\bin\install-service.bat to ensure that the service has the correct environment. On Linux and Solaris you should run UC_HOME\bin\uninstall-service and then UC_HOME\bin\install-service. Documentation The PDF documentation is available in the doc folder contained in the installation directory. The Online Help is accessible from the UC Console via the Help menu. The Online Help is only supported by the latest versions of Internet Explorer and Mozilla Firefox. Universal Collector User Guide 17

18 Requirements 18 Universal Collector User Guide

19 CHAPTER 2 Install UC Contents Recommendations Install Universal Collector in Graphical Mode Install Universal Collector in Console Mode Install Universal Collector in Silent Mode Recommendations According to your needs, you may decide to install UC on one or several machines. In all cases, make sure you have administrator rights on the machine where you want to install UC. Table 5 Types of installation recommended Type of installation Graphical mode Console mode Silent mode Recommended if... You want to install UC on a sole machine on a Windows platform. You want to install UC on a sole machine on a Red Hat, SuSE Linux Enterprise platform or Solaris. You want to rapidly deploy UCs on several machines with no interaction with your machine. Note: Upgrading UC from a 32-bit to a 64-bit platform and vice-versa is not supported. Universal Collector User Guide 19

20 Install Universal Collector in Graphical Mode Install Universal Collector in Graphical Mode First, you must modify the User Account Control settings. To do so: 1. Go to Control Panel > System and Security > click on Change User Accounts Control Settings. 2. In the User Account Control Settings screen, move the slider to the Never notify position and click OK. 3. If you are asked to enter an administrator password, enter it and confirm. Start Graphical Mode To start the Graphical Mode on MS Windows: 1. Log in as Local Administrator. There is no need to have Domain Administrator rights, a Local Administrator account level is enough to run the setup program. 2. Run UC setup program. An installation screen is displayed. To start the Graphical Mode on Red Hat or SuSE Linux Enterprise or Solaris: 1. Connect to the machine as user root. 2. Set the permission to access to the installer folder with the command: chmod 755 <installer file> 3. Launch the Universal Collector executable file and enter: loglogic-uc setup-unix.bin -i gui Install Universal Collector Note: Microsoft Visual C Redistributable SP1ATL will be automatically installed on your computer. This is the main component to make the UC run so do not uninstall it while the UC is running. 1. Read the Introduction screen and click Next. 2. Read the License Agreement screen then click the option to accept the license and click Next. 3. If necessary, change the installation folder path in the Choose Install Folder and click Next. 4. Enter the absolute path of the data folder in the Get User Input screen. The Data Folder will contain: The spool file with all the collected logs (default size is 100MB) Metadata about collected logs (to restore log collection) LogLogic Universal Collector internal logs 20 Universal Collector User Guide

21 CHAPTER 2 Install UC Note: A file spool of 100 MB is dedicated to each LMI connection. Therefore, a minimum of 200 MB of disk space is required to install the UC with one LMI connection. 5. Select Configure MC Agent if you want to Configure MC Agent now. MC is a product which enables you to centrally monitor health and status of UC Assets and perform upgrade tasks. If you do not intend to install MC, then you do not need to configure the Agent and click Next to go directly to the end of the installation process. 6. In the Get User Input - MC Agent Configuration screen, provide the following values and click Next: In the Management Center Host field, enter the IP Address of MC. This enables MC to manage its Agents and for each MC Agent to send information about its assets and health to MC. In the MC Notification Port field, enter the the port MC Agents use to communicate with Management Center. In the Repositories field, enter the URL that the MC Agent will use to discover upgradeable artifacts. 7. In the Get User Input - MC Agent Advanced Configuration screen, provide the following values and click Next: In the Server Port field, enter the server port used by the MC Agent for monitoring. In the Registry Port field, enter the registry port used by the MC Agent for monitoring. 8. In the Verify MC Agent Settings screen, check if all the data are correct and click Next. Note: The MC Agent is always installed in the mcagent directory located in your installation folder path. For more information on MC and MC Agent, including configuring and verifying MC Agent Settings after installation of MC, as well as instructions on installing the Agent as a service, refer to the MC documentation. 9. Keep the Launch UC Console selected if you want to start the UC Console as soon as the installation process. Otherwise deselect it and click Next. 10. When the installation is complete, click Done. If the installation has not been completed successfully, open the Universal_Collector_Install_[Timestamp].log to check error messages. This file is in <UC_HOME>\logs.A wrapper.log is also created when the UC service starts. Note: For security reasons, only the administrator group, the file owner and the system group are allowed to access this folder. Uninstall Universal Collector To uninstall UC on Windows: 1. On the Windows host, log in as Local Administrator. There is no need to have Domain Administrator rights, a Local Administrator account level is enough to run the uninstall program. Universal Collector User Guide 21

22 Install Universal Collector in Graphical Mode 2. Display the Uninstall window by going to Program Files > LogLogic > Universal Collector > click on Uninstall. 3. In the Uninstall Universal Collector screen, click on Uninstall. 4. Click on the Done button to close the window. To uninstall UC on Red Hat, SuSE Linux Enterprise: 1. Connect to the machine as user root. 2. Go to the installation folder and enter the following command:./uninstaller/uninstall 3. Follow the procedure for Windows. 22 Universal Collector User Guide

23 CHAPTER 2 Install UC Install Universal Collector in Console Mode Start Console Mode To start the Console Mode on Windows: 1. Connect to the machine as local administrator. 2. Open a command prompt. 3. From the installation folder, enter the command: loglogic-uc setup-windows.exe -i console To start the Console Mode on Red Hat or SuSE Linux Enterprise: 1. Connect to the machine as user root. 2. Set the permission to access to the installer folder with the command: chmod 755 <installer file> 3. From the installation folder, launch the Universal Collector executable file: loglogic-uc setup-unix.bin Install Universal Collector 1. Read the introduction and press Enter until you are asked to accept the license. 2. Press y (for yes) to accept the license and press Enter. 3. Enter the path to the UC Installation directory, then press Enter. 4. The default directory for data storage is usually appropriate, but it can be changed. Press Enter. 5. The default Data Folder is usually appropriate, but it can be changed. It will contain: the spool file containing all collected logs (default size is 100MB) metadata about collected logs (in order to restore log collection) Universal Collector internal logs 6. Verify the capacity of the directory in which to store collected data and click Next. 7. Type the number 2 if you want to configure the MC Agent now. MC is a product which enables you to centrally monitor health and status of UC Assets and perform upgrade tasks. If you do not intend to install MC, then you do not need to configure the Agent and click Next to go directly to the end of the installation process. Universal Collector User Guide 23

24 Install Universal Collector in Console Mode 8. In the Get User Input - MC Agent Configuration, provide the following values then press Enter. In the Management Center Host field, enter the IP Address of MC. This enables MC to manage its Agents and for each MC Agent to send information about its assets and health to MC. In the MC Notification Port field, enter the the port MC Agents use to communicate with Management Center. In the Repositories field, enter the URL that the MC Agent will use to discover upgradeable artifacts. 9. In the Get User Input - Servert Port, enter the enter the server port used by the MC Agent for monitoring. 10. In the Get User Input - Server Registry Port Registry Port, enter the registry port used by the MC Agent for monitoring. 11. In the Verify MC Agent Settings screen, check if all the data are correct and click Next. Note: The MC Agent is always installed in the mcagent directory located in your installation folder path. For more information on MC and MC Agent, including configuring and verifying MC Agent Settings after installation of MC, as well as instructions on installing the Agent as a service, refer to the MC documentation. 12. Click Install in the Pre-Installation Summary screen and wait until the installation process has finished. 13. An Installation Complete prompt appears. Press Enter to validate the installation. If the installation has not been completed successfully, open the Universal_Collector_Install_[Timestamp].log to check error messages. This file is located in <UC_HOME>/logs. Note: For security reasons, only the root account and the file owner are allowed to access this folder. Uninstall Universal Collector To uninstall UC on Windows: 1. From the uninstall folder, enter the following command: uninstall.exe -i console To uninstall UC on Red Hat or SuSE Linux Enterprise: 1. Connect to the machine as user root. 2. Go to the installation folder and enter the following command:./uninstaller/uninstall 3. Press Enter and decide whether you want to keep configuration data or not after the uninstallation process by selecting the relevant option. 24 Universal Collector User Guide

25 CHAPTER 2 Install UC 4. Press Enter. The uninstallation is in progress. Universal Collector User Guide 25

26 Install Universal Collector in Silent Mode Install Universal Collector in Silent Mode Start Silent Mode The silent mode is recommended when you decide to rapidly deploy UCs on several machines. You must create an installation file in which you must enter the following values: Table 6 Silent mode - Values to modify Values Description INSTALLER_UI Indicate the installation mode, e.g. silent Note: you can also indicate another type of installation mode such as console or gui. USER_INSTALL_DIR Indicate the UC s installation directory, e.g. /opt/ LogLogic/Universal_Collector USER_INPUT_INSTALL_DATA_DIR Indicate the UC s data directory, e.g. /opt/loglogic/ Universal_Collector configuremcagent Set this value to '1' to tell the installer you want the MC Agent configured and installed as a service. mchost Indicate the MC Host. Default value: but it should be changed to the IP address where MC is running. mcnotificationport Indicate the MC Notification Port. Default value: rmiserverport RMI Server Port. Default value: rmiregistryport RMI Registry Port. Default value: agentaddress Set IP address of the MC Agent. Default value: empty string. This field must be empty unless the MC Agent is hosted on a multi-homed host. enableheartbeat Default value: true. heartbeatinterval Default value: 720. org.ops4j.pax.url.mvn.repositor ies Set the location of repositories (to have the MC Agent point to multiple repositories in case the first one in the list is not available.) Install Universal Collector To install on Windows: 1. Create the installation file, e.g. silent.txt. 2. Enter the installation information: INSTALLER_UI=silent 26 Universal Collector User Guide

27 CHAPTER 2 Install UC USER_INSTALL_DIR=C:\\Program Files\\LogLogic\\Universal Collector USER_INPUT_INSTALL_DATA_DIR=C:\\Program Files\\LogLogic\\Universal Collector 3. Configure MC Agent if you want to manage UC using MC: configuremcagent=1 mchost=mc.host.loglogic.com mcnotificationport= Start the installation: C:\Temp\UC>loglogic-uc setup-windows.exe -f silent.txt 5. The installation is complete. 6. If the installation has not been completed successfully, open the Collector_install.log to check error messages. This file is by default located in C:\Program Files\LogLogic\Universal Collector\<DATA DIR>\data\logs. To install on Red Hat or SuSE Linux Enterprise: 1. Open a Shell program and login as root. 2. Set the permission to access to the installer folder with the command: chmod 755 <installer file> 3. Create and complete the installation file, e.g. silent.txt: 4. Start the installation: bash-3.2# cat silent.txt INSTALLER_UI=silent USER_INSTALL_DIR=/opt/LogLogic/uc USER_INPUT_INSTALL_DATA_DIR=/opt/LogLogic/uc configuremcagent=1 mchost=x.x.x.x mnotificationport=41616 org.ops4j.pax.url.mvn.repositories= bash-3.2# sh./loglogic-uc setup-unix.bin -f silent.txt 5. The installation is complete. Uninstall Universal Collector To uninstall on Windows: 1. In the silent.txt you have created when installing UC, enter the uninstall information: INSTALLER_UI=silent 2. Go to uninstall folder: C:\Program Files\LogLogic\Universal Collector\uninstaller Universal Collector User Guide 27

28 Install Universal Collector in Silent Mode 3. Start the uninstall: uninstall.exe 4. The uninstall is complete. To install on Red Hat or SuSE Linux Enterprise: 1. Open a Shell program and login as root. 2. In the silent.txt you have created when installing UC, enter the uninstall information: INSTALLER_UI=silent 3. Go to the installation folder: /opt/loglogic/universal_collector/uninstaller 4. Start the uninstall:./uninstall 5. The uninstall is complete. 28 Universal Collector User Guide

29 CHAPTER 3 Collect Logs Contents About the Log Sources Create and Configure Log Sources Edit Log Sources Sorting Log Sources About the Log Sources Real-Time File Logs The Universal Collector reads logs from local files - i.e. logs from files generated on the machine where UC is installed - and forwards them to either an LMI or a Syslog server. This chapter describes how Universal Collector handles file collection from files: - Collection of Single-line Messages - Where does collection starts? - What is collected? - What is forwarded? - Are messages lost when UC restarts? - Log File Rotation Collection of Multi-line log messages Universal Collector User Guide 29

30 About the Log Sources Collection of Single-line Messages Where does collection starts? When a file is collected, only the newly added logs at the end of the file are collected. Logs already available in a file before the UC log source creation will not be collected. What is collected? UC operates by monitoring specified text files that are receiving log output from log sources. The log sources append new logs to the end of the text file as events occur. As new records appear at the tail of the monitored file, they are instantly taken into account by UC. What is forwarded? UC forwards single-line log messages to an LMI or Syslog server. By default, UC sends a maximum of characters per line. Are messages lost when UC restarts? UC uses cursors to track the monitored files and to resume continuously after having stopped. 30 Universal Collector User Guide

31 CHAPTER 3 Collect Logs The cursors have information about the file positions at which to restart - called metadata - as well as file identification information. It can determine whether the file to be resumed is the file to which the saved position applies. In other terms, even if the UC is stopped for a while, all the messages contained in the file will be collected thanks to the position cursors, no messages will be lost. Log File Rotation Log file rotation is not managed by UC; it is the user's responsibility to manage the file rotation. In the case of log file rotation, a log file is retired and renamed to a rotated name, and the monitored file is replaced by a new log file. Therefore, periodically during the monitoring of a log file that is rotated, the file is replaced by a fresh log file. Depending on your log rotation tool, UC is able to manage rotation files in two different ways. 1) The log file name contains a date that changes during the rotation process The UC handles the rotation process of logs that contain a date in their name provided you correctly configured the File Log Source configuration file. If you enter the parameter [date] in the file path you must: 1. Activate the file rotation. Enable and enter a date format for the date pattern such as yyyymmdd (see docs.oracle.com/javase/7/docs/api/java/text/simpledateformat.html ) Example with yyyymmdd: Filenames: logfile log, logfile log... Absolute path: c:\logdir\logfile.[date].log 2) The log file name contains an id that changes during the rotation The UC handles the rotation process of logs that contain an Id in their name provided you correctly configured the File Log Source configuration file. Universal Collector User Guide 31

32 About the Log Sources If you enter the parameter [id] in the file path you must: 1. Activate the file rotation. 2. Enable and enter the number of digits expected (1-9) for the nbdigit parameter. Example with only one digit: Filenames: logfile.1.log, logfile.2.log... Absolute path: c:\logdir\logfile.[id].log You can combine the two examples to allow the use of both [id] and [date] parameters in the file path. Recommendations - In the case of resuming after having been stopped, if the log file has been rotated during the period in which the collector was stopped, some log data will be missed. Therefore, you must ensure that the collector is not temporarily stopped during an interval in which a rotation occurs. - To be collected, a file must have been modified after the latest collected file. - The log file name does not change during the rotation. The UC records the identity of a log file in the cursor as a hash of the first several bytes of the file. When the file is rotated and replaced with a fresh one, the hash will be different. File identity checking is performed throughout the log file monitoring process to detect log rotation. - If a log file needs to be replaced and enriched while UC is running, do not copy content in the file but move it on the same partition. Collection of Multi-line log messages The Universal Collector can combine multiple consecutive related lines or multi-lines in a source log file into a single line which will be sent to the LMI. Multi-line message groups may require analysis to determine the correct expression to use if the format is complex. Before sending, groups of lines that represent a logical message are converted to a single-line format. All of the original messages' data is kept intact nothing is altered. UC can collect multi-line messages from default application sources or custom ones: Table 7 Multi-line log sources supported by default Log Sources Tomcat / Servlet Container WebLogic Application Server Description Default log location is CATALINA_BASE/logs. Tomcat and application logs unless configured otherwise. The default format is multi-line, with the first line beginning with a timestamp. It may change due to localization. Logs are rotated daily by default Default log location is under the server root DOMAIN_NAME/servers/ ADMIN_SERVER_NAME/logs/. Each server or cluster maintains a server log and selected events are forwarded to a domain log. Most of the entries are single line, but can contain java exceptions. Each message begins with '####'. There may also be a web access log 32 Universal Collector User Guide

33 CHAPTER 3 Collect Logs Table 7 Multi-line log sources supported by default Log Sources WebSphere Application Server JBoss Application Server Description Default log location is under the WebSphere directory APPSERVER/profiles/PROFILENAME/ logs/servername/. There is no default log rotation. There are server start and stop logs (SystemErr.log, SystemOut.log), JVM log files (native_stderr.log, native_stdout.log), and process log files (startserver.log, stopserver.log). All of these logs contain entries describing the system environment that do not have a timestamp. The error logs do not contain any timestamps. Continuation lines are indented Default log location is JBOSS_HOME/server/NAME/log. The boot log records startup events prior to the initialization of the logging service. The server.log file records activity while the server is running. The boot.log file entries begin with a time with no date. The server.log file entries start with a timestamp in the form 'YYYY-MM-DD HH:MI:SS,FFF'. Log messages can be multi-line and the continuation lines are sometimes indented, but frequently not. Messages start with a timestamp. Note: The regex format for these default applications are indicated in <InstallationFolder>\runtime\conf\static\line_combiner.xml file. Custom multi-line: Custom regex can be defined for custom multiline logs. You need to define - the header regex pattern. - whether you keep orphaned lines, i.e UC sends messages that do not match the Header Regexp - the timeout after which messages are sent even if the regex is not found again. Custom Multi-line log sources: Custom regex can be defined for custom multiline logs. You need to define - the header regex pattern. - whether you keep orphaned lines, i.e UC sends messages that do not match the Header Regexp - the timeout after which messages are sent even if the regex is not found again. Example of a custom application logs: :09:41,344 WARN [main] file.fileimportsqldao (?(think)) - File not found (/home/exaprotect/conf/tbsmp6/report/ etc/export.properties) :09:41,344 WARN [main] config.configurationfactory (ConfigurationFactory.java:127) - No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: jar:file:/home/exaprotect/report_tbsmp6/webapps/ ExaReport/WEB-INF/lib/ehcache jar!/ehcache-failsafe.xml java version "1.6.0_18" Java(TM) SE Runtime Environment (build 1.6.0_18-b07) :09:50,723 INFO [main] config.facesconfigurator (FacesConfigurator.java:151) - Reading standard config org/apache/ myfaces/resource/standard-faces-config.xml Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode) Universal Collector User Guide 33

34 About the Log Sources In the UC Console, you can create a regex like: ^\d+-\d+-\d+\s\d+:\d+:\d+,\d+\s[^\s]+\s+\[[^\]]+\]\s with a timeout of 3 seconds and indicating that orphaned lines are kept. It will match the header of the multiline log (date and level), which is: :09:41,344 WARN [main] All the lines will be aggregated and then forwarded as a single log to LMI. The \r and \n will be replaced by escaped ones \\r and\\n), until UC finds another regex header. You can obtain something like: :09:41,344 WARN [main] file.fileimportsqldao (?(think)) - File not found (/home/exaprotect/conf/tbsmp6/report/ etc/export.properties) :09:41,344 WARN [main] config.configurationfactory (ConfigurationFactory.java:127) - No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: jar:file:/home/exaprotect/report_tbsmp6/webapps/ ExaReport/WEB-INF/lib/ehcache jar!/ ehcache-failsafe.xml\r\njava version "1.6.0_18"\r\nJava(TM) SE Runtime Environment (build 1.6.0_18-b07) :09:50,723 INFO [main] config.facesconfigurator (FacesConfigurator.java:151) - Reading standard config org/apache/ myfaces/resource/standard-faces-config.xml\r\n Java HotSpot(TM) 64-Bit Server VM (build 16.0-b13, mixed mode) Note: Refer to section Appendix to get the full content of the Real-Time File Log Source commented file. 34 Universal Collector User Guide

35 CHAPTER 3 Collect Logs Windows Event Logs The Universal Collector can collect Windows events. The supported Windows versions for remote collection are Windows 2003 R2 (32-bit/64-bit), Windows 2008 (32/64-bit), Windows 2008 R2 64-bit, Windows 7 (32/64-bit). Note: The Universal Collector forwards Windows logs to the LMI appliance by using an LMI connection. Windows logs collected from UC are forwarded in a format which is based upon the SNARE format although UC and SNARE formats are not 100% similar and a subtle difference may exist for certain messages. Collecting Windows Event Logs in Agent Mode (Windows only) Configuring UC in agent mode for local collection of Windows event logs is straightforward: Create a Windows log source Optionally fine tune the Windows Audit Policy When collecting Windows logs, all log source configuration files (XML files) must refer to the same Windows Journals - although the filters can be different. If Windows Journals are different, the log collection will be duplicated. Collecting Windows Event Logs in Collector mode (Windows, RHEL, SuSE, Solaris) There are 3 main steps to use the Collector mode: Step 1 - Collecting Logs Step 2 - Editing Registry of Remote Windows Log Sources (RHEL, SuSE and Solaris only) Step 3 - Verifying Connection To Remote Windows Log Sources Step 1 - Collecting Logs Firewall filtering when a UC is installed on Windows When a firewall is in the communication path of two Windows hosts, the firewall must be configured to allow communication between the hosts. To do so: 1. Make sure that RPC is allowed by opening the port 135/tcp (EPMAP service) on your firewall. Universal Collector User Guide 35

36 About the Log Sources Figure 2 TCP port required for Windows log collection 2. Open another TCP port. This dynamic port may vary depending on the WEL configuration on the remote host. 3. To fix a specific port for WEL, connect to the polled Windows machine and launch dcomcnfg.exe in a cmd. A graphical window is opened. 4. Expand Components Services > Computers > My Computer > DCOM Config. 5. Right click on Windows Management and Instrumentation > Properties > click on EndPoints tab. 6. Click Add, then choose the Use static endpoint option, specify a port to use and validate. 7. Restart the Windows Management and Instrumentation service and its possible dependencies. 8. On the firewall, open the port specified in step 6. Example of a uc logs - failed connection to the polled machine: Date= :09:02,771 Level=WARN Message=COM ERROR : Could not connect err code = 0x800706ba RPC server is not available. LS= Type=wmi When a UC is installed on RHEL, SuSE or Solaris To allow the collection between the Unix machine and the Windows machine via a firewall: 1. On the Windows machine, enter the following command to restrain the number of dynamic ports available for RPC: netsh int ipv4 set dynamicport tcp start=10000 num= Restart the Windows machine for the changes to be applied. 3. On the firewall, allow the TCP range , the TCP 135, and the TCP 445. Using Non-Admin Accounts For Remote Windows Log Source This chapter describes how to collect remote Windows Event Logs without using Windows domain administrator accounts. 36 Universal Collector User Guide

37 CHAPTER 3 Collect Logs To configure non-admin domain accounts, you must have access to Windows Event Logs. Note: If you are using Windows domain administrator account in your log source files, you can skip this chapter. Access To Windows Event Logs For Non-Administrators Domain Account LogLogic uses the Microsoft Standard Windows Event Logs and DCOM Interfaces to collect remotely the event logs. Depending on Microsoft server configuration normal, domain users have no access. 1. Make sure that the user cannot change the password at next logon and that the password never expires. 2. Configure the DCOM Connection by running on the Log Source DCOMCNFG.exe. 3. In the Component Services main screen, go to Component Services > Computers. 4. Right click on My Computer and select Properties. The My Computer Properties screen is displayed. 5. Click on the Com Security tab and click on the button Edit Default in Launch and Activation Permissions. 6. Add your service user and grant all permissions: Local and Remote Launch and Local and Remote Activation. Universal Collector User Guide 37

38 About the Log Sources Figure 3 Launch and Activation Permission 7. Start ServerManager.msc to configure Windows Event Logs Remote Access. 8. Go to Configuration > WMI Control > open the WMI Control Properties. 9. Select the Security tab. 38 Universal Collector User Guide

39 CHAPTER 3 Collect Logs Figure 4 Windows Event Logs Control Properties 10. Select ROOT/CIMV2 and press the Security button to configure the settings. The Service User needs the access rights as in the screenshot below. Universal Collector User Guide 39

40 About the Log Sources Figure 5 Windows Event Logs Control Properties 11. Make sure that This namespace and subnamespaces is selected in the list box. Note: When configuring a new Windows Log Source, only the newly Windows Event Logs are collected, not all the events included in the current journals. Access To Windows Event Logs For Non-Administrators Domain Account Non-Administrators domain accounts are not allowed to access Windows Event Logs. This procedure describes how to modify channel access to allow a specific user to access a specific Windows event log (for Windows 7, 2008 and 2008 R2). For windows 2003 server or distribution via group policy, read the following article: 1. Enter the following command to obtain the user s SID: wmic useraccount where name="useraccountname" The SID has the following format: "S " and will be used in step Universal Collector User Guide

41 CHAPTER 3 Collect Logs 2. Open the command prompt and run the following command to dump out the SDDL for the system log. wevtutil gl system 3. Copy out the channelaccess: entry. channelaccess:o:bag:syd:(a;;0xf0007;;;sy)(a;;0x7;;;ba)...(a;;0x1;;;iu) 4. Remove "channelaccess:" at the beginning and add "(A;;0x1;;;<SID>)" at the end. channelaccess:o:bag:syd:(a;;0xf0007;;;sy)(a;;0x7;;;ba)...(a;;0x1;;;iu) channelaccess:o:bag:syd:(a;;0xf0007;;;sy)(a;;0x7;;;ba)... (A;;0x1;;;IU)(A;;0x1;;;<SID>) This will add an entry in the ACL in order to: authorize (i.e. A) read access (i.e. 0x1) for user (< SID>) 5. Apply the ACL with the following commands: wevtutil sl system /ca: O:BAG:SYD:(A;;0xf0007;;;SY)(A;;0x7;;;BA)... (A;;0x1;;;IU)(A;;0x1;;;<SID>) 6. For each Windows Event Log, repeat step 2 to 5. The ACL is different for each Windows Event Log, so the wevtutil command has to be run each time. e.g. 1 - Security Windows Event Log: wevtutil gl security wevtutil sl security /ca: O:BAG:SYD: (...) (A;;0x1;;;<SID>) e.g. 2 - Application Windows Event Log: wevtutil gl application wevtutil sl application /ca: O:BAG:SYD: (...) (A;;0x1;;;<SID>) e.g. 3 - Any Windows Event Log: wevtutil gl AnyWindowsEventLog wevtutil sl AnyWindowsEventLog /ca: O:BAG:SYD: (...) (A;;0x1;;;<SID>) Create a test connection via WMIC additional Windows Event Logs should be available, e.g. wmic:root\cli>/node: /user:lab\alice /password:admin123! ntevent log list brief FileSize LogfileName Name NumberOfRecords Directory Service C:\Windows\System32\Winevt\Logs\Directory Service.evtx Internet Explorer C:\Windows\System32\Winevt\Logs\Internet Explorer.evtx 0 Universal Collector User Guide 41

42 About the Log Sources Security C:\Windows\System32\Winevt\Logs\Security.evtx System C:\Windows\System32\Winevt\Logs\System.evtx 296 If Event Logs folders are still missing, create the CustomSD REG_SZ within each folder type. Step 2 - Editing Registry of Remote Windows Log Sources (RHEL, SuSE and Solaris only) You can collect up to 1000 MPS from up to 500 Windows hosts when installed on a RHEL, SuSE or Solaris Operating System. To do so, login to the target remote host as an Administrator. Local Security Settings 1. Start the control panel and go to Administrative Tools > Local Security Policy. This will open up the Local Security Settings screen. 2. Go to Local Policies > Security Options > Network access: Sharing and security model for local accounts and switch to Classic. This procedure can only be applied on Windows computers that are not part of a domain. Remote Registry Service The Remote Registry service must be running to allow the collection from a remote UC under RHEL, SuSE or Solaris. 1. Go to the control panel and open Administrative Tools > Services. 2. Locate the Remote Registry service on the list and start this service. 3. Change the startup type to Automatic. WBEM Scripting Locator 1. Run the program Regedit. If you are asked to allow the Regedit program to make changes to the computer, click Yes. 2. Navigate to the Registry item: HKEY_CLASSES_ROOT\CLSID\{76a64158-cb41-11d1-8b d9b6} 3. Right click on this item and select Permissions. 4. Click Advanced and select the Owner tab. 5. In the Change Owner to... box, highlight the account you are currently logged on and click OK. 6. Click OK again and right click the registry item. 7. Select Permissions and highlight the Administrators group. 8. Give Full Control permissions to this group by checking the Allow box and click OK. Caution:A UC installed on RHEL, SuSE or Solaris cannot collect events from a Windows machine installed on a Read Only Domain controller (RODC). 42 Universal Collector User Guide

43 CHAPTER 3 Collect Logs Step 3 - Verifying Connection To Remote Windows Log Sources To test the Windows Event Logs connection and results, you can use a small tool shipped with Windows. This is only available in Collector mode. 1. Open a command line interface and enter the following command: Wmic /node:%host% /user:%domain%\%serviceaccount% /password:%password% Nteventlog list brief Example of Input: Wmic /node: /user:lab\alice /password:admin123! Nteventlog list brief Example of Input: FileSize LogfileName Name Internet Explorer C:\Windows\System32\Winevt\Logs\Internet Explorer.evtx Security C:\Windows\System32\Winevt\Logs\Security.evtx If the error message access denied occurs, then something with the Windows Event Logs and DCOM configuration is wrong. 2. Test the connection with a domain admin account. Additionally, a reboot after the configuration steps might be useful. 3. Test the connection via the GUI. Filtering Windows Logs It may be required to minimize Windows Audit events generated by certain UC activities via one of the following methods: 1. Removal of Object Access/ Success from the audit policy on Windows log sources. (For further details, reference Audit Policy Management on Windows below.) 2. Review the current Security Access Control List (SACL) settings for the Windows Event Logs namespace \\root\cimv2, and verify that Enable Account/Successful is not checked for accounts/group to which the UC is connected. If necessary, create a new policy for the UC for which the Enable Account/Successful is not checked. Note: if necessary, inheritance of SACL may have to be disabled for that namespace. Universal Collector User Guide 43

44 About the Log Sources Table 8 Audit Policy Platforms Windows 2003 R2/ Windows 2008 Windows 2008 only Description The audit policy in windows is configured via local policies and/or GPO linked to domain/ou/site. A good way to understand the resulting policy is to use Resulting set of policy snap-in of MMC. Check the current resulting policy is set to generate results for local host only. The current resulting policy can be found under Computer Configuration > Windows Settings > Local Policies > Audit Policy. With windows 2008, more granular settings are possible, called sub-category. Whatever the solution used, you can check the precise auditing policy with: auditpol /get /category:* For more information on sub-category audit capabilities, please reference the following: Article on Windows Event Logs namespaces mentioning specifically Windows Event Logs auditing Syslog Logs Collecting Syslog Logs The Universal Collector reads logs sent via the Syslog protocol. The protocol through which syslog logs will be collected can be TCP or UDP. Note that if you want to use both protocols, you must define two Log Sources. Table 9 Protocols Name UDP TCP Description Default configuration. It specifies that the syslog logs should be collected via UDP protocol. When modifying the UC s status (such as updating or stopping it) or when the UC is not running during the collection, messages may be lost. Indeed, contrary to the TCP protocol, the UDP protocol avoids the overhead of checking whether every packet actually arrived, which may lead to data loss Specify that the syslog logs should be collected via TCP protocol. If another Syslog log is running on the server where the UC is installed, the UC and syslog will not have the same port, IP and protocols. In that case, you must either stop the syslog or make the UC listen on another port for example Filtering Syslog Logs Before being forwarded, the Syslog logs can be filtered according to their severity and facility. 44 Universal Collector User Guide

45 CHAPTER 3 Collect Logs facility: type of message that must be collected. Refer to the following URL for further information about the BSD syslog Protocol: severity: levels of severity that will be reported. See RFC If a message has neither severity nor facility, UC automatically allocates the local use 7 facility and the debug severity to the message. It will then be automatically filtered. Notes for Red Hat and SuSE Linux Enterprise If you obtain a log message saying Syslog Unable to set the required socket buffer size, then it is recommended to increase the maximum size of the buffer on your RHEL, SuSE, Solaris system. Why? Under RHEL, SuSE or Solaris, the default maximum TCP/UDP buffer size is 128 KB. In the UC configuration file, the default value of the buffer socket size is 1MB. These parameters apply to all the Syslog Log Sources related to UC. Therefore, you must increase the maximum value of the Syslog buffer already set with a specific command. How? By changing the maximum value of this buffer: 1. Log in as root on the system. 2. Enter the following command (example with 1 Megabyte): sysctl -w net.core.rmem_max= (this value is expressed in bit) Note: The modification of the system parameter will impact the maximum limitations for all the sockets. Remote Files UC is able to collect files remotely and forward them to LMI. By default UC polls every 1 hour but it can also pull every X minutes, every X hour, daily at X time, weekly on Y day on X time. Caution:It is highly recommended to use a physical machine for remote file collection. It is not recommended to use the UC's remote file collection to collect large remote files (above 1GB) on Virtual Machine systems as it will slow down the system significantly. Remote File with Rotation Log file rotation is not managed by UC; it is the user's responsibility to manage the file rotation. Universal Collector User Guide 45

46 About the Log Sources In the case of log file rotation, a log file is retired and renamed to a rotated name, and the monitored file is replaced by a new log file. Therefore, periodically during the monitoring of a log file that is rotated, the file is replaced by a fresh log file. When the date field is checked for rotation, UC will only collect files that are modified after the remote file log source creation time. Depending on your log rotation tool, UC is able to manage rotation files in two different ways. Refer to section Log File Rotation for more information. File with No Rotation Single Files Make sure that you gave the correct file path on the remote file system to pull the file correctly. Directory Directory pull allows users to choose a directory and pull files from that directory based on the include or exclude options provided to the user. Directory pull does not support file rotation. Example: /loglogic/ directory has three files: a.txt, b.txt, c.txt Scenario 1: if users put * for include, it will pull a.txt, b.txt, c.txt Scenario 2: if users put *.txt for include and put a.txt for exclude, it will pull b.txt and c.txt Scenario 3: if users put a.txt for include and nothing for exclude, it will only pull a.txt. UC Internal Logs The Universal Collector generates its own logs when it is itself subjected to changes or errors (e.g. starting of the UC, creation of a Log Source, disconnection of the UC, etc.). These internal logs are also sent to the LMI and can be used to repair or troubleshoot the UC. Collecting UC Internal Logs The UC internal logs are automatically generated in the uc.log file, which is located in the UC installation folder in \LogLogic\UniversalCollector\data\logs (for Windows). The uc.log is forwarded to the LMI provided you correctly configured the forwarding process (LMI connection). The LMI connection used to forward the UC internal logs can be the same as any log source LMI connection. 46 Universal Collector User Guide

47 CHAPTER 3 Collect Logs Create and Configure Log Sources Create a Log Source Add a New Log Source The user can add a Log Source from scratch. To create a new Log Source: 1. Open the UC Console by clicking on the shortcut and click on the Collection tab. 2. Click on the New button and select the type of Log Sources you want to add, either Real Time File, Syslog, Windows Event Log or Remote Files. Note: if you do not have an LMI connection when creating a remote file yet, please refer to the Forward Logs section in this documentation. 3. In the Edition screen, enter the relevant information as explained in the Edition section of this documentation. 4. Save the Log Source. It is added in the list of Log Sources. Copy a Log Source The user can copy one or several Log Source configurations. To copy one or several Log Sources: 1. Open the UC Console by clicking on the shortcut. 2. Select one or several Log Sources (Ctrl + click to select more than one Log Source). 3. Click Copy and confirm. The new log source(s) is/are displayed below the list of log sources. You can edit and modify them as any other log sources. Note: By default the log source configurations are not enabled. Delete a Log Source To delete a Log Source: 1. Select one or several Log Sources (Ctrl + click to select more than one Log Source). 2. Click Delete. The list is automatically refreshed. Create Several Log Sources at a Time The user can import and create several Log Sources of the same type at a time. To do so, a CSV file with Log Source information must be available. Universal Collector User Guide 47

48 Create and Configure Log Sources To create a CSV File: 1. Open a program, e.g. Notepad. 2. In the header, on the first line, enter the following field names according to the type of Log Source you want to create: Table 10 CSV fields Log Sources File Syslog Windows Fields name, description, lmi_connection*, enabled, timeinutc, message_filter, match_filter, file_path*, usedaterolling, date_pattern, useidrolling, nbdigit, multiline_active,multiline_header_type,multiline header type, multiline_custom_regex, multiline_orphaned_lines, multiline_linetimeout, appname*, hostname*, maxlinelength, charset name, description, lmi_connection*, enabled, timeinutc, protocol, ip, port, severity, facilities, source_ip name, description, lmi_connection*, enabled, timeinutc, event_id_filter, filter_operator, source_filter, address*, domain, login, password, include_eventlogs, eventlogs_list, polling_period 48 Universal Collector User Guide

49 CHAPTER 3 Collect Logs Table 10 CSV fields Log Sources Fields Remote File name,description,enabled,lmi_connection,ip,protocol,time_zone,file_system_t ype,user_id,password,domain,share_name,path_type,path,original_name,incl ude,exclude,usedaterolling,date_pattern,useidrolling,nbdigit,useucip,uc_ip, every_minutes,every_hours,daily_at_time,weekly_at_time,weekly_at_day,de vice_type * mandatory fields Note1: LMI connection is mandatory only if there is more than one existing connections available. The sole connection will be taken by default. Note2: name is not mandatory as a name will be automatically created, such as Real Time File #n or Windows Event Log #n or Syslog #n. 3. On the lines below, fill in the fields with the correct values and save in CSV format. The CSV file must look like this light example: Table 11 CSV Example name,description,lmi_connection,timeinutc Log Source A, Windows Log Sources, LMI_Connection,true Note: A detailed example of the fields and values to enter in the CSV file is available from UC Console when importing the CSV file. To import and create Log Sources: 1. Open the UC Console by clicking on the shortcut. 2. In the Collection tab, click on New > Batch import. The Batch Import tab is displayed. 3. In the drop-down list, select the type of Log Sources you are going to import. 4. Browse the CSV file and click OK. 5. Click Import. The Log Sources are created under the Collection tab, e.g. Import #1 - LS #1 Create a Complete Configuration A configuration contains general settings, a list of Log Sources and one or several Forwarding connections. All of these items are configured via the Graphical User Interface and are stored in a UC Configuration file (*.ucc) that you can unzip to explore the content. Edit Configuration General Settings The user can edit the general settings of a configuration. Universal Collector User Guide 49

50 Create and Configure Log Sources To edit configuration general settings: 1. Open the UC Console by clicking on the shortcut. 2. Click on. 3. In the General Settings screen, modify the following information: Table 12 General Settings Edition Option Name Communication Port TCP/UDP socket buffer size UDP max packet size Description Name of the configuration Port used by the UC to get information (e.g. status, metrics, memory used...) via the CLI. Make sure this port is not already used. Otherwise the UC cannot work. TCP/UDP parameter and socket buffer size (in kilobytes) - this parameter applies to all the Syslog Log Sources associated to the UC UDP parameter and max packet size (in kilobytes) - this parameter applies to all the Syslog Log Sources associated to the UC 4. Click Save. the configuration has been updated. Add a New Configuration The user can create a blank configuration from scratch on the local UC. To create a new configuration: 1. Open the UC Console by clicking on the shortcut. 2. Go to Manage Configuration > New In the Browsing windows, select a folder where you will store your configuration. 4. Enter a configuration name - with a *.ucc extension - in the Filename field and click Save. The new configuration is automatically displayed in the UC console but is not active yet. Open a Stored Configuration The user can edit at any time an existing or stored configuration other than the one running on the local UC. To open a stored configuration: 1. Open the UC Console by clicking on the shortcut. 2. Go to Manage Configuration > Open... button and browse the UC configuration file (*.ucc). 3. Click OK. The configuration is displayed in the GUI. However this configuration is neither applied nor running. 50 Universal Collector User Guide

51 CHAPTER 3 Collect Logs Note: You can display back your active configuration at any time by selecting Manage Configuration > Open Active Configuration in the drop-down menu. Make a Configuration Active The user can make a stored configuration active at anytime. Then, all the modifications applied on the fresh active configuration will be automatically saved and updated each time the user validates the changes. To make a configuration active: 1. Open the UC Console by clicking on the shortcut. 2. Display the configuration you want to be active in the UC Console. 3. Click on. 4. A warning message is displayed, which indicates that the active configuration will be overwritten if you accept the activation of the configuration. Click Yes to accept. 5. If you do not want the active configuration to be erased, click No and make a copy of it before activating another configuration. The configuration is now active and can be modified. Save a Configuration The user can save an active or stored configuration on the local system. To manually save a configuration: 1. Open the UC Console by clicking on the shortcut. 2. Modify a configuration and go to Manage Configuration > Save as In the Browsing window, select the folder where you want to save the configuration. You can create a new folder by clicking on New Folder. 4. Name the configuration and click OK. A UC Configuration file with the *.ucc extension is created. Edit Log Sources Edit a Real-Time File Log Source To edit a Real-Time File Log Source: 1. Under the Collection tab, double click on the selected Log Source or just select it and click on the Edit button. The RT File Edition tab is displayed. Universal Collector User Guide 51

52 Edit Log Sources 2. In the General part of the screen, you can modify the following information: Table 13 Real-Time File edition parameters - General Option Log Source Enabled Name Description Description Click on ON or OFF to define whether the current Log Source is enabled or disabled. Name of the Log Source. Description of the Log Source. 3. In the Forwarding Connection part of the screen, you can modify the following information: Table 14 Real-Time File edition parameters - Forwarding Connection Option Name UC Collection date Description Select the Forwarding connection to which you want to forward collected RT File logs. A Log Source must be linked to an existing Forwarding connection, which can be edited under the Forwarding tab. Define whether the log message sent to the LMI server remains in a local system time zone or is converted into UTC time zone. 4. In the Message Filtering part of the screen, you can modify the following information: Table 15 Real-Time File edition parameters - Message Filtering Option [Filtering] Collect messages Filter Description Click on ON or OFF to activate or deactivate the option. Define whether you collect messages that: - match the regex (other logs are filtered) - do not match the regex (i.e. filter the logs that match the regex) Enter a case insensitive regular expression to specify the messages to be matched. E.g. if Mot matching regex is selected : "packet accepted" means that all the lines containing packet accepted are filtered. "^64\.242" means that all the lines that are beginning exactly with are filtered "846$" means that all the lines that are ending exactly with 846 are filtered. E.g. if Matching regex is selected : "packet accepted" means that only the lines containing packet accepted are kept. "^64\.242" means that only the lines that are beginning exactly with are kept. "846$" means that only the lines that are ending exactly with 846 are kept. 52 Universal Collector User Guide

53 CHAPTER 3 Collect Logs 5. In the Collection part of the screen, you can modify the following information: Table 16 Real-Time File edition parameters - Collection Option File Path [File rotation] Date pattern Max number of digits [Multiline messages] Multiline header type Custom header regex Send orphaned lines Multiline timeout after detected headers (ms) [Advanced parameters] Host name Application name Maximum messages length [Collected file] Character set Description Browse the log file to be collected. If the log file is rotated, you may enter [id] or [date] or both in the filename as well as configuring the File rotation parameters. E.g. c:\temp\logfile[date].log to obtain file names such as logfile log E.g. c:\temp\logfile[id].log to obtain file names such as logfile1.log Click on ON or OFF to activate or deactivate the option Enter the date format you want to use for the [date] parameter. E.g. yyyymmdd for (see docs/api/java/text/simpledateformat.html ) Check the box and indicate the maximum number of digits you want for the [id] parameter. UC can collect any file with an [id] whose number of digits is between 1 and 9 inclusive. E.g. If you set 5, the following [id] will be taken into account: 1, 054, 586, 00599, 78945, etc. Click on ON or OFF to activate or deactivate the option to define whether the single message has several lines Select the type of multi-line logs. E.g. 'jboss', 'tomcat', 'weblogic', 'websphere' or 'custom'. Set a regular expression matching the header of the first line of a log. Indicate whether you want the UC to send messages that do not match the Header Regexp. Indicate the number of milli-seconds after which the multi-line logs are ready to be sent. Click on the arrow to display advanced parameters. Enter the name of the host used to pair logs on the LMI server. E.g. customhostname.com If you enter an IP address, the device to be displayed in LMI will be referred with this IP address. Enter the name of the application used to identify logs on the LMI server. E.g. customapplicationname Indicate the possible maximum length for the message (in bytes). Default value: Select the data format. Default value: Use local system charset 6. Click Apply to validate the changes. Universal Collector User Guide 53

54 Edit Log Sources Edit Several Real Time File Log Sources To edit several Real-Time File Log Sources: 1. Under the Collection tab, select the Log Sources and click on the Edit button. The RT File Edition tab is displayed. Figure 6 RT File Edition tab 2. Check the boxes in front of the set of RT File parameters you want to change. 3. Modify the parameters as described in "Edit a Real-Time File Log Source". 54 Universal Collector User Guide

55 CHAPTER 3 Collect Logs Edit a Windows Event Log Source To edit a Windows Event Log Source: 1. Under the Collection tab, double click on the selected Log Source or just select it and click on the Edit button. The Windows Events Log Source Edition tab is displayed. 2. In the General part of the screen, you can modify the following information: Table 17 Windows Event Log edition parameters - General Option Log Source Enabled Name Description Description Click on the button to define whether the current Log Source is enabled or disabled. Enter the name of the Log Source. E.g. ls-win-template Enter the description of the Log Source 3. In the Forwarding Connection part of the screen, you can modify the following information: Table 18 Windows Event Log edition parameters - Forwarding Connection Option Name UC Collection date Description Select the Forwarding connection to which you want to forward collected events log. A Log Source must be linked to an existing Forwarding connection. This can be done under the Forwarding tab. Select the option to define whether the log message sent to the LMI server remains in a local system time zone or it is converted into UTC time zone. 4. In the Message Filtering part of the screen, you can modify the following information: Table 19 Windows Event Log edition parameters - Message Filtering Option [Filtering] Event ID Filter Description Click on ON or OFF to activate or deactivate the option. Regular expression to filter the Windows event ID. E.g. 567 ^58[1-9] means that the events with an Event ID containing 567 but also those from 581 to 589 inclusive are collected. ^(8.*) ^(5[2-9].*) means that the events with an ID starting with 8 but also those starting with 52 to 59 inclusive are filtered. If the field is empty or.* is set means that no filter is set. Refer to the section "Regular Expressions" in the Appendix to get the list of characters used in regular expressions. Universal Collector User Guide 55

56 Edit Log Sources Table 19 Windows Event Log edition parameters - Message Filtering Option and/or Source Filter Description Select if you want to use both filters at the same time or one or another Enter a regular expression to filter Windows events on source field. E.g.: Security means that all the events with a Security source field are filtered. DNS Client Events means that all the events with a DNS Client Events source field are filtered. Time-Service means that all the events with a time-service source field are filtered. If the field is empty or.* is set means that no filter is set. Refer to the section "Regular Expressions" in the Appendix to get the list of characters used in regular expressions. 5. In the Collection part of the screen, you can modify the following information: Table 20 Windows Event Log edition parameters - Collection Option [Location] Local/Remote host Host name [Credentials] Use UC service credentials/use custom credentials Domain (if Use custom credentials is set) Login (if Use custom credentials is set) Password (if Use custom credentials is set) [Windows Event Logs] Collect List Description Indicate whether the Windows host from which to poll logs is the local machine or a remote host. Enter the IP address to connect to the remote Windows server. Select the relevant options to use the correct Windows credentials. Enter the domain name to access the Windows server E.g. domain.company Enter the login to connect to the Windows server. Default value: jdoe To connect to the Windows server, enter a password. Define the Windows Event Logs journals to include. It can be either: - all event logs = all current and logs to come are collected - all event logs except the following ones = all current and event logs to come are collected except the one indicated in the List form. - only the following event logs = only the following event logs indicated in the List form are collected List of Event Logs to include or exclude. 56 Universal Collector User Guide

57 CHAPTER 3 Collect Logs Table 20 Windows Event Log edition parameters - Collection Option Button Edit List [Advanced] Polling Period Description Displays the Edit List window to select the event logs to be collected: 1 - In the Available Event Logs pane, select an event log and click on Add. This will add the logs to the list. 2 - If you want to remove them from the list, select them and click on Remove. 3 - If you want to manually add an Event Log, enter the name and click Add. Make sure you entered the name correctly as it is case-sensitive. 4 - Click OK. Note: if you want to display all the Event Logs available, click on the Discover Event Logs button. Enter the time period (in seconds) after which the UC checks for new Windows events. Default value: Click Apply to validate the changes. Universal Collector User Guide 57

58 Edit Log Sources Edit Several Windows Event Log Sources To edit several Windows Event Log Source: 1. Under the Collection tab, select the Log Sources and click on the Edit button. The Windows Event Log Edition tab is displayed. Figure 7 Windows Event Log Edition tab 2. Check the boxes in front of the set of Windows Event Logs parameters you want to change. 3. Modify the parameters as described in "Edit a Windows Event Log Source". Edit a Syslog Log Source To edit a Syslog Log Source: 1. Under the Collection tab, double click on the selected Log Source or select it and click on the Edit button. The Syslog Log Source Edition tab is displayed. 58 Universal Collector User Guide

59 CHAPTER 3 Collect Logs 2. In the General part of the screen, you can modify the following information: Table 21 Syslog edition parameters - Description Option Log Source Enabled Name Description Description Click on the button to define whether the current Log Source is enabled or disabled. Name of the Log Source Description of the Log Source 3. In the Forwarding Connection part of the screen, you can modify the following information: Table 22 Syslog edition parameters - Forwarding Connection Option Name UC Collection date Description Select the Forwarding connection to which you want to forward collected logs. A Log Source must be linked to an existing Forwarding connection, which can be edited under the Forwarding tab. Select the option to define whether the log message sent to the LMI server remains in a local system time zone or it is converted into UTC time zone. 4. In the Collection part of the screen, you can modify the following information: Table 23 Syslog edition parameters - Collection Option Protocol Port Binding interface Description Define whether the Log Source uses the udp/tcp SYSLOG protocol. In order to listen on both UDP and TCP protocols, you must create two Syslog Log Sources. Enter the port to listen to the Syslog flow. Default value: 514 If there are multiple network interfaces, enter the IP address to listen to the Syslog flow. Only one IP address is possible. To listen to all network interfaces, use To listen to a specific interface, use an address like Default value: Universal Collector User Guide 59

60 Edit Log Sources 5. In the Message Filtering part of the screen, you can modify the following information: Table 24 Syslog edition parameters - Message Filtering Option [Filtering] Maximum Severity Description Click on ON or OFF to activate or deactivate the option. If Message Filtering is set on OFF, messages with a debug severity are not collected (max severity set to 6). If a message has neither severity nor facility, UC automatically allocates the local use 7 facility and the debug severity to the message. It will then be automatically filtered. Select the maximum accepted severity (numerical code, see RFC 3164) 0 - Emergency: system is unusable 1 - Alert: action must be taken immediately 2 - Critical: critical conditions 3 - Error: error conditions 4 - Warning: warning conditions 5 - Notice: normal but significant condition 6 - Informational: informational messages 7 - Debug: debug-level messages Default value: 6 - Informational: informational messages 60 Universal Collector User Guide

61 CHAPTER 3 Collect Logs Table 24 Syslog edition parameters - Message Filtering Option Authorized facilities Authorized IP addresses Description Select one or several accepted facilities (see RFC 3164). The logs with these facilitities are kept. 0 - kernel messages 1 - user-level messages 2 - mail system 3 - system daemons 4 - security/authorization messages (note 1) 5 - messages generated internally by syslogd 6 - line printer subsystem 7 - network news subsystem 8 - UUCP subsystem 9 - clock daemon (note 2) 10 - security/authorization messages (note 1) 11 - FTP daemon 12 - NTP subsystem 13 - log audit (note 1) 14 - log alert (note 1) 15 - clock daemon (note 2) 16 - local use 0 (local0) 17 - local use 1 (local1) 18 - local use 2 (local2) 19 - local use 3 (local3) 20 - local use 4 (local4) 21 - local use 5 (local5) 22 - local use 6 (local6) 23 - local use 7 (local7) Default value: 0-23 Enter the regular expression to filter the accepted IP addresses and to filter the accepted host. All the logs from all IP addresses are collected if the field is blank (default). Universal Collector User Guide 61

62 Edit Log Sources Edit Several Syslog Log Sources To edit several Syslog Log Sources: 1. Under the Collection tab, double click on the selected Log Source or select it and click on the Edit button. The Syslog Log Source Edition tab is displayed. Figure 8 Syslog Edition tab 2. Check the boxes in front of the set of Syslog parameters you want to change. 3. Modify the parameters as described in "Edit a Syslog Log Source". 62 Universal Collector User Guide

63 CHAPTER 3 Collect Logs Edit a Remote File Log Source To edit a Remote File Log Source: 1. Under the Collection tab, double click on the selected Log Source or select it and click on the Edit button. The Remote File Log Source Edition tab is displayed. 2. In the General part of the screen, you can modify the following information: Table 25 Remote File edition parameters - Description Option Log Source Enabled Name Description Description Click on the button to define whether the current Log Source is enabled or disabled. Name of the Log Source. Description of the Log Source. 3. In the Forwarding Connection part of the screen, you can modify the following information: Table 26 Option Name Remote File edition parameters - Forwarding Connection Description Select the connection to which you want to forward collected logs. A Log Source must be linked to an existing Forwarding connection, which can be edited under the Forwarding tab. Note: Remote File Collection is only supported by LMI v5.4 or above and can only be forwarded to LMI, not generic syslog servers. 4. In the Collection part of the screen, you can modify the following information: Table 27 Remote File edition parameters - Collection Option Host IP/Name Protocol [If ftp is selected] TimeZone [If a non-local timezone is selected] File System Type [If cifs is selected] Domain/User name [If cifs is selected] User password [If cifs is selected] Share name User ID User password Description Enter the IP or name of the remote log source. Define whether the Log Source uses the ftp, sftp or cifs protocol. Select the timezone of the remote log source. Select the file system type. Enter the domain or user name. Enter the user password. Enter the cifs share name. Enter the User ID to connect to the remote log source. Enter the user password. Universal Collector User Guide 63

64 Edit Log Sources Table 27 Remote File edition parameters - Collection Option File / Directory File path [File rotation] [If File is selected] Original name [If File is selected] Date pattern [If File is selected] Max number of digits [If Directory is selected]directory path [If Directory is selected]file(s) Include [If Directory is selected]file(s) Exclude Device type Test connection [Advanced] Log Source IP Schedule Description Select the source of the collection, either a file or the content of a directory. If File is selected, enter the file path. Click on ON or OFF to activate or deactivate the option. Only available if File is selected. The file that is currently being written; it is usually the file without date or id tag. Enter the date format you want to use for the [date] parameter. E.g. yyyymmdd for (see docs/api/java/text/simpledateformat.html ) Check the box and indicate the maximum number of digits you want for the [id] parameter. UC can collect any file with an [id] whose number of digits is between 1 and 9 inclusive. E.g. If you set 5, the following [id] will be taken into account: 1, 054, 586, 00599, 78945, etc. If Directory is selected, enter the directory pathname. Enter the files that must be included in the collection. The field supports the standard common wildcard characters for matching file names (* and?). Enter the files that must be excluded from the collection. The field supports the standard common wildcard characters for matching file names (* and?). Select the type of logs to be collected. Click on this button to check if the connection to the remote log source is working. Select an option: Remote file server: selected by default. The IP is grabbed from the host IP that you previoulsy entered. UC: IP address of the workstation where UC is installed. You can change it as you want. Select the collection period, either per minute, hour, daily or weekly at a specific hour. Edit Different Log Sources at a Time You can edit several Log Sources of different types - except remote files - at a time. Only the common parameters will be editable. To edit all Log Sources at a Time: 1. Under the Collection tab, press Ctrl while clicking on the Log Sources to select them. 2. Click on Select screen to only select the list of visible Log Sources at a time or click on Select all to select all the lists of Log Sources. 64 Universal Collector User Guide

65 CHAPTER 3 Collect Logs 3. Click on the Edit button and select All. The All tab is displayed. Figure 9 All tab 4. In the General part of the screen, modify the following information: Table 28 All parameters - General Option Log Source Enabled Name Description Description Click on the button to define whether the current Log Source is enabled or disabled. Name of the Log Source Description of the Log Source 5. In the Forwarding Connection part of the screen, modify the following information: Table 29 All parameters - Forwarding Connection Option Name UC Collection date Description Select the Forwarding connection towards which you want to forward the collected logs. Select the option to define whether the log message sent to the LMI server remains in a local system time zone or it is converted into UTC time zone. 6. Click OK to save the changes. If you open again one of the Log Source you selected, you can see that the changes are applied. Universal Collector User Guide 65

66 Sorting Log Sources Sorting Log Sources Create Tags Tags are useful to store, sort and search for Log Sources among a list. For example, you want to easily find the logs coming from Windows server A to which the administrator has logged. You can create the tags: Server A, Connection, Administrator. You can create and apply up to 10 filters. To create a new tag: 1. Under the Collection tab, select one or several log sources. 2. In the Tag edition panel on the right, enter a tag in the combo box and click Add Tag. The tag is automatically saved. To apply a tag: Once you have created tags, you can apply them to one or several log sources. 1. Under the Collection tab, select the log source(s) to which you want to apply a tag. 2. In the combo box in the right hand panel, select the tag you want to apply and click Add Tag. The tag is displayed under the Tags column. To remove a tag: 1. Under the Collection tab, select the log source for which you want to remove the tag. 2. In the Tag edition panel, click on the cross of the tag you want to remove. It automatically updates the list. Sort Log Sources You can sort the list of log sources to display only the relevant items. To sort log sources: 1. In the left hand part of the configuration panel, click on the + Add Filter button. Two drop-down list boxes are displayed. 2. In the first drop down list, select the type of information you want to filter, either by Enabled, Name, Forwarder, Type, Collection or Tags. 66 Universal Collector User Guide

67 CHAPTER 3 Collect Logs 3. Then select the relevant values per type of information: Table 30 Filters Filter Enabled Name Forwarder Type Collection Tags Values Sorts log sources per status, i.e. off or on. Sorts log sources per name. Enter the log source name. E.g. ls-logsource-windows Sorts log sources per Forwarding connection (names of the connection file), e.g. uldp-sample Sorts log sources per type, i.e. file, syslog or windows. Sorts log sources per collection type, i.e. file, syslog or windows. Sorts log sources per user-created tags, e.g. server, web Click Apply to filter the list. 5. To add another filter, click on +Add Filter and repeat the procedure explained above. For example, to make a search on a specific forwarder AND a specific type of file, you will obtain something like this: Figure 10 Two filters are applied 6. If, for a same filter, you want to add another value, click on the + button and select the relevant value. For example, to find a File Log Source OR a Syslog log source, you will have to obtain something like this: Universal Collector User Guide 67

68 Sorting Log Sources Figure 11 Two values are applied 7. To remove a filter or only a value, click on the - button. 8. Click on a header (Name, Status...) to display the filtered list by alphabetical order. 9. Click on the Clear all button to disable the filters. 68 Universal Collector User Guide

69 CHAPTER 4 Forward Logs Contents Create a TCP (Syslog) or UDP (Syslog) Connection Create an LMI Connection Create a Connection in Authentication and/or Encryption Mode Manage the list of Forwardings The Universal Collector collects the information from various types of log sources and forwards them to an LMI server via the proprietary protocol (Universal Log Data Protocol) or a syslog server via the proprietary protocol (User Datagram Protocol, or Transmission Control Protocol) for the communication between the UC and the LMI server or syslog server. A file is identified by a file identifier (usually a string representing the path name of the file in the source device). Create a TCP (Syslog) or UDP (Syslog) Connection The user can add up to 10 Forwarding Connections. To create a Syslog TCP or UDP connection (no encryption): 1. Open the UC Console and click on the Forwarding tab. 2. Click on New > TCP (Syslog) or UDP (Syslog) menu. 3. In the General part of the screen, modify the name of the connection. 4. In the Security part of the screen, make sure the button is on OFF. 5. In the Forwarding part of the screen, modify the following values: Forwarding Address Enter the IPv4 address or host name of the TCP /UDP server. Port Enter a port. (Default: 514) [TCP Only] Test Test the connection between UC and the server. Connection Message Format Universal Collector User Guide 69

70 Create a TCP (Syslog) or UDP (Syslog) Connection Facility Severity Custom Header Advanced [TCP only] Session timeout UC Binding interface Select the facility to be applied to the log: 0 - kernel messages 1 - user-level messages 2 - mail system 3 - system daemons 4 - security/authorization messages (note 1) 5 - messages generated internally by syslogd 6 - line printer subsystem 7 - network news subsystem 8 - UUCP subsystem 9 - clock daemon (note 2) 10 - security/authorization messages (note 1) 11 - FTP daemon 12 - NTP subsystem 13 - log audit (note 1) 14 - log alert (note 1) 15 - clock daemon (note 2) 16 - local use 0 (local0) 17 - local use 1 (local1) 18 - local use 2 (local2) 19 - local use 3 (local3) 20 - local use 4 (local4) 21 - local use 5 (local5) 22 - local use 6 (local6) 23 - local use 7 (local7) Select the severity to be applied to the log: 0 - Emergency: system is unusable 1 - Alert: action must be taken immediately. 2 - Critical: critical conditions. 3- Error: error conditions. 4 - Warning: warning conditions. 5 - Notice: normal but significant condition. 6 - Informational: informational messages. 7 - Debug: debug-level messages. Indicate the header of the message. Enter the session timeout (in seconds) If there are multiple network interfaces, enter the IP address that the UC uses when establishing the connection. Default: Universal Collector User Guide

71 CHAPTER 4 Forward Logs 6. In the Message Buffering part of the screen, modify the following values: Message Buffering Buffer size (MB) Enter the buffer size in megabytes (100 MB - default value) 7. Click OK to save and close the screen. The list of connections is updated. Create an LMI Connection To create a new LMI Connection (no encryption): 1. Open the UC Console and click on the Forwarding tab. 2. Click on New > ULDP to open the LMI Connection tab. 3. In the General part of the screen, modify the name of the LMI connection. 4. In the Security part of the screen, make sure the button is on OFF. 5. In the Forwarding part of the screen, modify the following values: Forwarding Address Enter the IPv4 address or host name of the LMI Port Select the LMI port or enter a port for connection with LMI 5.0 and 5.1 (default value) for secured connection with LMI 5.0 or later (configurable in LMI) for connection with LMI 5.2 or later [Button] Test connection Test the connection between UC and LMI. Forward UC Internal Logs Define whether the UC internal logs are sent to the remote LMI by selecting ON. Compress Messages If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed by selecting ON. Advanced Reconnection Session timeout UC Binding interface Enter the reconnection frequency to the LMI (in seconds) Enter the session timeout to LMI (in seconds) If there are multiple network interfaces, enter the IP address that the UC uses when establishing the connection to LMI. Default: In the Message Buffering part of the screen, modify the following values: Message Buffering Buffer size (MB) Enter the buffer size in megabytes (100 MB - default value) Universal Collector User Guide 71

72 Create a Connection in Authentication and/or Encryption Mode Scheduled Forwarding Daily Start Daily Stop Define the period of time during which the logs are sent to the LMI (time window) by selecting ON. Note: Schedule forwarding is not recommended for pulling large files via remote file collection. Define the beginning of the time window. If sendingwindow = true in the above parameter, define the time (hour and minute) when the event starts to be sent (default value = 23:00) Define the end of the time window. If you set sendingwindow = true in the above parameter, define the time (hour and minute) when the event stops to be sent (default value = 05:00). 7. Click OK to save and close the screen. The list of LMI connections is updated. Create a Connection in Authentication and/or Encryption Mode In this chapter, we will see how the information delivered through the communication between the UC and LMI server or syslog server can be encrypted. To secure communications between the UC and LMI or syslog servers, the following information will be checked: LMI or a syslog server and UC identities and encryption of communication between UC and LMI or a syslog server (public and private key mechanism). As a requirement, you will need a PKI and OpenSSL or another compatible tool. Note: This section of the documentation is intended for advanced users with the necessary encryption and secure communication skills. 72 Universal Collector User Guide

73 CHAPTER 4 Forward Logs Figure 12 Overview of the creation of a UC certificate 1) A public key and a private key are used to create a Root Certificate Authority (Root CA). 2) A public key and a private key are generated to create the UC s Certificate Signing Request (CSR). 3) This request will be sent along with the UC s identity information and the public key. 4) The Root CA delivers the certificate by signing the Certificate Signing Request. The UC s certificate is then created and sent with the Authority s certificate. Step 1 - Get a Root Certificate Authority from your PKI When deploying an authentication process with UC, you need to use a Public Key Infrastructure (PKI) consisting of a certificate authority or CA (and a registration authority or RA) that issues and verifies digital certificate. A certificate includes the public key; one or more directories where the certificates (with their public keys) are held and a certificate management system. Universal Collector User Guide 73

74 Create a Connection in Authentication and/or Encryption Mode Figure 13 Generation of a Root CA A number of products that enable a company or group of companies to implement a PKI exist. To do so, use a tool such as OPENSSL. 1. Generate a public and a private key. The recommended and maximum size is 2048 bit and encrypted in AES 128 (3DES is also supported). Example: openssl genrsa -out ca.key -aes Generate the CA (valid for 7305 days) Example: openssl req -new -x509 -days key ca.key -out ca.pem Please refer to the SSL Certificates HOWTO to know how to create your Certificate Authority: SSL-Certificates-HOWTO.html Step 2 - Create a Certificate Signing Request You must now generate a Certificate Signing Request in a UC to be able to create a Certificate on a Certificate Authority. You will obtain a file with the *.csr extension. 74 Universal Collector User Guide

75 CHAPTER 4 Forward Logs Creation of a CSR There are two possibilities: Via the Internal Tool The tool is located in <INSTALL_DIR>/tools. 1. Enter the following command to start the tool: cert_mgt.bat (under Windows) cert_mgt (under RHEL, SuSE, Solaris) 2. Enter the following argument: <script-name> request 3. Enter the command to indicate the file path of the file to be generated. You have three possibilities according to the type of your certificates. [ -jks <file path of the generated *.ks containing the private key> ] [ -p12 <file path of the generated *.p12 certificate containing the private key> ] [ -pem <file path of the generated *.pem private key> ] -csr <file path of the generated Certificate Signing Request> [ -dn <CSR Distinguished Name> ] -pwd <mandatory password for the file containing the private key> This command generates 2 files containing the private key (i.e. a *.ks or *.p12 or *.pem) and a Certificate Signing Request (CSR). If it is not specified in the command line, by default the DN of the CSR is: CN=<UC-IP>, O=loglogic Example: Universal Collector User Guide 75

76 Create a Connection in Authentication and/or Encryption Mode cert_mgt request -jks uc.ks -pwd loglogic -csr uc.csr Via OpenSSL To do so, you need the UC s public and private keys and OpenSSL. 1. Generate the public and private keys. The recommended and maximum size is 2048 bit and encrypted in AES 128 (3DES is also supported): openssl genrsa -out uc.key -aes Create the CSR like: openssl req -new -key uc.key -out uc.csr Please refer to the SSL Certificates HOWTO to know how to create your Certificate Signing Request: SSL-Certificates-HOWTO.html Step 3 - Create a valid UC certificate using a CA and OpenSSL You must now create the valid Certificate issued by a Certificate Authority in the UC configuration. Figure 14 Creation of a valid certificate To do so, enter the following command: openssl ca -config conf_file.txt -days 730 -in uc.csr -out uc.pem -notext Note: In this example, a file has been defined (conf_file.txt). If no configuration file has been specified, then OpenSSL takes /usr/local/ssl/openssl.cnf by default. You will obtain a *.pem certificate that contains the UC s certificate. 76 Universal Collector User Guide

77 CHAPTER 4 Forward Logs Step 4 - Import the Certificate into *.ks or *.p12 This procedure is not necessary if you work with a *.pem certificate. This command allows you to import the UC certificate and/or the root CA certificate in a *.ks or the UC certificate in a *.p12 certificate. To do so, you must use the CLI provided by LogLogic. Here is the command to format the file: <script-name> import [ -jks <file path of the *.ks> ] [ -p12 <file path of the *.p12 certificate> ] -pwd <mandatory password> [ -cert <file path of the UC certificate in *.pem format> ] [ -rootcert <file path of the root CA certificate in *.pem format> ] This command imports the UC certificate and/or the root CA. You can obtain a *.ks certificate file that contains a Certificate Authority, private key and the UC s certificate or a *.p12 certificate binary code, which contains the UC s certificate and a private key encrypted by a passphrase. Example: cert_mgt import -jks uc.ks -pwd loglogic -cert uc-cert.pem -rootcert ca.pem Step 5 - Configure the Forwarding Process If the connection is authenticated or encrypted, the necessary cryptographic elements must be imported. Three formats are supported: Figure 15 The three supported formats *.ks A keystore in the JKS format containing the root CA, the private key and the associated UC certificate. Universal Collector User Guide 77

78 Create a Connection in Authentication and/or Encryption Mode Associated configuration elements are a keystore filename and a password for the keystore (mandatory) *.p12 A keystore in the PKCS#12 format, containing the private key and the associated UC certificate and the root CA (in *.pem format) as a separate file. Associated configuration elements are a PKCS#12 (.p12) file, a password protected PKCS#12 file (mandatory) and a root CA file. *.pem A private key (encrypted or not), a certificate to be used by UC in PEM format, a root CA certificate in PEM format. Associated configuration elements are a private key file, a password if the private key is encrypted (mandatory), a UC certificate file, a root CA certificate file. The Certificate Authority s certificate allows to check the validity of the LMI or syslog server s certificate towards the UC The UC Valid certificate allows to identify the UC from the LMI or syslog server. Caution: The Certificate Authority must be the one you previously used to validate the LMI or syslog server certificate. To configure the forwarding process in the GUI: 1. Open the UC Console and click on the Forwarding tab. 2. Click on the New Connection button to open the Edition tab. 3. In the Description part of the screen, modify the name of the LMI or syslog server connection. 4. In the Security part of the screen, activate the following options: Table 31 Forwarding Connection edition - Security Values Authentication Encryption Certificate [Button] Initialize Secured Connection Description Activates the authenticated communication when the button is ON Activates the encrypted communication when the button is ON Displays the certificate imported in UC Displays the screens to import the certificates First case: PEM 1. In the Secured Connection Initialization screen, select PEM and click Continue. 2. In the UC Certificate section, click on Import and select the UC Certificate in *.pem format. 3. In the new small windows, click on Import Private Key and select the file in PEM format. 78 Universal Collector User Guide

79 CHAPTER 4 Forward Logs 4. Enter the private key password and click OK. 5. In the Root CA Certificate section, click on Import and select the root CA certificate stored in *.pem format. 6. Click OK to close the window. The screen is automatically updated. Second case: P12 1. In the Secured Connection Initialization screen, select PEM and click Continue. 2. In the UC Certificate section, click on Import and select the UC PKCS#12 Certificate in *.p12 format. 3. Enter the certificate password and click OK. 4. In the Root CA Certificate section, click on Import and select the root CA certificate stored in *.pem format. 5. Click OK to close the window. The screen is automatically updated. Third case: JKS 1. In the Secured Connection Initialization screen, select JKS and click Continue. 2. In the Java Keystore section, click on Import and select the UC JKS Certificate in *.jks format. 3. Enter the certificate password and click OK. 4. Click OK to close the window. The screen is automatically updated. Configure the Forwarding process 1. In the Forwarding part of the screen, modify the following values: Forwarding Address Port Enter the IPv4 address or host name of the LMI. Select LMI port or enter a port for connection with LMI 5.0 and 5.1 (default value) for secured connection with LMI 5.0 or later (configurable in LMI) for connection with LMI 5.2 or later [Button] Test connection Test the connection between UC and LMI. Forward UC Internal Logs Define whether the UC internal logs are sent to the remote LMI by selecting ON. Compress Messages If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed by selecting ON. Advanced Reconnection Enter the reconnection frequency to the LMI (in seconds) Universal Collector User Guide 79

80 Create a Connection in Authentication and/or Encryption Mode Session timeout UC Binding interface Enter the session timeout to LMI (in seconds) If there are multiple network interfaces, enter the IP address that the UC uses when establishing the connection to LMI. Default: In the Message Buffering part of the screen, modify the following values: Message Buffering Buffer size (MB) Scheduled Forwarding Daily Start Daily Stop Enter the buffer size in megabytes (100 MB - default value) Define the period of time during which the logs are sent to the LMI (time window) by selecting ON. Define the beginning of the time window. Define the time (hour and minute) when the event starts to be sent (default value = 22:00) Define the end of the time window. Define the time (hour and minute) when the event stops to be sent (default value = 05:00). 3. Click OK to save and close the screen. The list of LMI connections is updated. Note: The configuration of UC has finished. When the certificate has expired, you must follow the procedure from the very beginning. You can use the same CSR you used if you have stored it. Step 6 - Enable Secure Connection As for LMI, two certificates are needed: The Root CA, delivered by a certificate authority server. It will check the UC s identity. A certificate signing request or CSR. In order to generate the signed certificate, manual steps are required unlike UC. Figure 16 LMI Connection 80 Universal Collector User Guide

81 CHAPTER 4 Forward Logs 1. Under the LogLogic CLI, create a Certificate Signing Request: system secureuldp create csr This will generate a private key as well as the CSR. The CSR is the values between the Begin Certificate and End Certificate lines. Make sure to include the Begin Certificate and End Certificate lines when pasting it in. 2. If you have already created your CSR and just want to display it again, enter: system secureuldp show csr 3. Copy the CSR and sign the CSR. Once the CA signs the CSR, it will generate a signed certificate called LMI. 4. Install this signed certificate back to the LMI Appliance by entering: system secureuldp install certificate 5. Paste the certificate in. Make sure to include the Begin Certificate and End Certificate lines when pasting it in. 6. Install the root CA certificate which will be the common certificate used for validation between the LMI and UC. To do so, enter: system secureuldp install rootca 7. Paste it in the root CA certificate. 8. You may need to restart the application: mtask stop ; mtask start 9. Once you have created all the certificates, you must go to Administration > System Settings > General and check the Yes radio button in front of Enable Secure ULDP. The communication between UC and LMI is now secure. Manage the list of Forwardings You can easily copy or delete forwardings. Graphical User Interface Overview Table 32 Forwarding GUI Labels and buttons Name Address Port [ULDP only] UC Logs Description Label of the configuration IPv4 address or host name of the server. Forwarding port. Indicates whether the UC internal logs are sent to the remote LMI or not. Universal Collector User Guide 81

82 Manage the list of Forwardings Table 32 Forwarding GUI Labels and buttons [ULDP only] Comp. Auth. Encrypt. Buffer (MB) [ULDP only] Sched. Description Indicates whether the logs are compressed or not. Communication authenticated or not Communication encrypted or not Buffer size in megabytes (100 MB - default value, 50 GB - maximum value) Indicate if the messages are sending to the server during a specified time window [Button] New Allows you to add new Forwardings to the list (Max. 10) [Button] Edit Allows you to edit Forwardings one by one [Button] Copy Allows you to copy Forwardings to the list. [Button] Delete Allows you to delete Forwardings from the list. Copy a Forwarding The user can copy a Forwarding one by one. The copied Forwardings keep the same configuration and the same name with the _Copy suffix. To copy a Forwarding: 1. Select the Forwarding you want to copy. 2. Click Copy. The new Forwarding(s) is/are displayed below the list of Forwardings. You can edit and modify them as any other ones. Note: By default the Forwarding is linked with no Log Source. Delete a Forwarding The user can delete Forwardings one by one. To delete a Forwarding: 1. Make sure that the Log sources linked to the Forwarding are removed or disabled. 2. Select the Forwardings and click Delete. The list is automatically refreshed. 82 Universal Collector User Guide

83 CHAPTER 5 Monitor UC Activities Contents Start UCMon Tool Summary Screen Status Screen Metrics Screen Trends Screen RealTime Screen Start UCMon Tool This chapter provides simple instructions for quickly checking that UC is working properly, troubleshooting UC -> Forwarding connection configuration and monitoring the activities of the different log sources. To start UCMon from UC Console: Open the UC Console and go to Manage Configuration > Monitor Active Configuration. To start UCMon manually: Open the UC installation folder and launch the executable file located in the tools folder: uc_monitor.exe (Windows) also available by clicking on the uc_monitor shortcut uc_monitor (RHEL, SuSE or Solaris) The UCMon is displayed. Universal Collector User Guide 83

84 Summary Screen Summary Screen Table 33 UCMon - Summary screen Label Uptime Current Time Totals for the UC Collected Filtered To Buffer UC Mem Config Description Time when the UC has been started Current date and time are automatically refreshed Total number of collected message for a given period of time Between brackets, number of collected message per second Total number of filtered message for a given period of time Between brackets, number of filtered message per second Total number of forwarded message for a given period of time Between brackets, number of forwarded message per second Current memory used / Total memory (Java Heap Size) Current configuration name Forwarding Connections and Log Sources All Forwarding Forwarding connection status Conn. Active: the Forwarding connection works correctly All Log Sources/ Syslog/Windows Event Log/RT File/Remote File Interactive menu Idle: Forwarding connection is OK but the connection is NOT established Error: there is an error on the Forwarding connection Off: indicates when the Forwarding connection is not used Total: total number of enabled Forwarding connections Log Sources status Active: the Log Sources are answering correctly Idle: Log Source not active at the moment Error: there is an error on the Log Source Off: indicates when a Log Source is inactive Total: total number of Log Sources < C > Changes the time value of the Totals for UC metrics. Each time you enter C, the value switches as follows: current value 1 minute 5 minutes 15 minutes 24 hours time when the UCMon has been started < M > Displays additional information < 1 > Displays the Summary view 84 Universal Collector User Guide

85 CHAPTER 5 Monitor UC Activities Table 33 UCMon - Summary screen Label Description < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon tool Universal Collector User Guide 85

86 Status Screen Status Screen Note: To switch between Log Sources and Forwarding connection views, press L. Log Source Table 34 UCMon - Log Source Status Title Uptime Current Time Description Time when the UC has been started Current date and time automatically refreshed Log Source: Name of the Log Source Status Type Collection Forwarding Connection Status of the current Log Source: Active: the connection is OK Err: the connection encountered an error Idle: the connection never received a message from the source or nothing at all for 24 hours Off: a Log Source is inactive Type of the Log Source: Win EL, RT File, Remote File or Syslog Connection parameters Win EL: Server IP or address Syslog: protocol/binded port RT File: Filename (no path) Remote: File path Current Forwarding connection associated with the current Log Source Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Displays the next or previous view of the list of Log Sources < E >rr first Sort Log Source status by Error (ERR) or alphabetical order < V >erbose mode Display additional information. < L >og Source/ Switch between Forwarding connections and Log Sources tables Forwarding < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon tool 86 Universal Collector User Guide

87 CHAPTER 5 Monitor UC Activities Forwarding Connection Table 35 UCMon - Forwarding Status Title Uptime Current Time Description Time when the UC has been started Current date and time automatically refreshed Forwarding Connection Status Status of the current Log Source: Active: the connection is OK Err: the connection encountered an error or the spool may be full Idle: no message transmitted from the source for 24 hours Off: a Forwarding connection is not used Address IP address and port of the remote Forwarding connection. S C A E Current Forwarding connection settings: S: Scheduled C: Compression A: Authentication E: Encryption Usage Spool load of the current Forwarding connection in % Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Displays the next or previous view of the list of Forwarding connections < E >rr first Sort Forwarding connection status by Error (ERR) or alphabetical order < V >erbose mode Display additional information < L>og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon tool Universal Collector User Guide 87

88 Metrics Screen Metrics Screen Note: To switch between Log Sources and Forwarding connection views, press L. Log Source Table 36 UCMon - Log Source Metrics Title Uptime Current Time Description Time when the UC has been started Current date and time automatically refreshed Log Source: Name of the Log Source Format Period Sort Forwarding Connection Collected Filtered To Buffer Format of the displayed values (messages or mps) Period of time when the data are displayed (since uptime, 1mn, 5 mn, 15 mn, 24h) Sorting order of Log Source: By name/ In values (descending) Define the current Forwarding connection with the Log Source Total number of collected message for a given period of time Total number of filtered message for a given period of time Total number of forwarded message for a given period of time Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Log Sources < F >ormat data Switch between messages or messages per seconds < L>og Source/ Switch between Forwarding connections and Log Sources tables Forwarding < C >ycle period Switch of time period (current, 1mn, 5mn, 15mn, 24h, uptime) < S >ort table Sort by collected values (descending) or by name < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon 88 Universal Collector User Guide

89 CHAPTER 5 Monitor UC Activities Forwarding Connection Table 37 UCMon - Forwarding Connection Metrics Title Uptime Current Time Forwarding Connection Format Period Sort IN OUT Usage Description Time when the UC has been started Current date and time automatically refreshed Format of the displayed values (messages or mps) Period of time when the data are displayed (since uptime, 1mn, 5 mn, 15 mn, 24h) Sorting order of Forwarding connection: By name/ In values (descending) Input log rate Number of forwarded logs coming out from the spool Current Forwarding connection spool load Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Forwarding connection and Log Sources < F >ormat data Switch between messages or messages per seconds < L>og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < C >ycle period Switch of time period (current, 1mn, 5mn, 15mn, 24h, uptime) < S >ort table Sort by IN (descending) or by name < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon Universal Collector User Guide 89

90 Trends Screen Trends Screen Note: To switch between Log Sources and Forwarding connection views, press L. Log Source Table 38 UCMon - Log Source Trends Title Uptime Current Time Description Time when the UC has been started Current date and time automatically refreshed Log Source: Name of the Log Source Format Display Forwarding Conn. current, 1min, 5min, 1h, 24h, since uptime Interactive menu Format of the displayed values (messages or mps) Type of display. The possible values are: Collected Filtered Forwarded Name of the Forwarding connection. Log rate over different time periods: n/a: value not available 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Forwarding connection and Log Sources < F >ormat data Switch between messages or messages per seconds < L>og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < D >isplay Displays the values for different probes: Forwarding connection data type: IN or OUT Log Source data type (Collected, Filtered, Forwarded) < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon 90 Universal Collector User Guide

91 CHAPTER 5 Monitor UC Activities Forwarding Connection Table 39 UCMon - Forwarding Connection Trends Title Uptime Current Time Forwarding Connection Format Display current, 1min, 5min, 1h, 24h, uptime Description Time when the UC has been started Current date and time automatically refreshed Format of the displayed values (messages or mps) Type of display. The possible values are: IN OUT Log rate over different time periods: n/a: value not available Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Forwarding connection and Log Sources < F >ormat data Switch between messages or messages per seconds < L>og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < D >isplay Displays the values for different probes: Forwarding connection data type: IN or OUT Log Source data type (Collected, Filtered, Forwarded) < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon Universal Collector User Guide 91

92 RealTime Screen RealTime Screen Note: To switch between Log Sources and Forwarding connection views, press L. Log Sources Table 40 UCMon - RealTime Title Uptime Current Time Description Time when the UC has been started Current date and time automatically refreshed Log Source: Name of the Log Source Display current, 1min, 5min, 1h, 24h, since uptime Interactive menu Type of display. The possible values are: Collected Filtered Forwarded Log rate over different time periods: n/a: value not available 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Forwarding connection and Log Sources < L >og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < D >isplay Displays the values for different probes: Forwarding connection data type: IN or OUT Log Source data type (Collected, Filtered, Forwarded) < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon 92 Universal Collector User Guide

93 CHAPTER 5 Monitor UC Activities Forwarding Connection Table 41 UCMon - Forwarding Connection RealTime Title Uptime Current Time Forwarding Connection Display current, 1min, 5min, 1h, 24h, uptime Description Time when the UC has been started Current date and time automatically refreshed Type of display. The possible values are: IN OUT Log rate over different time periods: n/a: value not available Interactive menu 1-n/n Scrolls the tables into view < N >ext/< P >revious Next or previous values of Forwarding connection and Log Sources < L>og Source/ Switch between Forwarding connection and Log Sources tables Forwarding < D >isplay Displays the values for different probes: Forwarding connection data type: IN or OUT Log Source data type (Collected, Filtered, Forwarded) < 1 > Displays the Summary view < 2> Displays the Status view < 3 > Displays the Metrics view < 4 > Displays the Trend view < 5 > Displays the Real Time view < Q > Quit the UCMon Universal Collector User Guide 93

94 RealTime Screen 94 Universal Collector User Guide

95 CHAPTER 6 Command Line Interface Contents Overview cert_mgt: Manage the Security Certificates uc_checkconf: Check the Current Configuration uc_createlogsources: Import and Create Several Log Sources at a Time uc_decodepwd: Decode Passwords for Windows Files uc_encryptpwd: Encrypt Passwords for Windows Files uc_monitor: UCMon Tool uc_reload: Reload Configuration uc_saveactiveconfas: Save an Active Configuration uc_switchto: Make Configuration Active Overview The Command Line Interface interacts with the local Universal Collector. You can make a configuration active and reload the current configuration, check the current configuration, manage the security certificates, encrypt password or import several Log Sources in a row. To start a Command Line Interface, open a shell into the following path: Table 42 Starting the CLI OS Windows RHEL, SuSE, Solaris CLI C:\Program Files\LogLogic\Universal Collector\tools\ /opt/loglogic/universal_collector/tools/ The extension of the file to execute in order to run the commands differs on each UC supported OS: Windows: uc_*.bat Universal Collector User Guide 95

96 cert_mgt: Manage the Security Certificates RHEL, SuSE, Solaris: no extension All the samples are given for RHEL, SuSE and Solaris environments. For Windows environment, use the same command with *.bat. cert_mgt: Manage the Security Certificates Note: UC does not have to be started. Table 43 cert_mgt Goal Syntax Options Request for *.pem cert_mgt request -pem <certfile> -csr <fileresult> -pwd <password> Request for *.ks cert_mgt request -jks <file.ks> -pwd <password> -csr <fileresult.csr> Request for *.p12 cert_mgt request -p12 <file.p12> -pwd <password> -csr <fileresult.csr> Import for *.ks cert_mgt import -jks <file.ks> -pwd <password> -cert <certtoimport> -rootcert <rootcertificate> Import for *.p12 cert_mgt import -p12 <file.p12> -pwd <password> -cert <certtoimport> Get help on the certificates cert_mgt cert_mgt request -h cert_mgt import Get information on the tool version cert_mgt -v <nameofconf> 96 Universal Collector User Guide

97 CHAPTER 6 Command Line Interface uc_checkconf: Check the Current Configuration Note: UC must be started. Table 44 uc_checkconf Goal Syntax Options Indicate validity of the configuration, and displays potential errors and warnings uc_checkconf -ucc <nameofconf> Get help on the tool uc_checkconf -h Indicate the port to connect to the UC uc_checkconf -ucc -p <portnumber> <nameofconf> Get information on the tool version uc_checkconf -v uc_createlogsources: Import and Create Several Log Sources at a Time Note: UC does not have to be started. Table 45 uc_createlogsources Goal Syntax Options Indicate the type of Log Sources to import (Windows Event Log, syslog, file, remotefile) Import a CSV file with Log Source information to create a Log Source Indicate the *.ucc file where to export the Log Source information Force the command without any confirmation uc_createlogsources uc_createlogsources uc_createlogsources uc_createlogsources -f -t <windows, syslog, file, remotefile> -in <pathname> -out <pathname> Universal Collector User Guide 97

98 uc_decodepwd: Decode Passwords for Windows Files uc_decodepwd: Decode Passwords for Windows Files Note: UC does not have to be started. Table 46 Command Line Interface to decode passwords Goal Syntax Option Allows encoding password /opt/loglogic/universalcollector/tools/ uc_decodepwd <passwordtodecode> uc_encryptpwd: Encrypt Passwords for Windows Files Note: UC does not have to be started. Table 47 Command Line Interface to encrypt passwords Goal Syntax Option Allows encoding password /opt/loglogic/universalcollector/tools/ uc_encryptpwd <passwordtoencrypt> uc_monitor: UCMon Tool Note: UC does not have to be started. Table 48 Command Line Interface Purpose Syntax Option Indicates the UC port to which the UCMon listens to (if not default port) /opt/loglogic/universalcollector/tools/ uc_monitor -p <portnumber> -p <portnumber> uc_reload: Reload Configuration Note: UC must be started. This command is used to update the active configuration without stopping the whole process. 98 Universal Collector User Guide

99 CHAPTER 6 Command Line Interface To update the current configuration, the command is: uc_reload.bat (under Windows) uc_reload (under RHEL, SuSE, Solaris) Example 1: The user wants to update the active configuration conf1 Enter the command to apply a new configuration to the UC via the CLI located in <INSTALL_DIR>/tools. \uc_reload.bat The active configuration is updated. Example 2: The user wants to check the impacted process during an update of the configuration Enter the following command: uc_reload.bat -dryrun -vb Table 49 uc_reload Goal Syntax Options Reload the current configuration to apply changes uc_reload Note: There is no need to enter the name of the configuration as it is the current configuration, which is automatically updated. Universal Collector User Guide 99

100 uc_saveactiveconfas: Save an Active Configuration uc_saveactiveconfas: Save an Active Configuration Note: UC does not have to be started. Table 50 uc_saveactiveconfas Goal Syntax Options Save a configuration currently in use Force to save a configuration currently in use even if it already exists uc_saveactiveconfas <pathname\confname.ucc> uc_saveactiveconfas <pathname\confname.ucc> -f uc_switchto: Make Configuration Active Note: UC must be started. Table 51 uc_switchto Goal Syntax Options Activate UC Configuration Simulate the change of the active UC configuration. Displays possible errors and warnings in the stored configuration and changes between active and stored configurations. uc_switchto -ucc <nameofconf> uc_switchto -ucc <nameofconf> -dryrun Get help on the Switch command uc_switchto -h Indicate the port to connect to the UC uc_switchto -ucc -p <portnumber> <nameofconf> Get information on the Switch version uc_switchto -v Activate UC Configuration and display verbose information uc_switchto -ucc <nameofconf> -vb Switching from One Configuration to Another It is possible to switch from one configuration to another one. To apply a new configuration, the command is: uc_switchto.bat -ucc {myconf}(under Windows) uc_switchto -ucc {myconf} (under RHEL, SuSE, Solaris) In case of an error, the configuration switch is interrupted and the configuration error is logged in the uc.log file. 100 Universal Collector User Guide

101 CHAPTER 6 Command Line Interface Example: The user wants to switch from the current configuration conf1 to conf2 Enter the command to apply a new configuration to the UC via the CLI located in <INSTALL_DIR>/tools. \uc_switchto.bat -ucc c:\tmp\conf2 The current configuration is now conf2. Checking the Impacted Processes It is possible to check which log sources and Forwarding connections are impacted by the new configuration - without having to apply it. To check the impact on the processes: -dryrun gives information on the switch or the update of configurations -dryrun -vb gives detailed information on the switch or the update of configurations Example: The user wants to check the impacted process during a switch of configurations Enter the following command: uc_switchto.bat -ucc {uc.conf.file}.ucc -dryrun -vb You can obtain something like this: 3 configuration files checked 1 Log Source config updated 1 SYSLOG Log Source config updated 2 Forwarding connection updated (1 created, 1 removed) 1 LS Config Updated ============================================================ syslog.1 UPDATE 2 Forwarding Config Updated ============================================================ MyCuteLmi2 MyCuteLmi REMOVE CREATE WARNING data may not have been collected during the switch configuration operation, the log sources [syslog.1] may have been impacted WARNING data contained in Forwarding connection spool of [MyCuteLmi2] may have been lost if remote Forwarding connection was not available SUCCESS-[conf3] DryRun mode : No change has been applied to the running configuration Universal Collector User Guide 101

102 uc_switchto: Make Configuration Active Limitations During a switch process, some limitations may occur. First case If you remove or update a Syslog Log Source, you may stop the flow and lose some data. Second case If you switch from a Forwarding connection to another one for a given Syslog Log Source, you may lose a few events. This behavior is rare though. Third case If you remove a Forwarding connection or modify the values of the buffer size while the connection to the Forwarding connection is not available (e.g. network failure), the Forwarding connection buffer will try to empty itself by sending the remaining data to the Forwarding connection. This will cause the loss of the buffer content during the time-out. 102 Universal Collector User Guide

103 Appendix Contents Sample Configuration Files [UC Configuration] uc.xml [LMI Connection] uldp-samplecommented.uldp.xml [LMI Connection] uldp-samplecommentedauthjks.uldp.xml [LMI Connection] uldp-samplecommentedauthpem.uldp.xml [LMI Connection] uldp-samplecommentedauthpks12.uldp.xml [Log Sources] file-samplecommented.ls.xml [Log Sources] syslog-samplecommented.ls.xml [Log Sources] wmi-samplecommented.ls.xml Regular Expressions Sample Configuration Files In the installation directory, the folder <config-samples> contains the templates you can copy to create a complete configuration manually without using UC Console. - sample-commented.ucc contains documented XML files, - sample-lite.ucc contains XML files with mandatory tags only without documentation, - sample.ucc contains XML files with all the tags without documentation. When you unzip one of them, you obtain: - a uc.xml file: allows the configuration of the UC s general information, - a log-sources sub-folder: contains documented templates to define a log source, it is what you can find under the Collection tab in the GUI - a uldp sub-folder: contains documented templates to define the Forwarding connections. It is what you can find under the Forwarding tab or when editing a Forwarding Connection in the GUI. Universal Collector User Guide 103

104 Sample Configuration Files [UC Configuration] uc.xml You must unzip sample.ucc to display the uc.xml file, which contains the information you can find under the General Settings tab in the GUI. <!-- This is the Universal Collector configuration file. The uc.xml file contains the Universal Collector general parameters. --> <uc schemaversion="2.0"> <!-- Enter the UC configuration label. This value is mandatory --> <configurationname>samplecommented</configurationname> <!-- Enter the port used by the UC to get information (e.g. status, metrics, memory used...) via the CLI. Make sure this port is not already used. Otherwise the UC cannot work. --> <uccommunicationport>1099</uccommunicationport> <!-- If a Syslog Log Source is used, enter general information about the Syslog collection process --> <syslogcollection> <!-- Enter the TCP/UDP parameter and socket buffer size (in kilobytes) - this parameter applies to all the Syslog Log Sources associated to the UC --> <socketbuffersize>1024</socketbuffersize> <!-- UDP parameter and max packet size (in kilobytes) - this parameter applies to all the Syslog Log Sources associated to the UC --> <udpmaxpacketsize>8</udpmaxpacketsize> </syslogcollection> </uc> [LMI Connection] uldp-samplecommented.uldp.xml <!-- The LMI Connection Configuration file defines the properties for connecting the Universal Collector (UC) with an LMI server. Log source logs are sent from the UC to the LMI server. IMPORTANT: this file is linked with the LMI Connection Configuration files and its name must be composed of: 104 Universal Collector User Guide

105 - an ID, e.g. uldp-sample - an extension, i.e. *.uldp.xml. --> <uldpconnection schemaversion="2.0"> <!-- Enter the label of the LMI connection --> <name>full_uldp_file</name> <!-- Enter the information about the modification of the LMI connection --> <revision> <!-- Enter the version number of the current LMI Connection file --> <version>12</version> <!-- Enter the name of the LMI connection author --> <author>admin</author> <!-- Enter the date and time of the LMI connection creation --> <creationdate> t01:00:00-05:00</creationdate> <!-- Enter the name of the user who last modified the LMI connection --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the LMI connection last modification --> <lastmodifieddate> t01:00:00-05:00</lastmodifieddate> </revision> <!-- Enter the IPv4 address or host name of the LMI --> <address> </address> <!-- Enter the LMI port (either encrypted or not) for connection with LMI 5.0 and 5.1 (default value) for secured connection with LMI 5.0 or later for connection with LMI 5.2 or later --> <port>5514</port> Universal Collector User Guide 105

106 Sample Configuration Files <!-- If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed (true) or not (false - default value). --> <compression>true</compression> <!-- Define whether the logs are sent to the LMI during a certain period of time (true) - called a time window - or not (false - default value) --> <sendingwindow>true</sendingwindow> <!-- Define the beginning of the time window. If sendingwindow = true in the above parameter, define the time (hour and minute) when the event starts to be sent (default value = 22:00). --> <sendingwindowstart>22:00</sendingwindowstart> <!-- Define the end of the time window. If you set sendingwindow = true in the above parameter, define the time (hour and minute) when the event stops to be sent (default value = 05:00). --> <sendingwindowstop>05:00</sendingwindowstop> <!-- Define whether the communication is authenticated (true) or not (false - default value) --> <authentication>false</authentication> <!-- Define whether the communication is encrypted (true) or not (false - default value) --> <encryption>false</encryption> <!-- Enter the general LMI connection properties --> <uldpforwarder> <!-- If there are multiple network interfaces, enter the IP address that the UC uses when establishing the connection to LMI. --> <ucbindingip> </ucbindingip> <!-- Enter the spooler size in megabytes (100 MB - default value, 50 GB - maximum value) --> <spoolersize>100</spoolersize> <!-- Enter the reconnection frequency to the LMI (in seconds) --> <reconnectionfrequency>60</reconnectionfrequency> 106 Universal Collector User Guide

107 <!-- Enter the session timeout to LMI (in seconds) --> <sessiontimeout>600</sessiontimeout> <!-- Define whether the UC internal logs are sent to the remote LMI (true) or not (false - default value) --> <internaluclogs>false</internaluclogs> </uldpforwarder> </uldpconnection> [LMI Connection] uldp-samplecommentedauthjks.uldp.xml <!-- The LMI Connection file defines the properties for connecting the Universal Collector (UC) with an LMI server. Log source logs are sent from the UC to the LMI server. IMPORTANT: this file is linked with the LMI connection files and its name must be composed of: - an ID, e.g. uldp-sample - an extension, i.e. *.uldp.xml. --> <uldpconnection schemaversion="2.0"> <!-- Enter the label of the LMI connection --> <name>full_uldp_file</name> <!-- Enter the information about the modification of the LMI connection --> <revision> <!-- Enter the version number of the current LMI connection --> <version>12</version> <!-- Enter the name of the LMI connection author --> <author>admin</author> <!-- Enter the date and time of the LMI connection creation --> <creationdate> t01:00:00-05:00</creationdate> Universal Collector User Guide 107

108 Sample Configuration Files <!-- Enter the name of the user who last modified the LMI connection --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the LMI connection last modification --> <lastmodifieddate> t01:00:00-05:00</lastmodifieddate> </revision> <!-- Enter the IPv4 address or host name of the LMI --> <address> </address> <!-- Enter the LMI port (either encrypted or not) for connection with LMI 5.0 and 5.1 (default value) for secured connection LMI 5.0 or later for connection with LMI 5.2 or later --> <port>5515</port> <!-- If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed (true) or not (false - default value). --> <compression>true</compression> <!-- Define whether the logs are sent to the LMI during a certain period of time (true) - called a time window - or not (false - default value) --> <sendingwindow>true</sendingwindow> <!-- Define the beginning of the time window. If sendingwindow = true in the above parameter, define the time (hour and minute) when the event starts to be sent (default value = 22:00). --> <sendingwindowstart>22:00</sendingwindowstart> <!-- Define the end of the time window. If you set sendingwindow = true in the above parameter, define the time (hour and minute) when the event stops to be sent (default value = 05:00). --> <sendingwindowstop>05:00</sendingwindowstop> <!-- Define whether the communication is authenticated (true) or not (false - default value) --> <authentication>true</authentication> 108 Universal Collector User Guide

109 <!-- Define whether the communication is encrypted (true) or not (false - default value) --> <encryption>false</encryption> <!-- Define the options of the certificate used for LMI connection --> <certificate> <jks> <!-- Enter the filename where the UC Java keystore will be generated --> <jksfile>sample.jks</jksfile> <!-- Enter the UC Java keystore mandatory password you have encrypted with the UC password encryption tool, e.g. "LSKS9bw/ t01fqnd4p3l3pgeoy/n/qqqlezg+qc/kfdq0lvxtpvgziq==" is the encrypted password for "jdoepassword".--> <password>lsks9bw/t01fqnd4p3l3pgeoy/n/qqqlezg+qc/ kfdq0lvxtpvgziq==</password> <!-- Define whether the UC internal logs are sent to the remote LMI (true) or not (false - default value) --> <internaluclogs>false</internaluclogs> </jks> </certificate> </uldpconnection> [LMI Connection] uldp-samplecommentedauthpem.uldp.xml <!-- The LMI Connection Configuration file defines the properties for connecting the Universal Collector (UC) with an LMI server. Log source logs are sent from the UC to the LMI server. IMPORTANT: this file is linked with the LMI Connection Configuration files and its name must be composed of: - an ID, e.g. uldp-sample - an extension, i.e. *.uldp.xml. --> <uldpconnection schemaversion="2.0"> Universal Collector User Guide 109

110 Sample Configuration Files <!-- Enter the label of the LMI connection --> <name>full_uldp_file</name> <!-- Enter the information about the modification of the LMI connection --> <revision> <!-- Enter the version number of the current LMI Connection Configuration file --> <version>12</version> <!-- Enter the name of the LMI connection author --> <author>admin</author> <!-- Enter the date and time of the LMI connection creation --> <creationdate> t01:00:00-05:00</creationdate> <!-- Enter the name of the user who last modified the LMI connection --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the LMI connection last modification --> <lastmodifieddate> t01:00:00-05:00</lastmodifieddate> </revision> <!-- Enter the IPv4 address or host name of the LMI --> <address> </address> <!-- Enter the LMI port (either encrypted or not) for connection with LMI 5.0 and 5.1 (default value) for secured connection LMI 5.0 or later for connection with LMI 5.2 or later --> <port>5515</port> <!-- If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed (true) or not (false - default value). --> <compression>true</compression> 110 Universal Collector User Guide

111 <!-- Define whether the logs are sent to the LMI during a certain period of time (true) - called a time window - or not (false - default value) --> <sendingwindow>true</sendingwindow> <!-- Define the beginning of the time window. If sendingwindow = true in the above parameter, define the time (hour and minute) when the event starts to be sent (default value = 22:00). --> <sendingwindowstart>22:00</sendingwindowstart> <!-- Define the end of the time window. If you set sendingwindow = true in the above parameter, define the time (hour and minute) when the event stops to be sent (default value = 05:00). --> <sendingwindowstop>05:00</sendingwindowstop> <!-- Define whether the communication is authenticated (true) or not (false - default value) --> <authentication>true</authentication> <!-- Define whether the communication is encrypted (true) or not (false - default value) --> <encryption>false</encryption> <!--Define the options of the certificate used for LMI connection--> <certificate> <!-- Define whether the UC internal logs are sent to the remote LMI (true) or not (false - default value) --> <internaluclogs>false</internaluclogs> <pem> <!-- Enter the filename of the UC private key stored in PEM format --> <pemprivkeyfile>pemprivkeyfile</pemprivkeyfile> <!-- Enter the private key mandatory password you have encrypted with the UC password encryption tool, e.g. "LSKS9bw/ t01fqnd4p3l3pgeoy/n/qqqlezg+qc/kfdq0lvxtpvgziq==" is the encrypted password for "jdoepassword".--> <password>lsks9bw/t01fqnd4p3l3pgeoy/n/qqqlezg+qc/ kfdq0lvxtpvgziq==</password> Universal Collector User Guide 111

112 Sample Configuration Files <!-- Enter the filename of the UC certificate stored in PEM format --> <pemcertfile>pemcertfile</pemcertfile> <!-- Enter the filename of the root CA certificate stored in PEM format --> <pemrootcertfile>pemrootcertfile</pemrootcertfile> </pem> </certificate> </uldpconnection> [LMI Connection] uldp-samplecommentedauthpks12.uldp.xml <!-- The LMI Connection Configuration file defines the properties for connecting the Universal Collector (UC) with an LMI server. Log source logs are sent from the UC to the LMI server. IMPORTANT: this file is linked with the LMI Connection Configuration files and its name must be composed of: - an ID, e.g. uldp-sample - an extension, i.e. *.uldp.xml. --> <uldpconnection schemaversion="2.0"> <!-- Enter the label of the LMI connection --> <name>full_uldp_file</name> <!-- Enter the information about the modification of the LMI connection --> <revision> <!-- Enter the version number of the current LMI Connection Configuration file --> <version>12</version> <!-- Enter the name of the LMI connection author --> <author>admin</author> 112 Universal Collector User Guide

113 <!-- Enter the date and time of the LMI connection creation --> <creationdate> t01:00:00-05:00</creationdate> <!-- Enter the name of the user who last modified the LMI connection --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the LMI connection last modification --> <lastmodifieddate> t01:00:00-05:00</lastmodifieddate> </revision> <!-- Enter the IPv4 address or host name of the LMI --> <address> </address> <!-- Enter the LMI port (either encrypted or not) for connection with LMI 5.0 and 5.1 (default value) for secured connection LMI 5.0 or later for connection with LMI 5.2 or later --> <port>5515</port> <!-- If the connection is slow, you can configure the logs to be compressed for a more rapid flow of data. Define whether the logs are compressed (true) or not (false - default value). --> <compression>true</compression> <!-- Define whether the logs are sent to the LMI during a certain period of time (true) - called a time window - or not (false - default value) --> <sendingwindow>true</sendingwindow> <!-- Define the beginning of the time window. If sendingwindow = true in the above parameter, define the time (hour and minute) when the event starts to be sent (default value = 22:00). --> <sendingwindowstart>22:00</sendingwindowstart> <!-- Define the end of the time window. If you set sendingwindow = true in the above parameter, define the time (hour and minute) when the event stops to be sent (default value = 05:00). --> <sendingwindowstop>05:00</sendingwindowstop> Universal Collector User Guide 113

114 Sample Configuration Files <!-- Define whether the communication is authenticated (true) or not (false - default value) --> <authentication>true</authentication> <!-- Define whether the communication is encrypted (true) or not (false - default value) --> <encryption>false</encryption> <!-- Define the options of the certificate used for LMI connection --> <certificate> <pkcs12> <!-- Enter the UC PKCS#12 certificate's filename --> <p12certfile>p12certfile</p12certfile> <!-- Enter the PKCS#12 certificate's mandatory password you have encrypted with the UC password encryption tool, e.g. "LSKS9bw/ t01fqnd4p3l3pgeoy/n/qqqlezg+qc/kfdq0lvxtpvgziq==" is the encrypted password for "jdoepassword".--> <password>lsks9bw/t01fqnd4p3l3pgeoy/n/qqqlezg+qc/ kfdq0lvxtpvgziq==</password> <!-- Enter the filename of the root CA certificate stored in PEM format --> <pemrootcertfile>pemrootcertfile</pemrootcertfile> <!-- Define whether the UC internal logs are sent to the remote LMI (true) or not (false - default value) --> <internaluclogs>false</internaluclogs> </pkcs12> </certificate> </uldpconnection> This file is located in <InstallFolder>\config-samples\. You must unzip sample.ucc and open the log-sources folder,. 114 Universal Collector User Guide

115 [Log Sources] file-samplecommented.ls.xml <!-- This is the FILE Log Source configuration file. The logs will be directly forwarded to the LMI appliance. IMPORTANT: The file name must be composed of: - an ID, e.g. file-sample - an extension, i.e. *.ls.xml. --> <!-- The Type refers to the type of Log Source. --> <logsource type="file" schemaversion="2.0"> <general> <!-- Define whether the current Log Source is active (true - default value) or inactive (false) --> <active>true</active> <!-- Enter the FILE configuration label --> <name>ls-file-template</name> <!-- Enter the FILE configuration description --> <description>comment of the ls-file-template</description> <!-- Enter the modification of the FILE configuration --> <revision> <!-- Enter the current FILE configuration file version number --> <version>12</version> <!-- Enter the FILE file author's name --> <author>admin</author> <!-- Enter the name of the user who last modified the file --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the FILE creation --> <creationdate> t01:00:00-01:00</creationdate> <!-- Enter the FILE last modification date and time --> Universal Collector User Guide 115

116 Sample Configuration Files <lastmodifieddate> t03:40:10-01:00</lastmodifieddate> </revision> </general> <!-- Enter log forwarding information --> <forwarding> <!-- Enter the information about the LMI connection necessary to send logs from the UC to the LMI server --> <uldp> <!-- Enter the LMI connection ID without the extension, e.g. uldp-sample --> <connectionid>uldp-samplecommented</connectionid> <!-- Define whether the log message sent to the LMI server remains in a local time zone (false - default value) or is converted into UTC (true) time zone --> <timeinutc>false</timeinutc> </uldp> </forwarding> <!-- Enter log collection information --> <collection> <!-- Enter the possible maximum length for the message ( default value) --> <maxlinelength>65000</maxlinelength> <!-- Enter the data format, e.g. UTF8 --> <charsetname></charsetname> <!-- Enter general information about the file where the logs are located--> <filename> <!-- Enter the absolute path of the log file to collect. If the log file is rotated, you may enter [id] or [date] in the filename. E.g. c:\temp\logfile[id].log to obtain file names such as logfile1.log or c:\temp\logfile[date].log to obtain file names such as logfile log--> <absolutepath>c:\temp\logfile.log</absolutepath> 116 Universal Collector User Guide

117 <!-- If you have entered [date] for the tag <absolutepath> above (e.g. c:\temp\logfile[date].log), you must set this parameter to true (false - default value) --> <usedaterolling>false</usedaterolling> <!-- If you have set the tag <usedaterolling> to true, you must enter a date format, e.g. yyyymmdd (see docs/api/java/text/simpledateformat.html ) --> <dateformat>yyyymmdd</dateformat> <!-- If you have entered [id] for the tag <absolutepath> above (e.g. c:\temp\logfile[id].log), you must set this parameter to true (false - default value) --> <useidrolling>false</useidrolling> <!-- If you have set the tag <useidrolling> to true, you must enter the number of digits expected (1-9). UC can collect any file with an [id] whose number of digits is between 1 and 9 inclusive. E.g. If you set 5, the following [id] will be taken into account: 1, 054, 586, 00599, 78945, etc.--> <nbdigit>2</nbdigit> </filename> </collection> <!-- Enter log processing information --> <processing> <!-- Define whether the single message has several lines --> <multiline> <!-- Define whether the current multi-line function is active (true) or inactive (false - default value) --> <active>false</active> <!-- Enter the type of multi-line logs, (jboss - default value) 'jboss', 'tomcat', 'weblogic', 'websphere' or 'custom' --> <linecombinerid>jboss</linecombinerid> <!-- If you set 'custom' in the <linecombinerid> parameter above, you must set a regular expression matching the header of the first line of a log --> <userdefinedregexp></userdefinedregexp> Universal Collector User Guide 117

118 Sample Configuration Files <!-- Enter whether you want the UC to send messages that do not match the Header Regexp (true) or not (false - default value)--> <keepheadlesslog>false</keepheadlesslog> <!-- Enter the number of ms after which the multi-line logs are ready to be sent --> <linetimeout>3000</linetimeout> </multiline> <!-- Enter the name of the host used to pair logs on the LMI server --> <hostname>customhostname.com</hostname> <!-- Enter the name of the application used to pair logs on the LMI server --> <appname>customapplicationname</appname> </processing> <!-- Enter log filtering information --> <filter> <!-- Enter a case insensitive regular expression to specify the messages to be matched. E.g. "packet accepted" means that all the lines containing packet accepted are filtered "^64\.242" means that all the lines that are beginning exactly with are filtered "846$" means that all the lines that are ending exactly with 846 are filtered --> <messagefilter>packet accepted</messagefilter> <!-- Define whether the matched messages are filtered (false - default value) or not (true) --> <matchacceptedmessage>false</matchacceptedmessage> </filter> <!-- Enter a tag to filter, sort and search for log sources. Tags are case sensitive. --> <tags> 118 Universal Collector User Guide

119 <!-- You can enter as many tags as you need. The possible values are._a-za-z0-9 and blank space. --> <tag>sample</tag> <tag>commented</tag> </tags> </logsource> Universal Collector User Guide 119

120 Sample Configuration Files [Log Sources] syslog-samplecommented.ls.xml <!-- This is the SYSLOG Log Source configuration file. The source of logs to be forwarded is a SYSLOG message. IMPORTANT: The file name must be composed of: - an ID, e.g. syslog-sample - an extension, i.e. *.ls.xml. --> <!-- The Type refers to the type of Log Source. --> <logsource type="syslog" schemaversion="2.0"> <general> <!-- Define whether the current Log Source is active (true - default value) or inactive (false) --> <active>true</active> <!-- Enter the SYSLOG configuration label --> <name>ls-syslog-template</name> <!-- Enter the SYSLOG file description information --> <description>comment of the ls-syslog-template</description> <!-- Enter the information about the modification of the SYSLOG configuration --> <revision> <!-- Enter the SYSLOG file author's name --> <author>admin</author> <!-- Enter the name of the user who last modified the SYSLOG file --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the SYSLOG file creation --> <creationdate> t01:00:00-01:00</creationdate> <!-- Enter the SYSLOG file last modification date and time --> <lastmodifieddate> t03:40:10-01:00</lastmodifieddate> 120 Universal Collector User Guide

121 </revision> </general> <!-- Enter log forwarding information --> <forwarding> <!-- Enter the information about the LMI connection necessary to send logs from the UC to the LMI server --> <uldp> <!-- Enter the LMI connection ID without the extension, e.g. uldp-sample --> <connectionid>uldp-sample</connectionid> <!-- Define whether the log message sent to the LMI server remains in a local time zone (false - default value) or is converted into UTC (true) time zone --> <timeinutc>false</timeinutc> </uldp> </forwarding> <!-- Enter log collection information --> <collection> <!-- If there are multiple network interfaces, enter the IP address to listen to the logs.otherwise, all the IP addresses are listened to. --> <ip> </ip> <!-- Enter the port to listen to logs --> <port>514</port> <!-- Define whether the Log Source uses the udp (default value) or tcp SYSLOG protocol. Attention: 'udp' or 'tcp' must be in lower case --> <protocol>udp</protocol> </collection> Universal Collector User Guide 121

122 Sample Configuration Files <!-- Enter log filtering information --> <filter> <!-- Enter the minimum accepted severity (see RFC 3164) --> <severity>6</severity> <!-- Enter the accepted facilities (see RFC 3164) To indicate what are the facilities to be accepted: - use a '-' to indicate a range, e.g use a ';' to indicate the exact facilities, e.g. 1;8;23 - use '-' and ';' to indicate the exact facilities and a range, e.g. 1;8-23 Note: 0-23 is the default value--> <facilities>0-23</facilities> <!-- Enter the regular expression to filter the accepted source host. All the logs from all the IP addresses are collected if.* (default value) is set. --> <sourceip>.*</sourceip> </filter> <!-- Enter a tag to filter, sort and search for log sources. Tags are case sensitive. --> <tags> <!-- You can enter as many tags as you need. The possible values are._a-za-z0-9 and blank space. --> <tag>sample</tag> <tag>commented</tag> </tags> </logsource> 122 Universal Collector User Guide

123 [Log Sources] wmi-samplecommented.ls.xml <!-- This is the WEL Log Source configuration file. All the events about the machine s Windows journals will be forwarded. IMPORTANT: The file name must be composed of: - an ID, e.g. wmi-sample - an extension, i.e. *.ls.xml.--> <!-- The Type refers to the type of Log Source. --> <logsource type="wmi" schemaversion="2.0"> <general> <!-- Define whether the current Log Source is active (true - default value) or inactive (false) --> <active>true</active> <!-- Enter the WEL configuration label --> <name>ls-win-template</name> <!-- Enter the WEL configuration file description --> <description>comment of the ls-win-template</description> <!-- Enter the modification of the WEL configuration --> <revision> <!-- Enter the current WEL configuration file version number --> <version>12</version> <!-- Enter the WEL file author's name --> <author>admin</author> <!-- Enter the name of the user who last modified the WEL file --> <lastmodifiedby>admin</lastmodifiedby> <!-- Enter the date and time of the WEL file creation --> <creationdate> t01:00:00-01:00</creationdate> Universal Collector User Guide 123

124 Sample Configuration Files <!-- Enter the WEL file last modification date and time --> <lastmodifieddate> t03:40:10-01:00</lastmodifieddate> </revision> </general> <!-- Enter log forwarding information --> <forwarding> <!-- Enter the information about the LMI connection necessary to send logs from the UC to the LMI server --> <uldp> <!-- Enter the LMI connection ID without the extension, e.g. uldp-sample --> <connectionid>uldp-samplecommented</connectionid> <!-- Define whether the log message sent to the LMI server remains in a local time zone (false - default value) or is converted into UTC (true) time zone --> <timeinutc>false</timeinutc> </uldp> </forwarding> <!-- Enter log collection information --> <collection> <!-- Enter the domain name to access the Windows server --> <domain>domain.company</domain> <!-- Enter the IP address to connect to the Windows server. For local collection, enter only a dot. --> <address> </address> <!-- Enter the login to connect to the Windows server --> <login>jdoe</login> 124 Universal Collector User Guide

125 <!-- To connect to the Windows server, enter the password you have encrypted with the UC password encryption tool, e.g. "LSKS9bw/ t01fqnd4p3l3pgeoy/n/qqqlezg+qc/kfdq0lvxtpvgziq==" is the encrypted password for "jdoepassword".--> <password>lsks9bw/t01fqnd4p3l3pgeoy/n/qqqlezg+qc/ kfdq0lvxtpvgziq==</password> <!-- Enter the time period (in seconds) after which the UC checks for new Windows events (10 - default value)--> <pollingperiod>10</pollingperiod> </collection> <!-- Enter filtering information --> <filter> <!-- Define the WEL journals to include. It can be either: - all journals = all (default value) - only the journals that are specified in the <journallist> block = only - all journals except those specified in the <journallist> block = all_except--> <includejournal>only</includejournal> <!-- Define the list of journals to include or exclude. Note that the journal name is case sensitive. --> <journallist> <journal>security</journal> <journal>application</journal> </journallist> <!-- Enter the regular expression to filter the WEL event ID. All the logs are collected if.* (default value) is set.--> <eventidfilter>.*</eventidfilter> <!-- EEnter the regular expression to filter Windows journal messages on source field. All the logs are collected if.* (default value) is set. --> <sourcefilter>.*</sourcefilter> <!-- Enter the filter operator for the <eventidfilter> and <sourcefilter> tags, It can be either: Universal Collector User Guide 125

126 Regular Expressions - both filters: and (default value) - only one: or --> <filteroperator>and</filteroperator> </filter> <!-- Enter a tag to filter, sort and search for log sources. Tags are case sensitive. --> <tags> <!-- You can enter as many tags as you need. The possible values are._a-za-z0-9 and blank space. --> <tag>sample</tag> <tag>commented</tag> </tags> </logsource> Regular Expressions Regular expressions provide a concise and flexible means for matching (specifying and recognizing) strings of text, such as particular characters, words, or patterns of characters. They are used when you configure Log Sources. Construct Matches Characters x The character x \\ The backslash character \0n The character with octal value 0n (0 <= n <= 7) \0nn The character with octal value 0nn (0 <= n <= 7) \0mnn The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7) \xhh The character with hexadecimal value 0xhh \uhhhh The character with hexadecimal value 0xhhhh \t The tab character ('\u0009') \n The newline (line feed) character ('\u000a') 126 Universal Collector User Guide

127 Construct Matches \r The carriage-return character ('\u000d') \f The form-feed character ('\u000c') \a The alert (bell) character ('\u0007') \e The escape character ('\u001b') \cx The control character corresponding to x Character classes [abc] [^abc] [a-za-z] [a-d[m-p]] [a-z&&[def]] [a-z&&[^bc]] [a-z&&[^m-p]] a, b, or c (simple class) Any character except a, b, or c (negation) a through z or A through Z, inclusive (range) a through d, or m through p: [a-dm-p] (union) d, e, or f (intersection) a through z, except for b and c: [ad-z] (subtraction) a through z, and not m through p: [a-lq-z] (subtraction) Predefined character classes. Any character (may or may not match line terminators) \d A digit: [0-9] \D A non-digit: [^0-9] \s A whitespace character: [\t\n\x0b\f\r] \S A non-whitespace character: [^\s] \w A word character: [a-za-z_0-9] \W A non-word character: [^\w] POSIX character classes (US-ASCII only) \p{lower} A lower-case alphabetic character: [a-z] \p{upper} An upper-case alphabetic character:[a-z] \p{ascii} All ASCII:[\x00-\x7F] \p{alpha} An alphabetic character: [\p{lower}\p{upper}] \p{digit} A decimal digit: [0-9] \p{alnum} An alphanumeric character: [\p{alpha}\p{digit}] \p{punct} Punctuation: One of!"#$%&'()*+,-./:;<=>?@[\]^_`{ }~ \p{graph} A visible character: [\p{alnum}\p{punct}] \p{print} A printable character: [\p{graph}] \p{blank} A space or a tab: [\t] \p{cntrl} A control character: [\x00-\x1f\x7f] \p{xdigit} A hexadecimal digit: [0-9a-fA-F] \p{space} A whitespace character: [\t\n\x0b\f\r] Classes for Unicode blocks and categories \p{ingreek} A character in the Greek block (simple block) \p{lu} An uppercase letter (simple category) \p{sc} A currency symbol Universal Collector User Guide 127

128 Regular Expressions Construct \P{InGreek} [\p{l}&&[^\p{lu}]] Matches Any character except one in the Greek block (negation) Any letter except an uppercase letter (subtraction) Boundary matchers ^ The beginning of a line $ The end of a line \b A word boundary \B A non-word boundary \A The beginning of the input \G The end of the previous match \Z The end of the input except for the final terminator, if any \z The end of the input Greedy quantifiers X? X, once or not at all X* X, zero or more times X+ X, one or more times X{n} X, exactly n times X{n,} X, at least n times X{n,m} X, at least n but not more than m times Reluctant quantifiers X?? X*? X+? X{n}? X{n,}? X{n,m}? Possessive quantifiers X?+ X*+ X++ X{n}+ X{n,}+ X{n,m}+ Logical operators XY X Y (X) X, once or not at all X, zero or more times X, one or more times X, exactly n times X, at least n times X, at least n but not more than m times X, once or not at all X, zero or more times X, one or more times X, exactly n times X, at least n times X, at least n but not more than m times X followed by Y Either X or Y X, as a capturing group Back references \n Whatever the nth capturing group matched 128 Universal Collector User Guide

129 Construct Matches Quotation \ Nothing, but quotes the subsequent character \Q Nothing, but quotes all characters until \E \E Nothing, but ends a quote started by \Q Special constructs (non-capturing) (?:X) X, as a non-capturing group (?idmsux-idmsux) Nothing, but turns match flags on - off (?idmsux-idmsux:x) X, as a non-capturing group with the given flags on - off (?=X) X, via zero-width positive look ahead (?!X) X, via zero-width negative look ahead (?<=X) X, via zero-width positive look behind (?<!X) X, via zero-width negative look behind (?>X) X, as an independent, non-capturing group Universal Collector User Guide 129

130 Regular Expressions 130 Universal Collector User Guide

131 Index C Collection File logs 29 Internal logs 46 Syslog logs 44 Windows logs 35 H Hardware Configuration 16 I Installation 20, 23 silent mode 26 uninstallation 21 L Log File rotation 31 M Monitoring activities 83 S Supported Platforms 15 U Universal Collector definition 13 Universal Collector User Guide 1

132 2 Universal Collector User Guide