The syslog-ng Premium Edition 5 LTS Administrator Guide

Transcription

1 The syslogng Premium Edition 5 LTS Administrator Guide Publication date October 12, 2015 Abstract This manual is the primary documentation of the syslogng Premium Edition 5 LTS product.

2 Copyright BalaBit SA Copyright 2015 BalaBit SA. All rights reserved. This document is protected by copyright and is distributed under licenses restricting its use, copying, distribution, and decompilation. No part of this document may be reproduced in any form by any means without prior written authorization of BalaBit. This documentation and the product it describes are considered protected by copyright according to the applicable laws. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit ( This product includes cryptographic software written by Eric Young AIX, AIX 5L, AS/400, BladeCenter, eserver, IBM, the IBM logo, IBM System i, IBM System i5, IBM System x, iseries, i5/os, Netfinity, NetServer, OpenPower, OS/400, PartnerWorld, POWER, ServerGuide, ServerProven, and xseries are trademarks or registered trademarks of International Business Machines. Alliance Log Agent for System i is a registered trademark of Patrick Townsend & Associates, Inc. The BalaBit name and the BalaBit logo are registered trademarks of BalaBit SA. Debian is a registered trademark of Software in the Public Interest Inc. Hadoop and the Hadoop elephant logo are trademarks of the Apache Software Foundation. Linux is a registered trademark of Linus Torvalds. MapR, is a trademark of MapR Technologies, Inc. Elasticsearch and Kibana is a trademark of Elasticsearch BV, registered in the U.S. and in other countries. Apache Kafka and the Apache Kafka Logo are trademarks of the Apache Software Foundation. MySQL is a registered trademark of Oracle and/or its affiliates. Oracle, JD Edwards, PeopleSoft, and Siebel are registered trademarks of Oracle Corporation and/or its affiliates. Red Hat, Inc., Red Hat Enterprise Linux and Red Hat Linux are trademarks of Red Hat, Inc. SUSE is a trademark of SUSE AG, a Novell business. Solaris is a registered trademark of Oracle and/or its affiliates. The syslogng name and the syslogng logo are registered trademarks of BalaBit. Windows 95, 98, ME, 2000, XP, Server 2003, Vista, Server 2008, 7, 8, and Server 2012 are registered trademarks of Microsoft Corporation. For details on FIPScompliance, see Section 11.3, Legal Notice of FIPS Compliance of Syslogng Premium Edition (p. 273). All other product names mentioned herein are the trademarks of their respective owners. DISCLAIMER BalaBit is not responsible for any thirdparty Web sites mentioned in this document. BalaBit does not endorse and is not responsible or liable for any content, advertising, products, or other material on or available from such sites or resources. BalaBit will not be responsible or liable for any damage or loss caused or alleged to be caused by or in connection with use of or reliance on any such content, goods, or services that are available on or through any such sites or resources. ii

3 Table of Contents Preface... xv 1. Summary of contents... xv 2. Target audience and prerequisites... xvi 3. Products covered in this guide... xvii 4. Typographical conventions... xvii 5. Contact and support information... xvii 5.1. Sales contact... xviii 5.2. Support contact... xviii 5.3. Training... xviii 6. About this document... xviii 6.1. Summary of changes... xviii 6.2. Feedback... xx 6.3. Acknowledgments... xx 1. Introduction to syslogng What syslogng is What syslogng is not Why is syslogng needed? What is new in syslogng Premium Edition 5 LTS? Who uses syslogng? Public references of syslogng Premium Edition Supported platforms Limitations on Microsoft Windows platforms Certified packages The concepts of syslogng The philosophy of syslogng Logging with syslogng The route of a log message in syslogng Modes of operation Client mode Relay mode Server mode Global objects Timezones and daylight saving How syslogng PE assigns timezone to the message A note on timezones and timestamps Versions and releases of syslogng PE Licensing GPL and LGPL licenses High availability support The structure of a log message BSDsyslog or legacysyslog messages IETFsyslog messages Message representation in syslogng PE Installing syslogng Prerequisites to installing syslogng PE iii

4 3.2. Installing syslogng using the.run installer Installing syslogng in client or relay mode Installing syslogng in server mode Installing syslogng without userinteraction Installing syslogng on RPMbased platforms (Red Hat, SUSE, AIX) Installing syslogng on Debianbased platforms Installing syslogng PE using.pkg installer Installing syslogng PE with userinteraction Installing syslogng PE without userinteraction Installing syslogng PE from a transformed PKG package Installing syslogng PE on Windows platforms Installing syslogng without userinteraction on Windows Upgrading syslogng PE Upgrading syslogng PE to other package versions Upgrading from previous syslogng PE versions to 5 LTS Upgrading from previous syslogng OSE versions Upgrading to syslogng PE 5 LTS Upgrading from syslogng PE to syslogng OSE Upgrading from complete syslogng PE to client setup version of syslogng PE Uninstalling syslogng PE Configuring Microsoft SQL Server to accept logs from syslogng The syslogng PE quickstart guide Configuring syslogng on client hosts Configuring syslogng on server hosts Configuring syslogng relays Configuring syslogng on relay hosts How relaying log messages works The syslogng PE configuration file Location of the syslogng configuration file The configuration syntax in detail Notes about the configuration syntax Global and environmental variables Logging configuration changes Modules in syslogng PE Loading modules Managing complex syslogng configurations Including configuration files Reusing configuration blocks Collecting log messages sources and source drivers How sources work Collecting messages from Windows eventlog sources eventlog() source options Limitations of using the EVT API on Windows Vista or newer Collecting internal messages internal() source options Collecting messages from text files Notes on reading kernel messages File sources and the RFC5424 message format iv

5 file() source options Collecting messages using the RFC3164 protocol (network() driver) network() source options Collecting messages from named pipes pipe() source options Receiving messages from external applications program() source options Collecting messages from tables or relational database Supported SQL sources by platform sql() source options Customizing SQL queries Collecting messages on Sun Solaris sunstreams() source options Collecting messages using the IETF syslog protocol (syslog() driver) syslog() source options Collecting the systemspecific log messages of a platform Collecting messages from remote hosts using the BSD syslog protocol tcp(), tcp6(), udp() and udp6() source options OBSOLETE Collecting messages from UNIX domain sockets unixstream() and unixdgram() source options Sending and storing log messages destinations and destination drivers Storing messages in plaintext files file() destination options Storing messages in encrypted files Displaying the contents of logstore files Journal files logstore() destination options Sending messages to a remote log server using the RFC3164 protocol (network() driver) network() destination options Sending messages to named pipes pipe() destination options Sending messages to external applications program() destination options Sending SNMP traps Converting Cisco syslog messages to "clogmessagegenerated" SNMP traps snmp() destination options Storing messages in an SQL database Using the sql() driver with an Oracle database Using the sql() driver with a Microsoft SQL database The way syslogng interacts with the database MySQLspecific interaction methods MsSQLspecific interaction methods Supported SQL destinations by platform sql() destination options Sending messages to a remote log server using the IETFsyslog protocol syslog() destination options Sending messages to a remote log server using the legacy BSDsyslog protocol (tcp(), udp() drivers) v

6 tcp(), tcp6(), udp(), and udp6() destination options Sending messages to UNIX domain sockets unixstream() and unixdgram() destination options Sending messages to a user terminal usertty() destination Routing messages: log paths, reliability, and filters Log paths Embedded log statements Log path flags Managing incoming and outgoing messages with flowcontrol Flowcontrol and multiple destinations Configuring flowcontrol Using diskbased and memory buffering Enabling reliable diskbased buffering Enabling normal diskbased buffering Enabling memory buffering Clientside failover Filters Using filters Combining filters with boolean operators Comparing macro values in filters Using wildcards, special characters, and regular expressions in filters Tagging messages Filter functions Dropping messages Global options of syslogng PE Configuring global syslogng options Global options TLSencrypted message transfer Secure logging using TLS Encrypting log messages with TLS Configuring TLS on the syslogng clients Configuring TLS on the syslogng server Mutual authentication using TLS Configuring TLS on the syslogng clients Configuring TLS on the syslogng server TLS options FIPScompliant syslogng Installing FIPScompliant syslogng PE Limitations of the FIPScompliant syslogng PE Legal Notice of FIPS Compliance of Syslogng Premium Edition Reliable Log Transfer Protocol Logging using RLTP How RLTP connections work Using RLTP in a clientrelayserver scenario RLTP options Examples for using RLTP Manipulating messages Customizing message format vi

7 Formatting messages, filenames, directories, and tablenames Templates and macros Daterelated macros Hard vs. soft macros Macros of syslogng PE Using template functions Template functions of syslogng PE Modifying messages Replacing message parts Setting message fields to specific values Creating custom SDATA fields Conditional rewrites Regular expressions Types and options of regular expressions Optimizing regular expressions Parsing and segmenting structured messages Parsing messages with commaseparated and similar values Options of CSV parsers Processing message content with a pattern database Classifying log messages The structure of the pattern database How pattern matching works Artificial ignorance Using pattern databases Using parser results in filters and templates Downloading sample pattern databases Correlating log messages Referencing earlier messages of the context Triggering actions for identified messages Conditional actions External actions Actions and message correlation Creating pattern databases Using pattern parsers What's new in the syslogng pattern database format V The syslogng pattern database format Statistics of syslogng Multithreading and scaling in syslogng PE Multithreading concepts of syslogng PE Configuring multithreading Optimizing multithreaded performance Troubleshooting syslogng Possible causes of losing log messages Creating syslogng core files Collecting debugging information with strace, truss, or tusc Running a failure script Stopping syslogng Best practices and examples vii

8 19.1. General recommendations Handling large message load Using name resolution in syslogng Resolving hostnames locally Collecting logs from chroot Configuring log rotation Appendix A. The syslogng manual pages dqtool loggen lgstool pdbtool persisttool syslogng syslogng.conf syslogngctl Appendix B. License contract for BalaBit Product Appendix C. GNU General Public License C.1. Preamble C.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION C.2.1. Section C.2.2. Section C.2.3. Section C.2.4. Section C.2.5. Section C.2.6. Section C.2.7. Section C.2.8. Section C.2.9. Section C Section C Section C NO WARRANTY Section C Section C.3. How to Apply These Terms to Your New Programs Appendix D. GNU Lesser General Public License D.1. Preamble D.2. TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION D.2.1. Section D.2.2. Section D.2.3. Section D.2.4. Section D.2.5. Section D.2.6. Section D.2.7. Section D.2.8. Section D.2.9. Section D Section viii

9 D Section D Section D Section D Section D Section D NO WARRANTY Section D Section D.3. How to Apply These Terms to Your New Libraries Appendix E. Creative Commons Attribution Noncommercial No Derivatives (byncnd) License Glossary List of syslogng PE parameters Index ix

10 List of Examples 2.1. Licensing syslogng PE Licensing syslogng PE Licensing syslogng PE Counting log source hosts Extracting syslogng PE from a transformed PKG package The default configuration file of syslogng PE A simple configuration for clients A simple configuration for servers A simple configuration for relays A simple configuration file Using required and optional parameters Using global variables Reusing configuration blocks Defining blocks with multiple elements Passing arguments to blocks A simple source statement A source statement using two source drivers Setting default priority and facility Source statement on a Linux based operating system Using the eventlog() driver Using the internal() driver Using the file() driver Tailing files Using wildcards in the filename Filerelated information in message Initial window size of file sources Processing Tomcat logs Monitoring multiple directories Using the network() driver Initial window size of a connection Processing Tomcat logs Using the pipe() driver Initial window size of a connection Using the program() driver Initial window size of a connection Using a MySQL source A sample archive query SQL source option columns A sample connect query SQL source option datetimecolumn(col_name, [format]) A sample fetch query Initial window size of a connection SQL source option prefix() SQL source option template() SQL source fetchquery() x

11 6.31. Using the sunstreams() driver Initial window size of a connection Using the syslog() driver Initial window size of a connection Processing Tomcat logs Using the unixstream() and unixdgram() drivers Initial window size of a connection Processing Tomcat logs A simple destination statement Using the file() driver Using the file() driver with macros in the file name and a template for the message Using the logstore() driver Calculating memory usage of logstore journals Setting journal block number and size Setting journal block number and size Using the network() driver Examples for using diskbuffer() Specifying failover servers for syslog() destinations Spoofing the source address on Microsoft Windows Using the pipe() driver Using the program() destination driver Examples for using diskbuffer() Using the snmp() destination driver Defining a Ciscospecific SNMP destination Defining SNMP objects Using the sql() driver Using the sql() driver with an Oracle database Using the sql() driver with an MSSQL database Examples for using diskbuffer() Setting flags for SQL destinations Using SQL NULL values Value: default Using the syslog() driver Examples for using diskbuffer() Specifying failover servers for syslog() destinations Spoofing the source address on Microsoft Windows Using the unixstream() driver Examples for using diskbuffer() Using the usertty() driver A simple log statement Using embedded log paths Using log path flags Soft flowcontrol Hard flowcontrol Sizing parameters for flowcontrol Example for using reliable diskbased buffering Example for using normal diskbased buffering Example for using memory buffering xi

12 8.10. A simple filter statement Comparing macro values in filters Filtering with widcards Adding tags and filtering messages with tags Skipping messages Using global options Calculating memory usage of logstore journals Limiting the memory use of journal files A destination statement using TLS A source statement using TLS Disabling mutual authentication A destination statement using mutual authentication A source statement using TLS Simple RLTP connection RLTP with TLS encryption Using templates and macros Using ${RCPTID} macro Using SDATA macros Using the formatwelf() template function Using the grep template function Using pattern databases and the if template function Using the indentmultiline template function Using numerical template functions Using substitution rules Setting message fields to a particular value Rewriting custom SDATA fields Using conditional rewriting Using Posix regular expressions Using PCRE regular expressions Optimizing regular expressions in filters Segmenting hostnames separated with a dash Parsing Apache log files Segmenting a part of a message Adding the end of the message to the last column Defining pattern databases Using classification results Using classification results for filtering messages Using pattern parsers as macros How syslogng PE calculates contexttimeout Using message correlation Sending triggered messages to the internal() source Generating messages for pattern database matches Sending triggered messages to external applications Pattern parser syntax Using the STRING and ESTRING parsers A V4 pattern database containing a single rule Enabling multithreading File destination for log rotation xii

13 19.2. Logstore destination for log rotation Command for cron for log rotation A.1. lgstool cat filter A.2. lgstool tail filter A.3. Using required and optional parameters A.4. Using global options xiii

14 List of Procedures The route of a log message in syslogng How syslogng PE assigns timezone to the message Installing syslogng in client or relay mode Installing syslogng in server mode Installing syslogng on RPMbased platforms (Red Hat, SUSE, AIX) Installing syslogng on Debianbased platforms Installing syslogng PE with userinteraction Installing syslogng PE without userinteraction Installing syslogng PE from a transformed PKG package Installing syslogng PE on Windows platforms Upgrading to syslogng PE 5 LTS Configuring Microsoft SQL Server to accept logs from syslogng Configuring syslogng on client hosts Configuring syslogng on server hosts Configuring syslogng on relay hosts Change an old source driver to the network() driver Change an old destination driver to the network() driver Configuring TLS on the syslogng clients Configuring TLS on the syslogng server Configuring TLS on the syslogng clients Configuring TLS on the syslogng server How RLTP connections work How conditional rewriting works Creating syslogng core files Resolving hostnames locally Collecting logs from chroot xiv

15 Summary of contents Preface Welcome to the syslogng Premium Edition 5 LTS Administrator Guide! This document describes how to configure and manage syslogng. Background information for the technology and concepts used by the product is also discussed. 1. Summary of contents Chapter 1, Introduction to syslogng (p. 1) describes the main functionality and purpose of syslogng PE. Chapter 2, The concepts of syslogng (p. 8) discusses the technical concepts and philosophies behind syslogng PE. Chapter 3, Installing syslogng (p. 24) describes how to install syslogng PE on various UNIXbased platforms using the precompiled binaries. Chapter 4, The syslogng PE quickstart guide (p. 55) provides a briefly explains how to perform the most common log collecting tasks with syslogng PE. Chapter 5, The syslogng PE configuration file (p. 62) discusses the configuration file format and syntax in detail, and explains how to manage largescale configurations using included files and reusable configuration snippets. Chapter 6, Collecting log messages sources and source drivers (p. 71) explains how to collect and receive log messages from various sources. Chapter 7, Sending and storing log messages destinations and destination drivers (p. 144) describes the different methods to store and forward log messages. Chapter 8, Routing messages: log paths, reliability, and filters (p. 226) explains how to route and sort log messages, and how to use filters to select specific messages. Chapter 9, Global options of syslogng PE (p. 247) lists the global options of syslogng PE and explains how to use them. Chapter 10, TLSencrypted message transfer (p. 261) shows how to secure and authenticate log transport using TLS encryption. Chapter 12, Reliable Log Transfer Protocol (p. 274) describes the reliable log transport that prevents message loss. Chapter 13, Manipulating messages (p. 279) describes how to customize message format using templates and macros, how to rewrite and modify messages, and how to use regular expressions. Chapter 14, Parsing and segmenting structured messages (p. 300) describes how to segment and process structured messages like commaseparated values. xv

16 Target audience and prerequisites Chapter 15, Processing message content with a pattern database (p. 305) explains how to identify and process log messages using a pattern database. Chapter 16, Statistics of syslogng (p. 324) details the available statistics that syslogng PE collects about the processed log messages. Chapter 17, Multithreading and scaling in syslogng PE (p. 327) describes how to configure syslogng PE to use multiple processors, and how to optimize its performance. Chapter 18, Troubleshooting syslogng (p. 330) offers tips to solving problems. Chapter 19, Best practices and examples (p. 334) gives recommendations to configure special features of syslogng PE. Appendix A, The syslogng manual pages (p. 338) contains the manual pages of the syslogng PE application. Appendix B, License contract for BalaBit Product (p. 376) includes the text of the EndUser License Agreement applicable to syslogng Premium Edition. Appendix D, GNU Lesser General Public License (p. 388) includes the text of the LGPLv2.1 license applicable to the core of syslogng Premium Edition. Appendix C, GNU General Public License (p. 382) includes the text of the GPLv2 license applicable to syslogng Premium Edition. Appendix E, Creative Commons Attribution Noncommercial No Derivatives (byncnd) License (p. 397) includes the text of the Creative Commons Attribution Noncommercial No Derivatives (byncnd) License applicable to The syslogng Premium Edition 5 LTS Administrator Guide. Glossary (p. 402) defines the important terms used in this guide. List of syslogng PE parameters (p. 406) provides crossreferences to the definitions of options, parameters, and macros available in syslogng PE. The Index provides crossreferences to important terms used in this guide. 2. Target audience and prerequisites This guide is intended for system administrators and consultants responsible for designing and maintaining logging solutions and log centers. It is also useful for IT decision makers looking for a tool to implement centralized logging in heterogeneous environments. The following skills and knowledge are necessary for a successful syslogng administrator: At least basic system administration knowledge. An understanding of networks, TCP/IP protocols, and general network terminology. Working knowledge of the UNIX or Linux operating system. Indepth knowledge of the logging process of various platforms and applications. An understanding of the legacy syslog (BSDsyslog) protocol) and the new syslog (IETFsyslog) protocol) standard. xvi

17 Products covered in this guide 3. Products covered in this guide This guide describes the use of the following products: syslogng Premium Edition (syslogng PE) and later 4. Typographical conventions Before you start using this guide, it is important to understand the terms and typographical conventions used in the documentation. For more information on specialized terms and abbreviations used in the documentation, see the Glossary at the end of this document. The following kinds of text formatting and icons identify special information in the document. Tip Tips provide best practices and recommendations. Note Notes provide additional information on a topic, and emphasize important facts and considerations. Warning Warnings mark situations where loss of data or misconfiguration of the device is possible if the instructions are not obeyed. Command Emphasis /path/to/file Parameters Label Menu Button Commands you have to execute. Reference items, additional readings. File names. Parameter and attribute names. GUI output messages or dialog labels. A submenu or menu item in the menu bar. Buttons in dialog windows. 5. Contact and support information This product is developed and maintained by BalaBitEurope. We are located in Budapest, Hungary. Our address is: BalaBitEurope 2 Alíz Street xvii

18 Sales contact H1117 Budapest, Hungary Tel: Fax: <info@balabit.com> Web: Sales contact You can directly contact us with sales related topics at the address <sales@balabit.com>, or leave us your contact information and we call you back Support contact In case you experience a problem that is not covered in this guide, visit the syslogng wiki or post it on syslogng mailing list. Product support, including 7x24 online support is available in various packages. For support options, visit the following page. To access the BalaBit Online Support System (BOSS), sign up for an account at the MyBalaBit page and request access to the BalaBit Online Support System (BOSS). Online support is available 24 hours a day. BOSS is available only for registered users with a valid support package. Support address: <support@balabit.com>. Support hotline: (available from 9 AM to 5 PM CET on weekdays) 5.3. Training BalaBitEurope holds courses on using its products for new and experienced users. For dates, details, and application forms, visit the webpage. 6. About this document This guide is a workinprogress document with new versions appearing periodically. The latest version of this document can be downloaded from the BalaBit website here Summary of changes Version 4 F2 5 LTS Changes in product: Procedure 3.6, Installing syslogng PE on Windows platforms (p. 38), Section 1.6.1, Limitations on Microsoft Windows platforms (p. 6), and Section 6.2, Collecting messages from Windows eventlog sources (p. 73) has been added to the document. xviii

19 Summary of changes Several macros that were available only in syslogng Agent for Windows are now available in syslogng PE as well. These have been added to Section , Macros of syslogng PE (p. 282). The allowcompress() and tlsrequired() options have been documented in Section 12.2, RLTP options (p. 276). The usesyslogngpid() option has been documented. The certsubject() option has been documented in Section 10.4, TLS options (p. 267). The customdomain() option has been documented in Section customdomain() (p. 249). The spoofinterface() option of the udp() destination has been documented. The qoutsize() diskbuffer option has been documented. The syslogngctl options reload and stop have been documented in syslogngctl(1) (p. 373). The description of escaping special characters used in templates has been updated in Section , Templates and macros (p. 280). The manual page of the pdbtool utility has been updated, and descriptions to missing options and commands has been added. For details, see pdbtool(1) (p. 352). The tags() option has been documented in Section 6.3.1, internal() source options (p. 77). The expecthostname source flag has been documented. The caps option has been documented in syslogng(8) (p. 363). The Section prototemplate() (p. 256) and Section filetemplate() (p. 250) global options have been documented. Section sessionstatements() (p. 204) has been added to the document. Section indentmultiline (p. 291) has been added to the document. The allowcompress() option has been documented. For details, see Section 10.4, TLS options (p. 267). HPUX 11v3 support on Itanium64 has been added to Section 1.6, Supported platforms (p. 3). Changes in documentation: The synopsis and usage of the facility() and priority() filters have been corrected. The default ports of the syslog drivers have been corrected. Erroneous references to RFC5427 and RFC5428 have been removed. Missing facility names have been added to Section facility() (p. 242). xix

20 Feedback The description of the chainhostnames global option has been extended. For details, see Section chainhostnames() (p. 247). The document index has been updated. The descriptions of statistics types have been clarified in Chapter 16, Statistics of syslogng (p. 324). The description of pipe() has been clarified. Descriptions of the multilineprefix and multilinegarbage options have been removed from the sources that do not support them. Clarified the use of doublequotes and special characters in Section 13.3, Regular expressions (p. 296). Added a note about unsupported column types to Section 7.7.2, Using the sql() driver with a Microsoft SQL database (p. 195). The maximal number of workerthreads has been clarified in Chapter 17, Multithreading and scaling in syslogng PE (p. 327). Corrected a note about persistent message contexts in Section 15.3, Correlating log messages (p. 311). Clarifications in Section 7.7.1, Using the sql() driver with an Oracle database (p. 194). A description of the BOM character has been added to BOM (p. 402). The description of the delimiter option of csvparser() has been clarified. For details, see Section delimiters (p. 303). A few notes regarding kernel messages and file sources have been reorganized to Section 6.4.1, Notes on reading kernel messages (p. 78) Feedback Any feedback is greatly appreciated, especially on what else this document should cover. General comments, errors found in the text, and any suggestions about how to improve the documentation is welcome at documentation@balabit.com Acknowledgments BalaBit would like to express its gratitude to the syslogng users and the syslogng community for their invaluable help and support, including the community members listed at syslogng Community Page. xx

21 What syslogng is Chapter 1. Introduction to syslogng This chapter introduces the syslogng Premium Edition application in a nontechnical manner, discussing how and why is it useful, and the benefits it offers to an existing IT infrastructure What syslogng is The syslogng application is a flexible and highly scalable system logging application that is ideal for creating centralized and trusted logging solutions. The main features of syslogng are summarized below. Reliable log transfer: The syslogng application enables you to send the log messages of your hosts to remote servers using the latest protocol standards. The logs of different servers can be collected and stored centrally on dedicated log servers. Transferring log messages using the RLTP protocol ensures that no messages are lost. Secure logging using TLS: Log messages may contain sensitive information that should not be accessed by third parties. Therefore, syslogng supports the Transport Layer Security (TLS) protocol to encrypt the communication. TLS also allows the mutual authentication of the host and the server using X.509 certificates. Clientside failover: When transferring messages to a remote server, the syslogng PE clients can be configured to send the log messages to secondary servers if the primary server becomes unaccessible. Diskbased message buffering: The Premium Edition of syslogng stores messages on the local hard disk if the central log server or the network connection becomes unavailable. The syslogng application automatically sends the stored messages to the server when the connection is reestablished, in the same order the messages were received. The disk buffer is persistent no messages are lost even if syslogng is restarted. Direct database access: Storing your log messages in a database allows you to easily search and query the messages and interoperate with log analyzing applications. The syslogng application supports the following databases: MSSQL, MySQL, Oracle, PostgreSQL, and SQLite. Encrypted and timestamped log storage: The Premium Edition of syslogng can store log messages securely in encrypted, compressed, and timestamped binary files. Timestamps can be requested from an external Timestamping Authority (TSA). Heterogeneous environments: The syslogng application is the ideal choice to collect logs in massively heterogeneous environments using several different operating systems and hardware platforms, including Linux, Unix, BSD, Sun Solaris, HPUX, Tru64, and AIX. Filter and classify: The syslogng application can sort the incoming log messages based on their content and various parameters like the source host, application, and priority. Directories, files, and database tables can be created dynamically using macros. Complex filtering using regular expressions and boolean operators offers almost unlimited flexibility to forward only the important log messages to the selected destinations. Parse and rewrite: The syslogng application can segment log messages to named fields or columns, and also modify the values of these fields. IPv4 and IPv6 support: The syslogng application can operate in both IPv4 and IPv6 network environments; it can receive and send messages to both types of networks. 1

22 What syslogng is not Depending on the exact syslogng PE configuration, environment, and other parameters, syslogng PE is capable of processing: over messages per second when receiving messages from a single connection and storing them in text files; over messages per second when receiving messages from a single connection and storing them in logstore files; over messages per second when receiving messages from multiple connections and storing them in text files; over messages per second when receiving messages from multiple connections and storing them in logstore files; over messages per second when receiving messages from secure (TLSencrypted) connections and storing them in text files What syslogng is not The syslogng application is not log analysis software. It can filter log messages and select only the ones matching certain criteria. It can even convert the messages and restructure them to a predefined format, or parse the messages and segment them into different fields. But syslogng cannot interpret and analyze the meaning behind the messages, or recognize patterns in the occurrence of different messages Why is syslogng needed? Log messages contain information about the events happening on the hosts. Monitoring system events is essential for security and system health monitoring reasons. The original syslog protocol separates messages based on the priority of the message and the facility sending the message. These two parameters alone are often inadequate to consistently classify messages, as many applications might use the same facility and the facility itself is not even included in the log message. To make things worse, many log messages contain unimportant information. The syslogng application helps you to select only the really interesting messages, and forward them to a central server. Company policies or other regulations often require log messages to be archived. Storing the important messages in a central location greatly simplifies this process. For details on how can you use syslogng to comply with various regulations, see the Regulatory compliance and system logging whitepaper available here 1.4. What is new in syslogng Premium Edition 5 LTS? For details on the news and highlights of syslogng Premium Edition 5 LTS, see the What is new in syslogng Premium Edition 5 LTS. For details on changes in The syslogng Premium Edition 5 LTS Administrator Guide, see Section 6.1.1, Version 4 F2 5 LTS (p. xviii). 2

23 Who uses syslogng? 1.5. Who uses syslogng? The syslogng application is used worldwide by companies and institutions who collect and manage the logs of several hosts, and want to store them in a centralized, organized way. Using syslogng is particularly advantageous for: Internet Service Providers; Financial institutions and companies requiring policy compliance; Server, web, and application hosting companies; Datacenters; Wide area network (WAN) operators; Server farm administrators Public references of syslogng Premium Edition Among others, the following companies decided to use syslogng PE in their production environment: Air France Coop Denmark DataPath, Inc. (Read Case Study) Facebook Hush Communications Canada Inc. Tecnocom Espana Solutions, S.L. (Read Case Study) Telenor Norge AS (Read Case Study) 1.6. Supported platforms The syslogng Premium Edition application is officially supported on the following platforms. Note that the following table is for general reference only, and is not always accurate about the supported platforms and options available for specific platforms. The latest version of this table is available at Unless explicitly noted otherwise, the subsequent releases of the platform (for example, Windows Server 2008 R2 and its service packs in case of Windows Server 2008) are also supported. x86 x86_64 SUN SPARC SUN SPARC64 ppc32 ppc64 PARISC IA64 ALPHA AIX 5.2 & 5.3 AIX 6.1 AIX 7.1 Debian 3. 1 (sarge) 3

24 ALPHA IA64 PARISC ppc64 ppc32 SUN SPARC64 SUN SPARC x86_64 x86 Debian 4.0 (etch) Debian 5. 0 (lenny) Debian 6. 0 (squeeze) FreeBSD 6.1 FreeBSD 7.1 FreeBSD 8.0 HPUX 11i HPUX 11v2 HPUX 11v3 I B M System i opensuse 10.0 opensuse 10.1 opensuse 11.0 Red Hat Enterprise Linux 2 Red Hat Enterprise Linux 3 Red Hat ES 4 Red Hat ES 5 4 Supported platforms

25 Supported platforms x86 x86_64 SUN SPARC SUN SPARC64 ppc32 ppc64 PARISC IA64 ALPHA Red Hat ES 6 SLES 10 SLES 10 SP1 SLES 11.0 Solaris 8 Solaris 9 Solaris 10 Tru64 Ubuntu 8.04 LTS (Hardy Heron) Ubuntu L T S (Lucid Lynx) Windows Table 1.1. Platforms supported by syslogng Premium Edition The central syslogng PE server can be installed on Microsoft Windows platforms as well, in this case you can configure syslogng PE using a configuration file, like on any other platform. However, if you want only to forward eventlog and other log messages from Windows to your central logserver, you can use the syslogng Agent for Windows application. The syslogng Agent for Windows can be managed centrally from a domain controller, and can be configured from a graphical interface as well. The syslogng Agent for Windows application is available as part of syslogng Premium Edition. The syslogng Premium Edition and syslogng Agent for Windows applications are available for the x86 and x86_64 architectures for the following Windows operating systems: Microsoft Windows Server 2003 Microsoft Windows Server 2008 Microsoft Windows Server 2012 Microsoft Windows XP SP3 Microsoft Windows Vista Microsoft Windows 7 Microsoft Windows 8 5

26 Limitations on Microsoft Windows platforms For details about the syslogng Agent for Windows application, see The syslogng Agent for Windows 5 LTS Administrator Guide Limitations on Microsoft Windows platforms The following features and options of syslogng PE are not available on Microsoft Windows platforms. IPv6 is not supported, only IPv4 The pipe() source The pipe() destination The program() source The program() destination The snmp() destination The sql() source The sql() destination The sunstreams() source The unixdgram() source The unixdgram() destination The unixstream() source The unixstream() destination Certified packages 6

27 Certified packages Starting from version 4.0, syslogng Premium Edition is Novell Ready certified for the following platforms: SUSE Linux Enterprise Server 10 on the x86 and x86_64 AMD64 & Intel EM64T architectures SUSE Linux Enterprise Server 11 on the x86 and x86_64 AMD64 & Intel EM64T architectures Starting from version 4.0, syslogng Premium Edition is RedHat Ready certified for the following platforms: Red Hat Enterprise Linux 2.1 on the x86 architecture Red Hat Enterprise Linux 3 on the x86_64 AMD64 & Intel EM64T architecture Red Hat Enterprise Linux 4 on the x86 and x86_64 AMD64 & Intel EM64T architectures Red Hat Enterprise Linux 5 on the x86 and x86_64 AMD64 & Intel EM64T architectures Red Hat Enterprise Linux 6 on the x86 and x86_64 AMD64 & Intel EM64T architectures 7

28 The philosophy of syslogng Chapter 2. The concepts of syslogng This chapter discusses the technical concepts of syslogng The philosophy of syslogng Typically, syslogng is used to manage log messages and implement centralized logging, where the aim is to collect the log messages of several devices on a single, central log server. The different devices called syslogng clients all run syslogng, and collect the log messages from the various applications, files, and other sources. The clients send all important log messages to the remote syslogng server, which sorts and stores them Logging with syslogng The syslogng application reads incoming messages and forwards them to the selected destinations. The syslogng application can receive messages from files, remote hosts, and other sources. Log messages enter syslogng in one of the defined sources, and are sent to one or more destinations. Sources and destinations are independent objects; log paths define what syslogng does with a message, connecting the sources to the destinations. A log path consists of one or more sources and one or more destinations; messages arriving from a source are sent to every destination listed in the log path. A log path defined in syslogng is called a log statement. Optionally, log paths can include filters. Filters are rules that select only certain messages, for example, selecting only messages sent by a specific application. If a log path includes filters, syslogng sends only the messages satisfying the filter rules to the destinations set in the log path. Other optional elements that can appear in log statements are parsers and rewriting rules. Parsers segment messages into different fields to help processing the messages, while rewrite rules modify the messages by adding, replacing, or removing parts of the messages Procedure The route of a log message in syslogng Purpose: The following procedure illustrates the route of a log message from its source on the syslogng client to its final destination on the central syslogng server. 8

29 Logging with syslogng Figure 2.1. The route of a log message Steps: Step 1. A device or application sends a log message to a source on the syslogng client. For example, an Apache web server running on Linux enters a message into the /var/log/apache file. Step 2. The syslogng client running on the web server reads the message from its /var/log/apache source. Step 3. The syslogng client processes the first log statement that includes the /var/log/apache source. Step 4. The syslogng client performs optional operations (message filtering, parsing, and rewriting) on the message; for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslogng sends the message to the destinations set in the log statement, for example, to the remote syslogng server. Warning Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement. Note The syslogng client sends a message to all matching destinations by default. As a result, a message may be sent to a destination more than once, if the destination is used in multiple log statements. To prevent such situations, use the final flag in the destination statements. For details, see Table 8.1, Log statement flags (p. 229). 9

30 Modes of operation Step 5. The syslogng client processes the next log statement that includes the /var/log/apache source, repeating Steps 34. Step 6. The message sent by the syslogng client arrives from a source set in the syslogng server. Step 7. The syslogng server reads the message from its source and processes the first log statement that includes that source. Step 8. The syslogng server performs optional operations (message filtering, parsing, and rewriting) on the message; for example, it compares the message to the filters of the log statement (if any). If the message complies with all filter rules, syslogng sends the message to the destinations set in the log statement. Warning Message filtering, parsing, and rewriting is performed in the order that the operations appear in the log statement. Step 9. The syslogng server processes the next log statement, repeating Steps 79. Note The syslogng application can stop reading messages from its sources if the destinations cannot process the sent messages. This feature is called flowcontrol and is detailed in Section 8.2, Managing incoming and outgoing messages with flowcontrol (p. 230) Modes of operation The syslogng Premium Edition application has three distinct operation scenarios: Client, Server, and Relay. The syslogng PE application running on a host determines the mode of operation automatically based on the license and the configuration file Client mode Figure 2.2. Clientmode operation In client mode, syslogng collects the local logs generated by the host and forwards them through a network connection to the central syslogng server or to a relay. Clients often also log the messages locally into files. No license file is required to run syslogng in client mode. 10