What is new in syslog-ng Agent for Windows 5 LTS

Transcription

1 What is new in syslog-ng Agent for Windows 5 LTS May 24, 2016 Copyright Balabit SA

2 Table of Contents 1. Preface Reliable Log Transfer Protocol Customizable hostnames Control over internal messages Flow control File sources Wildcards in Event Source names Macros and template functions Customizable MARK messages New supported platforms Changes in troubleshooting Changes on the graphical interface Removed functionality Other changes

3 Preface 1. Preface Welcome to syslog-ng Agent for Windows (syslog-ng Agent) version 5 LTS and thank you for choosing our product. This document describes the new features and most important changes since the latest release of syslog-ng Agent. The main aim of this paper is to aid system administrators in planning the migration to the new version of syslog-ng Agent. The following sections describe the news and highlights of syslog-ng Agent 5 LTS. This document covers the 5 LTS release of the syslog-ng Agent for Windows product. Long Term Supported or LTS releases (for example, syslog-ng Agent 4 LTS) are supported for 3 years after their original publication date and for 1 year after the next LTS release is published (whichever date is later). The second digit of the revisions of such releases is 0 (for example, syslog-ng PE 4.0.1). Maintenance releases to LTS releases contain only bugfixes and security updates. Feature releases (for example, syslog-ng Agent 4 F1) are supported for 6 months after their original publication date and for 2 months after succeeding Feature or LTS Release is published (whichever date is later). Feature releases contain enhancements and new features, presumably 1-3 new feature per release. Only the last of the feature releases is supported (for example when a new feature release comes out, the last one becomes unsupported). For a full description on stable and feature releases, see Stable and feature releases. Warning Downgrading from a feature release to an earlier (and thus unsupported) feature release, or to the previous LTS release is officially not supported, but usually works as long as your syslog-ng PE configuration file is appropriate for the old syslog-ng PE version. However, persistent data like the position of the last processed message in a file source will be probably lost. Logstore files created with a newer version of syslog-ng PE might not be readable with an older version of syslog-ng PE. 3

4 Reliable Log Transfer Protocol 2. Reliable Log Transfer Protocol The syslog-ng Agent application can send log messages in a reliable way over the TCP transport layer using the Reliable Log Transfer Protocol (RLTP ). The RLTP transport protocol prevents message loss during connection breaks. It detects the last received message on the receiving end and then starts resending messages from that point. Therefore, messages are not duplicated at the receiving end in case of a connection break. For details on Reliable Log Transfer Protocol, see Chapter 12, Reliable Log Transfer Protocol in The syslog-ng Premium Edition 5 LTS Administrator Guide. 4

5 Customizable hostnames 3. Customizable hostnames In earlier versions, the hostname in a message could differ even on the same host. For example, the hostname was different if the host was a member: eventlog messages used the FQDN of the host (for example, myhost.mydomain), while messages from file sources used the short hostname (for example, myhost). Version 5 LTS of syslog-ng Agent makes it possible to set a standard format for the hostname, and include it in every message, regardless of domain membership, message source, and other factors. It is also possible to automatically convert the hostnames to lowercase. For details on setting and customizing the hostname, see Procedure 5.5, Configuring the hostname format in The syslog-ng Agent for Windows 5 LTS Administrator Guide. 5

6 Control over internal messages 4. Control over internal messages With syslog-ng Agent for Windows 5 LTS, you can control which internal messages of syslog-ng Agent should be sent to the eventlog, or to the remote destinations. For example, you can send all warning-level messages to the remote destinations, and store info-level messages only locally in the Application eventlog container. For details on the internal messages of syslog-ng Agent, see Procedure 5.3, Managing the internal source in The syslog-ng Agent for Windows 5 LTS Administrator Guide. 6

7 Flow control 5. Flow control The destinations in syslog-ng Agent 5 LTS can be flow-controlled. This means that syslog-ng Agent adapts the rate of sending messages to the speed of the server that receives the messages. If you use multiple destinations and enable flow-control for them, syslog-ng Agent will send the messages according to the slowest destination. This functionality replaces the Primary server option. When upgrading your syslog-ng Agent to version 5 LTS, flow-control will be automatically enabled for your primary servers. 7

8 File sources 6. File sources The syslog-ng Agent application can properly handle file sources that use the following encodings: 1200 (UTF-16LE), 1201 (UTF-16BE), (UTF-32LE), (UTF-32BE). Similarly to syslog-ng Premium Edition, the syslog-ng Agent for Windows application will automatically remove the last CRLF control character from multi-line messages. 8

9 Wildcards in Event Source names 7. Wildcards in Event Source names Starting with syslog-ng Agent for Windows 5 LTS, you can use the * and? wildcard characters in the names of event containers. Every time the syslog-ng Agent application is restarted, it will automatically check for new event containers that match the pattern and start sending messages from the new containers. For details on using wildcards in event sources, see Procedure 5.1.2, Adding eventlog sources in The syslog-ng Agent for Windows 5 LTS Administrator Guide. 9

10 Macros and template functions 8. Macros and template functions Version 5 LTS of the syslog-ng Agent for Windows allows you to use several new macros and template functions in your protocol and message templates. Practically, every macro and template function of syslog-ng PE is available in syslog-ng Agent as well. 10

11 Customizable MARK messages 9. Customizable MARK messages The method and frequency of sending MARK messages can be customized. Note that the format of the MARK messages has changed to follow the general practice of such messages. If you are monitoring these messages, adjust your monitoring rules. The following list show the old and new MARK messages: Legacy BSD protocol (RFC3164): Version 4.x: <46>Apr 18 11:34:21 <hostname> syslog-ng-agent[9528]: --- MARK --- Version 5 LTS: <46>Apr 18 11:34:21 <hostname> -- MARK -- Snare protocol: Version 4.x: <46>Apr 18 11:34:21 <hostname> --- MARK --- Version 5 LTS: <46>Apr 18 11:34:21 <hostname> -- MARK -- Syslog protocol (RFC5424): Version 4.x: 149 <46> T15:51:53+02:00 <hostname> syslog-ng-agent [meta sequenceid="1" sysuptime="60001"][origin ip=" "] --- MARK --- Version 5 LTS: 82 <46> T10:51:29+02:00 <hostname> [meta sequenceid="3"] -- MARK -- For details on configuring MARK messages, see Procedure 4.3, Sending MARK messages in The syslog-ng Agent for Windows 5 LTS Administrator Guide. 11

12 New supported platforms 10. New supported platforms Starting with syslog-ng Agent for Windows version 5 LTS, Windows 8 and Windows 2012 platforms are also supported. Note The regular.exe installer of syslog-ng Agent for Windows requires the Microsoft.NET Framework version 2.0, 3.0 or 3.5. This package is usually already installed on most hosts. If it is not, you can download the.net package here. On Windows Server 2012, follow these steps to enable.net 3.5: Version 4.0 of the.net Framework is NOT supported. The nosnapin and the.msi version of the installer does not install the graphical MMC snapin of syslog-ng Agent, and does not require the.net Framework. 12

13 Changes in troubleshooting 11. Changes in troubleshooting Instead of setting debug logging options in the registry, you can set these options using an INI file where you can also configure other debugging-related features. For details setting debug logging options, see Section 10.2, Debugging syslog-ng Agent in The syslog-ng Agent for Windows 5 LTS Administrator Guide. Example 1. Content of the debug.ini file The debug.ini can consist of the following entries: [AgentDbgLog] enabled=on/off path=<debug_file_folder_path> [GpoDbgLog] enabled=on/off path=<debug_file_folder_path> [WriteMiniDump] enabled=on/off Note Starting with syslog-ng Agent for Windows version 5 LTS, it is not possible to display the debug logs using the DebugView application. 13

14 Changes on the graphical interface 12. Changes on the graphical interface In addition to the changes described in other sections, the following has changed on the grapical user interface of syslog-ng Agent: The Throttleoption is now available as a global destination option at syslog-ng Agent settings > Destinations > Destination Global Settings. The Structured Data for event messagesoption is now available as a global destination option at Destinations > Destination global options > Include Eventlog message metadata as SDATA. The Force DNSoption is now available as a global option called Use FQDNat syslog-ng Agent Settings > Global Settings > Hostname > Use FQDN. 14

15 Removed functionality 13. Removed functionality Compared to version 4.0.x, the following features are not available in syslog-ng Agent 5 LTS: The Windows 2000 platform is no longer supported. The IIS 5.x Logoption of file sources has been removed. The Server Properties > Messages > Metadata to include > EventData (deprecated Agent v3.1 functionality)option has been removed. The Server Properties > Messages > Message Type > Agent v3.1 Snare Compatible Message Type (deprecated)option has been removed. The /SENDOLDMSGS option of the syslog-ng Agent installer has been removed. This feature can be set for each source in the configuration of the syslog-ng Agent. The /e command-line option (start the syslog-ng Agent in debug mode and send the messages to the Application eventlog container) has been removed. 15

16 Other changes 14. Other changes The method how syslog-ng Agent computes the fingerprint of its configuration has changed. As a result, the configuration fingerprint will be different after upgrading a host to 5 LTS. If you are using a monitoring tool that alerts if the configuration of the syslog-ng Agent hosts changes, adjust the reference fingerprint after the upgrade. Version 5 LTS of syslog-ng Agent uses the PCRE engine to process regular expressions. This is compatible with the POSIX engine used in earlier versions. The $DATE, $S_DATE and $R_DATE macros use the BSD timestamp format by default, instead of the ISO timestamp format used in earlier versions. The default templates of syslog-ng Agent did not use these macros. If you used the $DATE, $S_DATE, or $R_DATE macros in a custom template, replace them with their $ISODATE, $S_ISODATE, or $R_ISODATE, respectively. In version 5 LTS of syslog-ng Agent, the $TZ macro contains the timezone offset instead of the timezone name (similarly to syslog-ng Premium Edition). For example it will change from "Central Europe Daylight Time" (4.0.3) to "+02:00" (5.0.1). The default templates of syslog-ng Agent did not use this macro. When using the syslog protocol (the RFC5424 message format), syslog-ng Agent will not include a macro in the SDATA if the value of the macro is empty. Earlier versions worked similarly, but the $EVENT_SID_TYPE macro was added even if it was empty (in that case, syslog-ng Agent replaced its value with N/A). When using the syslog protocol (the RFC5424 message format), syslog-ng Agent will not include the [origin ip="<value>" software="<value>"] block in the SDATA anymore. 16