Linux Syslog Messages in IBM Director

Size: px

Start display at page:

Download "Linux Syslog Messages in IBM Director"

Ashlee Campbell
8 years ago
Views:

1 Ever want those pesky little Linux syslog messages (/var/log/messages) to forward to IBM Director? Well, it s not built in, but it s pretty easy to setup. You can forward syslog messages from an IBM Director Agent (Level-2 agent) to it s Director server in three (3) easy steps: 1. Create a FIFO (more about FIFOs later). 2. Configure the syslog daemon to send messages to the FIFO. 3. Write a script to read the FIFO and generate Director events. Easy, huh? If you re interested, just continue reading. Create a FIFO So what is a FIFO? A FIFO (for First In First Out) is also called a named pipe. Now if you ve worked with Linux or other Unix systems, you re quite familiar with pipes: ls /usr/sbin/ more The pipe ( ) takes the output of the first command ( ls /usr/sbin/ ) and pipes it to the second command ( more ). A FIFO merely takes this pipe concept and gives it a name. A FIFO appears in the file system although it does not take up space in the file system. You create a FIFO using the mkfifo command, specifying the location of the FIFO: mkfifo /opt/ibm/director/logwatch I chose the /opt/ibm/director directory since this FIFO is specifically for IBM Director, but you can put it anywhere. Configure syslog Telling the syslog daemon to send messages to the FIFO is pretty simple as well. Syslog is configured through the /etc/syslog.conf file. Each line tells syslog what to do with messages and consists of two entries, called selectors and actions. The selectors are made up of providers and priorities separated by a period (.). For example: mail.* means all mail messages, news.crit means critical news messages and *.warning means all warnings. Actions are basically regular files to record the message in, Linux consoles to post messages to, remote machines to notify or FIFOs to write to. (See the syslog.conf manpage for more details on configuring the syslog daemon.) To get the syslog daemon to send warning, critical and error messages to our new FIFO, simply add the following line to your /etc/syslog.conf and restart the daemon: *.warning;*.crit;*.err /opt/ibm/director/logwatch Event Handler script Now all we need is a script to forward these messages as IBM Director events. Here s a simple script to do just that: Web location of document ( 1

Configure the syslog daemon to send messages to the FIFO. 3. Write a script to read the FIFO and generate Director events. Easy, huh? If you re interested, just continue reading.

2 #!/bin/bash # logwatcher.sh # A simple script to generate IBM Director events from incoming messages true=0 while [ $true ]; do read line Genevent /TYPE:"Director.Linux.LogWatcher" /TEXT:"$line" done Figure 1 - logwatcher.sh script Now all this script does read input and generate an IBM Director event for each line. To use run the following command: logwatcher.sh < fifo Now every line written to the FIFO will be sent as an IBM Director event. A Better Mousetrap The one limitation in the above script is the fact that the severity of the event is not included in the Director event. Since the priority of the message is not included when syslog generates the message, the best way I ve thought to include severities in the messages is to use multiple FIFOs: prw-r--r-- 1 root root 0 Jul 4 16:16 logwatch.critical prw-r--r-- 1 root root 0 Jul 4 15:02 logwatch.fatal prw-r--r-- 1 root root 0 Jul 4 16:18 logwatch.harmless prw-r--r-- 1 root root 0 Jul 4 16:19 logwatch.minor prw-r--r-- 1 root root 0 Jul 4 16:06 logwatch.unknown prw-r--r-- 1 root root 0 Jul 7 09:46 logwatch.warning Figure 2 - Sample FIFOs I created a FIFO for each priority event I want to forward to Director. # LogWatcher FIFOs *.=info;*.=notice /opt/ibm/director/logwatch.info *.=warning /opt/ibm/director/logwatch.warning *.=err;*.=crit /opt/ibm/director/logwatch.critical *.=emerg /opt/ibm/director/logwatch.fatal Figure 3 - Sample syslog.conf entries I then created a script that you could tell which FIFO to listen to and generate Director events of the proper severity. Web location of document ( 2

sh < fifo Now every line written to the FIFO will be sent as an IBM Director event.

3 #!/usr/bin/perl # Logwatcher # Version 1.0 # # Written by Paul Engel my $fifo_root = "/opt/ibm/director/logwatch"; my $logfile = "/var/log/logwatcher"; my $pidfile = "/var/run/logwatcher.pid"; sub watcher { my $severity = shift; my $dir_sev = $severity; $dir_sev =~ s/^(.)/\u$1/; my $fifo = "${fifo_root.${severity"; defined(my $pid = fork) or die "Can't fork watcher process $severity: $!"; if ($pid){ return $pid else{ open STDERR, ">>$logfile"; open LOG, ">>$logfile"; select LOG; $ = 1; open FILE, "<$fifo" or die "Unable to open FIFO $fifo: $! "; while (1) { while (<FILE>) { chomp; my $date = `date`; chomp $date; print "\n"; print "---\> $date \<--- \n"; my $cmd = "Genevent /TYPE:\"Director.Linux.LogWatcher\" /TEXT:\"$_\" /SEV:$dir_sev >> $logfile"; system $cmd; print "====================================\n"; close FILE; close LOG; close STDERR; Web location of document ( 3

${severity"; defined(my $pid = fork) or die "Can't fork watcher process $severity: $!

4 sub close_watchers { my $signal = shift; print "Shutting down watchers..."; kill unlink $signal; print " [done]\n"; defined(my $pid = fork) or die "Can't fork main daemon process: $!"; if ($pid){ open FILE, ">$pidfile" or die "Unable to open PID file $pidfile: $! "; print FILE "$pid\n"; close FILE; else{ print "Starting watchers..."; push (@pids, watcher('critical')); push (@pids, watcher('fatal')); push (@pids, watcher('harmless')); push (@pids, watcher('minor')); push (@pids, watcher('unknown')); push (@pids, watcher('warning')); print " [done]\n"; $SIG{HUP = \&close_watchers; $SIG{INT = \&close_watchers; $SIG{KILL = \&close_watchers; $SIG{TERM = \&close_watchers; $SIG{QUIT = \&close_watchers; waitpid($pids[$#pids],0); Figure 4 - logwatcher.pl script OK, so this is a little more complicated. First, I used Perl since I could get away from file redirection. Basically this script forks processes to watch each of the FIFOs listed. It then records the process ID (PID) of the controlling daemon. When that daemon is shut down (with a KILL or INT signal) it first shuts down all the child watchers. Also, events are written to the /var/log/logwatcher file including errors and successful message from the Genevent command. Starting the logwatcher To make life easier, I created an init script to start and stop Logwatcher. Save this in your /etc/init.d directory and you can simply use the following call to start Logwatcher: /etc/init.d/logwatcher start Web location of document ( 4

$watcher('warning')); print " [done]\n"; $SIG{HUP = \&close_watchers; $SIG{INT = \&close_watchers; $SIG{KILL = \&close_watchers; $SIG{TERM = \&close_watchers; $SIG{QUIT = \&close_watchers;$

5 #!/bin/bash # Logwatcher init script # Version 1.0 # # Written by: Paul Engel pidfile="/var/run/logwatcher.pid" start() { stop() { if [ -e "$pidfile" ]; then echo "PID file found ($pidfile)!" echo Logwatcher may be running echo Startup Aborted! else echo "Starting LogWatcher " /opt/ibm/director/logwatcher.pl fi if [ -e "$pidfile" ];then echo Stopping LogWatcher kill `cat $pidfile` rm $pidfile else echo Logwatcher does not appear to be running. fi status() { if [ -e "$pidfile" ]; then echo "Logwatcher appears to be running (PID: `cat $pidfile`)" else echo "Logwatcher doesn't appear to be running" fi case "$1" in start) start;; stop) stop;; status) status;; restart) stop sleep 5 start ;; *) Web location of document ( 5

else echo "Starting LogWatcher " /opt/ibm/director/logwatcher.

6 esac echo "usage: $0 {start stop restart" exit 1 ;; Figure 5 - logwatcher init script What you get Now events will start flowing to Director. They will look something like this: Figure 6 - Logwatcher event in Director Event Log You can treat these events like you would any other Director event. Enjoy. Web location of document ( 6

They will look something like this: Figure 6 - Logwatcher event in Director Event Log You

Syslog & xinetd. Stephen Pilon

Syslog & xinetd. Stephen Pilon Syslog & xinetd Stephen Pilon What create log files? Logging Policies Throw away all data immediately Reset log files at periodic intervals Rotate log files, keeping data for a fixed time Compress and