4/7/16. Logging for IR. Planning Linux Logs

Transcription

1 Logging for IR Planning Linux Logs 1

2 Objectives Explanation of Linux logging What does Linux normally logs What can you log in Linux **Suggestions for both** What to study? Linux does not refer to one system There are different distributions of Linux Each has a slightly different set of logs Ubuntu (originally based on Debian) has a good sampling of them 2

3 Ubuntu Logs Ubuntu keeps most of its log files in the /var/log directory Some common files are Auth.log (Rsyslog) Daemon.log (Rsyslog) Debug (Rsyslog) Syslog (Rsyslog) Kern.log (Rsyslog) FailLog Lastlog Wtmp Various program specific logs in their own directories Ubuntu Logs: Non Human Readable WTMP: Keeps track of log ins and log outs (Utmp in some distributions) Stored in binary and not human readable Commands will allow viewing W: The w command tells you who is logged in and what they are doing Who: Tells the user who is logged at this moment Lastlog: Also keeps track of logs (Actually looks at wtmp) Stored in binary and not human readable Can be accessed with the last command Faillog: Part of a set of commands to lock out users after so many unsuccessful attempts This log will keep track of the failures Usually empty but accessed by the command faillog 3

4 Ubuntu Logs: Non Human Readable Kernel Ring buffer This is what usually stores the boot up messages Not a log file it is an area of the kernel Can use DMESG to access it and pull out data Most info duplicated in /var/log/dmesg Ubuntu Logs: Program Specific Some logs will only exist if the appropriate software is installed on the server Apache logs /var/log/apache/access.log Keeps track of pages served Requesting IP Time/date Browser ID string Result code (200 OK or 404 file not found) Query text Administrators can use this information to come to many conclusions /var/log/apache2/error.log Error conditions Identify bugs 4

5 Ubuntu Logs: Program Specific Cups: Common Unix printing system /var/log/cups/error.log Contains printing errors for assisting in troubshooting problems SAMBA: Linux file sharing with windows /var/log/samba Log.nmbd: Messages related to netbios Log.smdb: Messages related to file and print sharing Log.[ip_address]: Messages related to a specific IP address that interacted with the server X11 /var/xorg..log Messages related to Linux GUI (Called X) The many faces of syslog The other logs are all related to Rsyslog Linux tries to follow a simple concept Build software to do a specific task well Many people make different products The best one gets used Can cause confusion and conflict 5

6 The many faces of syslog Syslog was one of the first Created in the 1980 s Originally for Sendmail Proved so useful that it became the standard for logging The many faces of syslog Syslog had some failings Several groups produced newer products to fill in the gaps Syslog NG became the dominate product Syslog NG however split their product into free and paid versions Rsyslog was created to do what Syslog NG could do but remain open source 6

7 The many faces of syslog Ubuntu, Debian, and Redhat adopted Rsyslog as its standard logging system We are going to study rsyslog closely Many of the concepts are the same between systems Some of the settings have slightly different syntax Ubuntu Logging: Rsyslog Rsyslog: The Rocket-fast SYStem for LOG processing Rsyslog is a service which runs on an Ubuntu server The operating system, as well as many programs, send their logs to the daemon Each log is assigned two pieces of information Severity: How important the message is from debug à Emergency Facility: A generic designation of what type of program created the log The rsyslog daemon uses severity and facility to determine where a log message is to be sent To a specific file To another computer To a terminal No where Multiple places How rsyslog routes a message is user maintainable through configuration files When Ubuntu is installed the configuration files are set with what the admins thought most people could use. 7

8 Ubuntu Logging: Default Rsyslog Authorization log /var/log/auth.log Track usage of approval mechanism for allowing users access to areas of the software Tracks use of the sudo command Daemon log /var/log/daemon.log May not be present Contains information about the various daemons running on the server Starts Ends Errors Debug log /var/log/debug Usually empty Only used if debug messages are turned on Ubuntu Logging: Default Rsyslog Kernel log /var/log/kern.log Detailed kernel messages Usually used to debug kernel panics System log /var/log/syslog Where most messages end up Details of many system messages errors and warnings Some warnings and data duplicated here UFW /var/log/ufw.log When the uncomplicated firewall is on it will log messages Firewall logs 8

9 Linux Logs Lab 4 Check out Linux logs Ubuntu Logging: Rsyslog settings There are two files that a user can manipulate to modify Rsyslog /etc/rsyslog.conf This file allows users to configure many of the main features of Rsyslog signs are commented line Allowances for if then else Rules: Actions to be taken based on message Modules: Code that can be added to the functionality of Rsyslog -- They can be written by any third party Further configuration files can be integrated 50-Default.conf 9

10 Ubuntu Logging: Rsyslog.conf (1 of 2) /etc/rsyslog.confconfiguration file for rsyslog. For more information see /us r/s hare/doc /rs y slog-doc/html/rsyslog_conf.html Default logging rules can be found in /etc/rsyslog.d/50-default.conf MODULES $ModLoad imuxsock provides support for local system logging $ModLoad imklog provides kernel logging support $ModLoad immark provides --MARK-- message capability provides UDP syslog reception $ModLoad imudp $UDPServerRun 514 provides TCP syslog reception $ModLoad imtcp $InputTCPServerRun 514 Enable non-kernel facility klog messages $KLogPermitNonKernelFacility on Ubuntu Logging: Rsyslog.conf (2 of 2) GLOBAL DIRECTIVES Use traditional timestamp format. To enable high precision timestamps, comment out the following line. $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat Filter duplicated messages $RepeatedMsgReduction on Set the default permissions for all log files. $FileOwner syslog $FileGroup adm $FileCreateMode 0640 $DirCreateMode 0755 $Umask 0022 $PrivDropToUser syslog $PrivDropToGroup syslog Where to place spool and state files $WorkDirectory /var/spool/rsyslog Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf 10

11 Ubuntu Logging: 50-default.conf (1 of 3) Default rules for rsyslog. For more information see rsyslog.conf(5) and /etc/rsyslog.conf First some standard log files. Log by facility. auth,authpriv.* *.*;auth,authpriv.none cron.* daemon.* kern.* lpr.* mail.* user.* Logging for the mail system. Split it up so that it is easy to write scripts to parse these files. mail.inf o mail.w arn mail.err Logging for INN news system. news.crit news.err news.notice /var/log/auth.log -/var/log/syslog /var/log/cron.log -/var/log/daemon.log -/var/log/kern.log -/var/log/lpr.log -/var/log/mail.log -/var/log/user.log -/var/log/mail.info -/var/log/mail.warn /var/log/mail.err /var/log/news/news.crit /var/log/news/news.err -/var/log/news/news.notice Ubuntu Logging: 50-default.conf (2 of 3) Some "catch-all" log files. *.=debug;\ auth,authpriv.none;\ news.none;mail.none -/var/log/debug *.=info;*.=notice;*.=warn;\ auth,authpriv.none;\ cron,daemon.none;\ mail,news.none -/var/log/messages Emergencies are sent to everybody logged in. *.emerg :omusrmsg:* I like to have messages displayed on the console, but only on a virtual console I usually leave idle. daemon,mail.*;\ news.=crit;news.=err;news.=notice;\ *.=debug;*.=info;\ *.=notice;*.=warn /dev/tty8 11

12 Ubuntu Logging: 50-default.conf (3 of 3) The named pipe /dev/xconsole is for the `xconsole' utility. To use it, you must invoke `xconsole' with the `-file' option: $ xconsole -file /dev/xconsole [...] NOTE: adjust the list below, or you'll go crazy if you have a reasonably busy site.. daemon.*;mail.*;\ news.err;\ *.=debug;*.=info;\ *.=notice;*.=warn /dev/xconsole Ubuntu logging: Log Rotation Logs will eventually fill up It is necessary to rotate the log files in order to keep logging Some log files have a number Syslog.1 Syslog.2.gz Syslog.3.gz These are the old versions of the logs that have been archived The program that does this is called LogRotate 12

13 Ubuntu logging: LogRotate LogRotate is installed in by default It takes the log files, makes copies of them, and then allows more logs to be created Different log files have different rules applied to them The rules are defined by /etc/logrotate.conf It is possible to set rules like Criterion for rotation (Size or time) How many old files to keep File permissions Compression Linux Logs Lab 5 Check out Rsyslog.conf Logrotate.conf 13

14 Ubuntu logging: System Accounting System accounting gathers information about the system Similar to the data given by vmstat CPU usage Memory usage Disk and File I/O Data is collected every few minutes via a cron job Allows an administrator to monitor system utilization Find a baseline If the system deviates from the baseline then there could be an issue Ubuntu logging: System Accounting System Accounting is provided by the sysstat package It must be installed Sudo apt-get install sysstat Then it must be turned on Edit the /etc/default/sysstat Change ENABLED from false to true Finally it must be scheduled sudo nano /etc/cron.d/sysstat 5-55/10 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1 */2 * * * * root command -v debian-sa1 > /dev/null && debian-sa1 1 1 This sets it to run every two minutes (may be excessive) 14

15 Linux Logs Lab 6 Install Sysstat Ubuntu logging: System Accounting The data is stored in /var/logs/sysstat It is not human readable The data can be retrieved using the sar command Sudo sar A >$(date + `hostname`-%d-%m-%y- %H%M.log ) Will create a log file with todays date on it Note that the ` is the character under the tilde Note that capitalization is important 15

16 Ubuntu logging: Process Accounting It is also possible to log individual processes The tools are called psacct or acct Both will do it and were originally designed to allow for chargebacks on mainframes Usually considered overkill Can cause 10%-20% performance degradation Ubuntu logging: Helpful hints Make changes only for specific servers Make sure that special services are logging as you see fit Install system monitoring Watch for messages like imuxsock begins to drop messages due to rate-limiting This could be desired if there is an attack filling your logs However the problem is that it could also be robbing you of important knowlege 16