Andrew Moore Amsterdam 2015

Size: px

Start display at page:

Download "Andrew Moore Amsterdam 2015"

Julius Foster
8 years ago
Views:

1 Andrew Moore Amsterdam 2015

2 Agenda Why log How to log Audit plugins Log analysis Demos

3 Logs [timestamp]: [some useful data]

4 Why log? Error Log Binary Log Slow Log General Log

5 Why log?

6 Why log?

7 Why log?

8 How [not] to audit log General log Slow log Binary log Sniff network In schema init_connect

9 MySQL Pluggable Audit Interface Available since Audit interface notifies plugin of General log messages Error log messages Query results sent to client *

messages Error log messages Query results sent to

10 MySQL Pluggable Audit Interface Custom plugin Most popular open source plugins McAfee Percona MariaDB

11 CLI: Installing Plugins INSTALL PLUGIN plugin_name SONAME='shared_lib_name.so' my.cnf: (RECOMMENDED) plugin-load=plugin_name=shared_lib_name.so Startup: mysqld plugin-load='plugin_name'='shared_lib_name.so'

12 Force: Installing Plugins plugin_name=force_plus_permanent Configuration Options: (vary from plugin to plugin) Filtering Sync/performance Output

13 Available for 5.1+ Binary hooking McAfee Audit Plugin Great community support Most filtering options JSON output format Socket and file options

14 McAfee Audit Plugin Install may require generation of offsets (requires GDB)./offset-extract.sh \ /path/to/mysqld \ /path/to/mysqld.debug

$sh \ /path/to/mysqld \$

15 Filtering McAfee Audit Plugin audit_record_cmds audit_record_objects audit_whitelist_users audit_whitelist_cmds

16 McAfee Audit Plugin {"msg-type":"activity", "date":" ", "thread-id":"2", "query-id":"17", "user":"root", "priv_user":"root", "host":"localhost", "ip":"", "cmd":"select", "objects":[{"db":"test","name":"people","obj_type":"table"}], "query":"select * from people"}

$"objects":[{"db":"test","name":"people","obj_type":"table"}],$

17 McAfee Audit Plugin {"msg-type":"activity", {"msg-type":"activity", "date":" ", "date":" ", "thread-id":"3", "thread-id":"2", "query-id":"29", "query-id":"17", "user":"root", "user":"root", "priv_user":"root", "priv_user":"root", "host":"localhost", "host":"localhost", "ip":"", "ip":"", "cmd":"select", "cmd":"select", "objects": "objects":[{"db":"test","name":"people","obj_type":"table"}], [{"db":"test","name":"people_vw","obj_type":"view"}, "query":"select {"db":"test","name":"people","obj_type":"table"}], * from people"} "query":"select * from people_vw"}

$"host":"localhost", "ip":"", "ip":"", "cmd":"select", "cmd":"select", "objects": "objects":[{"db":"test","name":"people","obj_type":"table"}],$

18 Percona Audit Plugin Available for Ships with Percona Server Drop in replacement for Oracle's plugin Limited filtering JSON, XML, CSV output

19 Percona Audit Plugin audit_log_strategy ASYNCHRONOUS PERFORMANCE SEMISYNCHRONOUS SYNCHRONOUS

20 Percona Audit Plugin Support for syslog audit_log_handler = FILE SYSLOG

21 Percona Audit Plugin <AUDIT_RECORD NAME="Query" RECORD="20_ T06:49:49" TIMESTAMP=" T06:53:55 UTC" COMMAND_CLASS="select" CONNECTION_ID="3" STATUS="0" SQLTEXT="select * from people" localhost []" HOST="localhost" OS_USER="" IP="" />

22 Percona Audit Plugin <AUDIT_RECORD NAME="Query" RECORD="32_ T06:49:49" TIMESTAMP=" T06:55:35 UTC" COMMAND_CLASS="select" CONNECTION_ID="4" STATUS="0" SQLTEXT="select * from people_vw" localhost []" HOST="localhost" OS_USER="" IP="" />

23 MariaDB Audit Plugin Available for 5.5+ Expanded Audit API Included by default

24 MariaDB Audit Plugin Table level events CSV output Syslog Plain text passwords Fixed (1.2.0)

25 MariaDB Audit Plugin Filtering server_audit_events server_audit_excl_users server_audit_incl_users

26 MariaDB Audit Plugin :07:25,localhost.localdomain,root,localhost,3,10,QUERY,test,' select * from people',0

27 MariaDB Audit Plugin :11:18,localhost.localdomain,root,localhost,3,9,READ,test,pe ople, :11:18,localhost.localdomain,root,localhost,3,9,QUERY,test,'s elect * from people_vw',0

28 Percona and MariaDB Replication McAfee Slaves log replication events by default *Whitelist blank user {} to prevent

29 Best practices Secure data OS level not logged Utilize log rotation

30 Best practices Sequential IO keep away from random access IO (data files) Use FS with journal to be crash safe(r), EXT4 Synchronizing writes kills performance

31 Performance

32 Audit off Audit on

33 Performance Audit off Audit on

34 Audit off Audit on

35 Log File Storage Secure storage (encryption) Sign logs to ensure not altered Set permissions Store offsite (encrypted of course) Store on read only media

36 Log Aggregation Commercial Oracle Audit Vault McAfee DAM Splunk Open source ELK Stack??? Oracle Audit grep tool

38 ELK

39 Elasticsearch Full text and analytics Apache Lucene RESTful web interface 'Schema-free' JSON documents

40 Elasticsearch Index = database Type = table Document = row

41 Logstash Extract [input] Read files, syslog, socket Transform [filter] Filter, mutate, add & drop Load [output] Write to a destination

42 Logstash Output plugins available Nagios and Nagios_nsca XMPP (hipchat, slack, etc.) Pager duty

43 logstash.conf input { file { path => "/var/log/mysql/audit.log" type => "mysql-audit" }

44 logstash.conf filter { if type == audit_log { grok { source => {%GREEDYDATA} target => new_field } } }

45 logstash.conf output { elasticsearch { cluster => "logstash" host => elasticsearch1 } }

46 Kibana Javascript & node.js browser based dashboards Real-time search and analytics Seamless integration with Elasticsearch No auth, use something else!

47 Typical ELK pipeline

48 Alerts Output workflow for Pager duty nagios_nsca hipchat

49 Fin! Useful links The definitive guide to elasticsearch The logstash book

Who did what, when, where and how MySQL Audit Logging. Jeremy Glick & Andrew Moore 20/10/14

Who did what, when, where and how MySQL Audit Logging Jeremy Glick & Andrew Moore 20/10/14 Intro 2 Hello! Intro 3 Jeremy Glick MySQL DBA Head honcho of Chicago MySQL meetup 13 years industry experience