Big Data Meets Infosec Visualiza4on. Forensics Challenge 10 Honeynet.org

Transcription

1 Big Data Meets Infosec Visualiza4on Forensics Challenge 10 Honeynet.org

2 Challenge Design and build a visualiza4on that describes the aaacks that were analyzed in FC5. Use the three prize winners solu4ons as references and to give you a head start on the data analysis. Use the FC5 dataset to create your FC10 visualiza4on.

3 Solu4on 4 Cloudera CDH3 Nodes Containing HUE and Hive Created auth.log table (Hive data parser) Used exis4ng Apache log (Hive parser) Imported FC5 log data to Hadoop Hive Queried data and stored output to files Used csv files for visualiza4ons

4 Hadoop

5 Hive Parser auth.log CREATE TABLE authlog ( month STRING, \ Day STRING, \ 4me STRING, \ something STRING, \ identd STRING, \ user STRING, \ ipaddress STRING, \ port INT,\ applica4on string \ ) ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.dynamic_type.dynamicserde' WITH SERDEPROPERTIES ( 'serializa4on.format'='org.apache.hadoop.hive.serde2.thrid.tctlseparatedprotocol', 'quote.delim'='(" \\[ \\])', 'field.delim'=' ', 'serializa4on.null.format'='- ') STORED AS TEXTFILE;

6 Hive Queries SELECT * FROM authlog WHERE identd = "Accepted password for root"; SELECT * FROM authlog WHERE identd = "Failed password for root ; CREATE TABLE ipsummary (ipaddress STRING, numrequest INT); INSERT OVERWRITE TABLE ipsummary SELECT ipaddress, COUNT(1) FROM accepted_temp_log GROUP BY ipaddress; SELECT ipsummary.ipaddress, ipsummary.numrequest FROM (SELECT MAX(numrequest) AS themax FROM ipsummary) ipsummarymax JOIN ipsummary ON ipsummarymax.themax = ipsummary.numrequest;

7 Analysis and Visualiza4on Tools Logstash (adhoc queries to Hive data) OpenHeatMap (visualiza4on of aaackers) Excel (Top 10 Brute Force AAacker Chart) Google Charts (adhoc analysis) Maltego (Link analysis of Brute/Accepted)

8 Brute Force Report World Report

9

10

11

12 Link to Open Heat Map hap:// map=squeakersdjambicarrosserie

13 Top 10 Brute Force Report

14 Top 10 A(acks Top 10 AAacks

15 Top 10 AAackers Word Cloud

16 Successful login report

17 Hive Query Output for Accepted Mar 29 13:27:26 app- 1 sshd[21556]: Accepted password for root from port ssh2 Apr 19 05:41:44 app- 1 sshd[8810]: Accepted password for root from port ssh2 Apr 19 05:42:27 app- 1 sshd[9031]: Accepted password for root from port ssh2 Apr 19 05:55:20 app- 1 sshd[12996]: Accepted password for root from port ssh2 Apr 19 05:56:05 app- 1 sshd[13218]: Accepted password for root from port ssh2 Apr 19 10:45:36 app- 1 sshd[28030]: Accepted password for root from port ssh2 Apr 19 11:03:44 app- 1 sshd[30277]: Accepted password for root from port ssh2 Apr 19 11:15:26 app- 1 sshd[30364]: Accepted password for root from port ssh2 Apr 19 22:37:24 app- 1 sshd[2012]: Accepted password for root from port ssh2 Apr 19 22:54:06 app- 1 sshd[2149]: Accepted password for root from port ssh2 Apr 19 23:02:25 app- 1 sshd[2210]: Accepted password for root from port ssh2 Apr 20 06:13:03 app- 1 sshd[26712]: Accepted password for root from port ssh2 Apr 21 11:51:38 app- 1 sshd[2649]: Accepted password for root from port ssh2 Apr 21 11:56:37 app- 1 sshd[2686]: Accepted password for root from port ssh2 Apr 22 01:30:27 app- 1 sshd[4877]: Accepted password for root from port ssh2 Apr 22 06:41:38 app- 1 sshd[5876]: Accepted password for root from port ssh2 Apr 22 11:02:15 app- 1 sshd[7940]: Accepted password for root from port ssh2 Apr 23 03:11:03 app- 1 sshd[13633]: Accepted password for root from port ssh2 Apr 23 03:20:41 app- 1 sshd[13930]: Accepted password for root from port ssh2 Apr 24 11:36:19 app- 1 sshd[24436]: Accepted password for root from port ssh2 Apr 24 15:28:37 app- 1 sshd[31338]: Accepted password for root from port ssh2 Apr 24 16:33:36 app- 1 sshd[31845]: Accepted password for root from port 1844 ssh2 Apr 24 19:15:54 app- 1 sshd[32299]: Accepted password for root from port ssh2 Apr 25 10:38:56 app- 1 sshd[9560]: Accepted password for root from port ssh2 Apr 26 04:42:55 app- 1 sshd[20096]: Accepted password for root from port 3527 ssh2 Apr 26 04:59:02 app- 1 sshd[20491]: Accepted password for root from port 3561 ssh2 Apr 26 08:47:28 app- 1 sshd[23501]: Accepted password for root from port 4271 ssh2 Apr 26 08:51:50 app- 1 sshd[23542]: Accepted password for root from port 4280 ssh2

18 Successful intrusion - visualiza4on Import Failed logins and Accepted logins csv s into Maltego

19

20

21

22

23 Dashboard Top 10 AAackers

24 <!DOCTYPE html PUBLIC "- //W3C//DTD XHTML 1.0 Strict//EN" "hap:// strict.dtd"> <html xmlns="hap:// <head> <meta hap- equiv="content- type" content="text/html; charset=u{- 8"/> <4tle> Google Visualiza4on API Sample </4tle> <script type="text/javascript" src="hap:// <script type="text/javascript"> google.load('visualiza4on', '1.1', {packages: ['controls']}); </script> <script type="text/javascript"> func4on drawvisualiza4on() { // Prepare the data var data = google.visualiza4on.arraytodatatable([ ['Name', 'Loca4on', 'Login', 'Number of AAacks'], [' ', 'United States', 0, 913], [' ', 'China', 0, 1214], [' ', 'Russia', 1, 1237], [' ', 'China', 1, 1256], [' ', 'China', 1, 1751], [' ', 'China', 1, 1823], [' ', 'China', 1, 4212], [' ', 'China', 1, 2878], [' ', 'United States', 0, 8917], [' ', 'China', 1, 26097] ]); // Define a slider control for the Age column. var slider = new google.visualiza4on.controlwrapper({ 'controltype': 'NumberRangeFilter', 'containerid': 'control1', 'op4ons': { 'filtercolumnlabel': 'Login', 'ui': {'labelstacking': 'ver4cal'} } }); Google Charts <!- - You are free to copy and use this sample in accordance with the terms of the Apache license (hap:// 2.0.html) - - >

25 // Define a category picker control for the Gender column var categorypicker = new google.visualiza4on.controlwrapper({ 'controltype': 'CategoryFilter', 'containerid': 'control2', 'op4ons': { 'filtercolumnlabel': 'Loca4on', 'ui': { 'labelstacking': 'ver4cal', 'allowtyping': false, 'allowmul4ple': false } } }); // Define a Pie chart var pie = new google.visualiza4on.chartwrapper({ 'charttype': 'PieChart', 'containerid': 'chart1', 'op4ons': { 'width': 300, 'height': 300, 'legend': 'none', '4tle': 'Top 10 AAackers', 'chartarea': {'led': 15, 'top': 15, 'right': 0, 'boaom': 0}, 'pieslicetext': 'label' }, // Instruct the piechart to use colums 0 (Name) and 3 (Donuts Eaten) // from the 'data' DataTable. 'view': {'columns': [0, 3]} }); // Define a table var table = new google.visualiza4on.chartwrapper({ 'charttype': 'Table', 'containerid': 'chart2', 'op4ons': { 'width': '400px' }

26 }); // Create a dashboard new google.visualiza4on.dashboard(document.getelementbyid('dashboard')). // Establish bindings, declaring the both the slider and the category // picker will drive both charts. bind([slider, categorypicker], [pie, table]). // Draw the en4re dashboard. draw(data); } google.setonloadcallback(drawvisualiza4on); </script> </head> <body style="font- family: Arial;border: 0 none;"> <div id="dashboard"> <table> <tr style='ver4cal- align: top'> <td style='width: 300px; font- size: 0.9em;'> <div id="control1"></div> <div id="control2"></div> <div id="control3"></div> </td> <td style='width: 700px'> <div style="float: led;" id="chart1"></div> <div style="float: led;" id="chart2"></div> <div style="float: led;" id="chart3"></div> </td> </tr> </table> </div> </body> </html>

27

28

29

30

31

32