What s Up With That Airplane? Visualizing DoD Knowledge Using Splunk Dashboards. Ken Mattern

Transcription

1 What s Up With That Airplane? Visualizing DoD Knowledge Using Splunk Dashboards Ken Mattern

2 Ken Mattern Senior Systems Analyst Data Miner Aranea Solutions, Inc. Huntsville, Alabama Department of Defense System Integrator Operating Environment Splunk Liferay Portal Custom Built Software to Manage Data Movement 10/2/14 2

3 Today's Agenda Introduction of Terms, Data Types, and our Customers History and Goals of the Project Development of Splunk Dashboards Simplifying Things with Liferay Code Samples and Other Resources 10/2/14 3

4 FLEET Structure B Co. A Co. A Co. B Co. Mid Tier 1 Mid Tier 3 C Co. FLEET Top Tier Automated push/pull C Co. Splunk Dashboards Mid Tier 2 A Co. C Co. B Co

5 What Kind of Data Aircraft Data Bus data (Black Box) Health and usage Vibration analysis Engine metrics Safety of flight messages Technical documents and media Maintenance data SQL Server databases Derived data files Data movement logs IIS Custom Windows event logs Liferay portal 10/2/14 5

6 Who is the Customer Aircraft program management offices Subject Matter Experts (SMEs) Maintenance officers Webmasters Data customers Aircraft manufacturers Logistics and support agencies 10/2/14 6

7 The Challenge Aggregate terabytes of disparate data from remote, tactical locations Transform data into actionable information The Solution Splunk Provide management consoles for operational decision making Close to real-time metrics Granular Situational Awareness To unit level Historical Data for Trend Analysis Enable visibility of portal site user interaction 10/2/14 7

8 What You Will See Demonstration version of our DoD operations, Maintenance, and management portal FLEET Sanitized data Decommissioned weapon system F-14D Tomcat Approved for public release 10/2/14 8

9 Where We Started 10/2/14 9

10 Where We Are Now Sun Java System Portal Server 7.2 Transitioning this month to Liferay Windows Server & IIS data transfer platform Custom developed data transfer software F5 LDAP SSO authentication Common access card Direct link to Splunk dashboards 10/2/14 10

11 Where We Are Now Splunk platform Version Sideview Utils Nick Mealy Whoami Brian Wooden Google Maps Customized Cascading Style Sheets Splunk Database Connector (DBX) Splunk SDK for Java and PHP 10/2/14 11

12 10/2/14 12

13 Typical Stats Dashboard 10/2/14 13

14 10/2/14 14

15 Generating Version Info The Search <param name="search"> inputlookup VersionInfo.csv where view= F14" fields view,version,app eval version=if(view="f14",version,"") eval app=if(view="f14",app,"") table version, app </param> The HTML <table width=650><tr> <td><div class="style1"><img src="/static/app/common/images/ FleetLogo.png"></div></td> <td><div class="style0">version: $results[0].version$ </div></td> </tr></table> 10/2/14 15

16 Where We Are Going Liferay web portal Utilizing Liferay portlets Splunk access through Java SDK Splunk data displayed in a single portlet 10/2/14 16

17 Stats with FLEET Top Tier 10/2/14 17

18 Stats Portlet 10/2/14 18

19 Liferay Advantage One script versus 24 advanced XML dashboards Elimination of all Splunk users in LDAP and F5 Elimination of all Splunk Groups One secure login for all Much lighter IA impact Less to maintain Only five searches to worry about rather than over 100 Which would you chose? 10/2/14 19

20 We Also Do Visualization of aggregated data into dashboards that provide actionable information Such as Mid-Tier Status 10/2/14 20

21 Mid-Tier Activity and Data Movement 10/2/14 21

22 Mid-Tier Activity and Data Movement table cs_username, path, Status, MidTierName, Account, "Download MBs", "Upload MBs" <module name="html" group="row.fields.status"> <param name="html"><![cdata[ <img src="/static/app/mid-tierandlogbookstats/images/$row.fields.status $.png"> ]]></param> </module> OK.png Monitor.png Contact.png 10/2/14 22

23 10/2/14 23

24 10/2/14 24

25 Internal User Access Aranea personnel System Engineering Developers Sustainment Splunk administrator Fielding Team that builds and deploys Mid-Tier networks Support 10/2/14 25

26 The Landing Zone 10/2/14 26

27 Mid-Tier Activity and Data Movement <param name="search"> whoami fieldname=user fields user lookup UserDashboards.csv user output Dashboard, url stats first(url) as url by Dashboard table Dashboard, url </param> <module name="table" layoutpanel="panel_row2_col1_grp1"> <param name="hiddenfields">url</param> <param name="count">10</param> <module name="redirector"> <param name="popup">true</param> <param name="url">$row.fields.url$</param> </module> </module> 10/2/14 27

28 Fleet Management 10/2/14 28

29 10/2/14 29

30 Aircraft Mission Status 10/2/14 30

31 10/2/14 31

32 10/2/14 32

33 10/2/14 33

34 10/2/14 34

35 10/2/14 35

36 10/2/14 36

37 10/2/14 37

38 10/2/14 38

39 10/2/14 39

40 What s Up With That Airplane? Visualizing DoD Knowledge Using Splunk Dashboards Ken Mattern For More Information Contact Aranea Solutions, Inc Bradford Dr. Bldg. 1, Suite 210 Huntsville, AL info@araneasolutions.com kmattern@araneasolutions.com

41 Code Samples In the following pages the note sections include expanded code samples of information discussed in the presentation 10/2/14 41

42 Version Info Detail Slide 15 The notes section of this slide includes the XML, search and Sideview HTML module code used to generate the Version Info, Logout and Help page links. 10/2/14 42

43 Portlet PHP Prototype Detail Slide 18 The notes section of this slide includes a snippet of the PHP code used to generate the first Liferay Portlet. This sample shows the search for the display of the number of page views and distinct users 10/2/14 43

44 Displaying Status Icons Detail Slide 22 The notes section of this slide includes a snippet of the XML, search and Sideview HTML module code used to generate the Mid-Tier Status icons 10/2/14 44

45 Displaying Dashboards and Searches Detail Slide 27 The notes section of this slide includes a snippet of the XML, search and Sideview HTML module code used to generate the list of favorite dashboards and saved searches for the user in the Landing Zone dashboard 10/2/14 45