BIG-IP Access Policy Manager and Splunk Templates

Transcription

1 BIG-IP Access Policy Manager and Splunk Templates Summary BIG-IP Access Policy Manager (APM) provides 28 reports to ease the integration of F5 BIG-IP APM logs and the Splunk reporting system. Three are in advanced view report format, two are in advanced form report format, and nineteen are in saved extended fields search report format. You can use these reports as-is or as templates to create your own customized reports. Fifteen of the reports can be displayed in graphical form on the BIG-IP APM Dashboard. The reports are grouped into three search categories: Geolocation, Session, and Access. Prerequisites By default, a Splunk server must be installed and configured to receive syslog entries on UDP port 514. BIG-IP APM-specific logs are automatically grouped into the sourcetype apm_log. BIG-IP APM Splunk templates are specifically looking for syslog entries that contain sourcetype= apm_log. To view Combined Reports in Splunk, you need to enable logging of the session.user.* and session.client.* session variables in the access policy. Refer to for details. Note: To distinguish output from multiple BIG-IP APM syslog sources, you could add a qualifier to the search command. For example, host= sourcetype= apm_log and so on. These reports were developed and tested using BIG-IP version

2 Customization F5 Networks Access Policy Manager dashboard and saved search reports are placed in your Splunk installation server s $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default directory in XML format. You can add or remove search groups in the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/nav/default.xml file. You can add or remove graphical reports in the APM_dashboard.xml file located in the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views directory. You can add or remove saved search reports in the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/savedsearch.conf file. Refer to for detailed customization instructions. Advanced Search Three advanced view search reports, Geolocation by VIP Report, Sessions policy steps Report, Session variables by session ID Report, and two advanced form search reports, Geolocation by state by VIP Report and Top Users by Access Type Report, can be found under the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views directory. The files are named geoview.xml, session_policy.xml, session_vars.xml, geobyvip.xml, and topusersbyaccess.xml. You can find instructions about how to build advanced form searches on a Splunk server at

3 BIG-IP APM Dashboard The BIG-IP APM Dashboard includes 15 graphical reports and 1 raw event report:

4 Geolocation by state bar chart presentation of users by state in the US. Geolocation by country column chart presentation of users by country.

5 Geolocation by region pie chart presentation of users by region. ActiveSync by User pie chart presentation of the top ActiveSync users.

6 ActiveSync by Device pie chart presentation of top devices used to access ActiveSync. Max Concurrent Sessions area chart presentation of maximum concurrency over time.

7 Session Throughput area chart presentation of bytes in and out over time. Access by User Agent pie chart presentation of top access user agent count.

8 Access Types pie chart presentation of top access type count. Top Users by login pie chart presentation of top user access by login.

9 Top Users by throughput pie chart presentation of top user access by throughput. Client Type over Platform column chart presentation of client access type by operating system.

10 Auth Success vs. Failure column chart presentation of authentication success versus failure over time. Access by IP pie chart presentation of top IP addresses that accessed the system.

11 Last 5 BIG-IP APM Events raw syslog entry presentation of the last 5 BIG-IP APM events that occurred. Unique client IP-addresses count shows the number of unique client IP addresses. Note: Most widgets on the dashboard have own TimeRangePicker. One of the options it provides is All time. By default, this option searches through all events on the Splunk server. This behavior may be unacceptable if there is a large amount of data on the server. So, you can change the search range by deleting comments in savedsearches.conf on all lines that contain dispatch.*_time and setting a more limited time period. Or, you can disable the All time option by creating a file called times.conf that contains the following lines: [all_time] disabled = 1

12 Details on Splunk BIG-IP APM Reports ActiveSync by User Report This report searches for User=* syslog entries and charts the count by user. It then sorts by count. Actual search command: search= sourcetype="apm_log" User="*" chart count by User sort count

13

14 ActiveSync by Device Report This report searches for DeviceType=* syslog entries and charts the count by device type. It then sorts by count. Actual search command: search= sourcetype="apm_log" DeviceType="*" chart count by DeviceType sort count

15

16 Geolocation by state Report This report searches for New session syslog entries. It then uses a regular expression to parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then it charts the number of users by state and country. Actual search command: search = sourcetype="apm_log" New session rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" where len(geo_state) > 0 chart count over geo_state by geo_country sort count Note: Further customization can be done by modifying the where clause to create Geolocation by state by VIP Report, for example, where len(geo_state) >0 AND vip=

17

18 Geolocation by country Report This report searches for New session syslog entries. It then uses a regular expression to parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then it charts the count by country and region. Actual search command: search = sourcetype="apm_log" New session rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" where len(geo_country) > 0 chart count over geo_country by geo_region sort count Note: Further customization can be done by modifying the where clause to create Geolocation by country by VIP Report, for example, where len(geo_country) > 0 AND vip=

19 Geolocation by region Report This report searches for New session syslog entries. It then uses a regular expression to parse geo_state, geo_country, and geo_region. It removes empty geolocation entries. Then it charts the count by region. Actual search command: search = sourcetype="apm_log" New session rex field=_raw "\(ST=(?<geo_state>.*)/CC=(?<geo_country>.*)/C=(?<geo_region>.*)\) at VIP (?<vip>\\d+\.\\d+\.\\d+\.\\d+)" where len(geo_region) > 0 chart count by geo_region sort count

20

21 Geolocation by state by VIP Report This is an advanced form search report. Please refer to $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geobyvip.xml for detailed implementation information. The following search template is used to generate the report: sourcetype= apm_log New session rex field=_raw (ST=(?<geo_state>.*) /CC=(?<geo_country>.*) /C=(?<geo_region>.*)\) at VIP (?<vip>\d+.\d+.\d+.\d+) where len(geo_state) > 0 AND vip= $virtualip$ chart count over geo_state by geo_country sort count It performs another search for virtual IP addresses ($virtualip$) and puts them into a dropdown list: sourcetype= apm_log New session rex field=_raw (ST=(?<geo_state>.*) /CC=(?<geo_country>.*) /C=(?<geo_region>.*)\) at VIP (?<vip>\d+.\d+.\d+.\d+) Note: This second search uses its own time interval. It can be set by changing the earliest and latest parameters of the populating search tag in the $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geobyvip.xml file.

22

23 Geolocation by VIP Report This is an advanced view search report. It uses the Splunk modules: TimeRangePicker, SearchSelectLister, CovertToIntention, HiddenSearch, SubmitButton, HiddenChartFormatter, FlashChart, and ViewRedirectorLink. Please refer to $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/geoview.xml for detailed implementation information. Geolocation by VIP Report contains two subreports: Geolocation by state and Geolocation by country. You can specify a pre-populated search of virtual IP address to generate the report.

24 Allow ACL Report This report searches for allow ACL syslog entries. It then uses a regular expression to parse out allowed URLs, network protocol, client IP addresses, and server IP addresses. Then it charts the count by URL and sorts them by count. Actual search command: search = sourcetype="apm_log" allow ACL rex field=_raw "packet: (?<url>.\\s+) (?<net_protocol>.\\w+) (?<clntip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+) -> (?<svrip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+)" chart count by url sort count

25 Note: The search can be further refined to generate other types of reports. For example, In the Top Access by Backend Server IP Address Report, you could slightly change the search command to: search = sourcetype="apm_log" allow ACL rex field=_raw "packet: (?<url>.\\s+) (?<net_protocol>.\\w+) (?<clntip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+) -> (?<svrip>.\\d+\.\\d+\.\\d+\.\\d+:\\d+)" chart count by svrip sort count

26

27 For the Top Access by Client IP Address Report, you could use chart count by clntip to extract the information you need.

28 Session Variables Summary This report searches for session.* syslog entries. It then uses a regular expression to parse the session variable and the session variable value. Then it tabulates the session variables. Actual search command: search = sourcetype="apm_log" session.* rex "(?<sess_var>session\..\\s+) is (?<sess_var_val>.*\\z)" top limit=100 sess_var

29

30 Session Variables by Session Id This report searches for session.* syslog entries. It then uses a regular expression to parse the session variables and the session variable value. Then it charts the count by session ID and sorts by count. Actual search command: search = sourcetype="apm_log" session.* rex "(?<sess_var>session\..\\s+) is (?<sess_var_val>.*\\z)" chart count by session_id sort count

31

32 Session Duration Report This report searches for New session or Session deleted syslog entries. It tracks transactions on session ID because they start with New session and end with Session deleted. It calculates the concurrency weight based on the session duration. It then sorts by duration, joins session_id by performing a subsearch for username. Then it tabulates the username and duration. Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) transaction session_id startswith="new session" endswith="session deleted" concurrency duration=duration sort - duration eval str_duration=tostring(duration, "duration") join session_id [search sourcetype="apm_log" Username rex field=_raw "Username '(?<username>.\\s+)'"] table username, str_duration

33 Note: The pie chart only displays in this report if you change table username, str_duration to table username, duration in the search command.

34 Active Session Report This report searches for New session or Session deleted syslog entries. It tracks transactions on session ID to start with New session, end with Session deleted, and keep, the evicted entries. It performs a subsequent search for event count equals 1 transactions. Then it filters out the Session deleted transactions leaving only the open and active New session sessions. Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) transaction session_id startswith="new session" endswith="session deleted" keepevicted=t search eventcount="1" NOT deleted

35 Max Concurrent Sessions Report This report searches for New session or Session deleted syslog entries. It tracks transactions on session ID to start with New session, end with Session deleted, and keeps the evicted entries. It then calculates the concurrency weight based on the session duration. It charts the maximum concurrent sessions over time. Actual search command: search = sourcetype="apm_log" ((New session) OR (Session deleted)) transaction session_id startswith="new session" endswith="session deleted" keepevicted=t concurrency duration=duration eval duration=tostring(duration, "duration") fields session_id, duration, concurrency chart max(concurrency) over _time

36

37 Session Throughput Report This report searches for statistics syslog entries and filter out empty statistics. It then uses a regular expression to parse bytes in and bytes out values. Then it tabulates bytes in and bytes out over time. Actual search command: search = sourcetype="apm_log" statistics NOT "0," rex field=_raw "bytes in: (?<bytes_in>.\\d+), bytes out: (?<bytes_out>.\\d+)" table _time, bytes_in, bytes_out sort + time

38

39 Access Type Report This report searches for Access policy result: syslog entries. It then uses a regular expression to parse the access types. It then charts the count by access type. Actual search command: search = sourcetype="apm_log" "Access policy result:" rex field=_raw "Access policy result: (?<access_type>.*$)" chart count by access_type

40

41 Top Users by login Report This report searches for Username syslog entries. It then uses a regular expression to pick up the login name. It performs a statistics count by login name and sorts by count. Actual search command: search = sourcetype="apm_log" Username rex field=_raw "Username '(?<login_name>.\\s+)'" stats count by login_name sort count

42

43 Note: You can also combine two search reports into one. For example, to create a Top Users by login and Access Type Report, two search commands can be joined using the same session ID to create the report. Here is how: sourcetype="apm_log" "Access policy result:" rex field=_raw "Access policy result: (?<access_type>.*$)" join session_id [search sourcetype="apm_log" Username rex field=_raw "Username '(?<login_name>.\\s+)'"] chart count over login_name by access_type sort count This creates the following report.

44 You can further refine the report so that it shows only BIG-IP Local Traffic Manager and Access Policy Manager_Mode (also called ActiveSync mode) by replacing chart count over login_name by access_type sort count with where Access_type= LTM+APM_Mode chart count by login_name sort count.

45

46 Top Users by Access Type Report This is an advanced form search report. Please refer to $SPLUNK/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/topusersbyaccess.xml for detailed implementation information. The following search template is used to generate the report: sourcetype="apm_log" "Access policy result: " rex field=_raw "Access policy result: (?<access_type>.*$)" join session_id [search sourcetype="apm_log" Username rex field=_raw "Username '(?<login_name>.\s+)' "] where access_type="$accesstype$" chart count by login_name sort count is used to generate the report. It performs another search by access type ($accesstype$) and puts them into a dropdown list. sourcetype="apm_log" "Access policy result:" rex field=_raw "Access policy result: (?<atype>.*$)" stats count by atype

47

48

49 Top Users by Throughput Report This report searches for Username syslog entries and It then uses a regular expression to extract the login name. It then joins login entries with session ID from the second search pipeline to obtain bytes in bytes out statistic information. It charts max(throughput) over login name and sorts by max(throughput). Actual search command: search = sourcetype="apm_log" Username rex field=_raw "Username '(?<username>.\\s+)'" join session_id [search sourcetype="apm_log" statistics NOT "0," rex field=_raw "bytes in: (?<bytes_in>.\\d+), bytes out: (?<bytes_out>.\\d+)" eval throughput=bytes_in+bytes_out ] chart max(throughput) over username sort - max(throughput)

50

51 Auth Success vs. Failure Report This report searches for Access policy result: syslog entries, and It then uses a regular expression to extract the access types. It then charts the count of the eval Logon_Deny access type and other access types by host. Actual search command: search = sourcetype="apm_log" "Access policy result:" rex field=_raw "Access policy result: (?<access_type>.*$)" chart count(eval(access_type="logon_deny")) AS "Auth Failure", count(eval(not access_type="logon_deny")) AS "Auth Success" by host

52 Access by User Agent Report This report searches for session.user.agent and then uses a regular expression to extract the session user agent string. It then statistically counts by user agent string and sorts by count. Actual search command: search = sourcetype="apm_log" session.user.agent rex field=_raw "session.user.agent is (?<user_agent>.*$)" stats count by user_agent sort count

53

54 Client Type over Platform Report This report searches for session.client.browscap_info syslog entries. It simply charts the count over client type by client platform and sorts by count. Splunk searches automatically to extract the client type and client platform key values. Actual search command: search = sourcetype="apm_log" session.client.browscap_info chart count over ctype by cplatform sort count

55 The following report shows the stacked format of column diagram.

56 PPP tunnels by session id This report searches for PPP tunnel started or PPP tunnel closed syslog entries. It extracts the tunnel ID. It tracks transactions on tunnel ID and session ID to start with started, and end with closed. It then calculates the concurrency weight based on the session duration, sorts by duration, and casts duration as a human-readable string. It also tabulates time, session_id, tunnel_id, and duration. Actual search command: search = sourcetype=apm_log ((PPP tunnel started) OR (PPP tunnel closed)) rex field=_raw "PPP tunnel (?<tunnel_id>0x[a-f0-9]+) " transaction session_id, tunnel_id startswith="started" endswith="closed" concurrency duration=duration sort - duration eval str_duration=tostring(duration,"duration") table _time, session_id, tunnel_id, str_duration

57 PPP tunnels IP addresses per session This report searches for PPP and IPv4 syslog entries. It extracts the tunnel IP address, and sorts by time. It then tabulates the time, session ID, and tunnel IP address. Actual search command: search = sourcetype=apm_log PPP IPv4 rex field=_raw "PPP IPv4: (?<tunnel_ip>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})" sort - _time table _time, session_id, tunnel_ip

58 Access by IP Report This report searches for New session syslog entries. It then extracts the client IP address. Then it charts the count by client IP address and displays the top 20. Actual search command: search = sourcetype=apm_log New session rex "client IP (?<client_ip>[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})" chart count by client_ip sort limit=20 count

59

60 Session policy steps This is an advanced view search report. Please refer to $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/session_policy.xml for detailed implementation information. It contains two subreports: User sessions and Events for session.

61 Session variables by session id This is an advanced view search report. Please refer to $SPLUNK_DIR/etc/apps/SplunkforF5AccessAPM/default/data/ui/views/session_vars.xml for detailed implementation information. This report allows you to display all session variables by chosen session ID.

62 2014 F5 Networks, Inc. All rights reserved. F5, F5 Networks, the F5 logo, BIG-IP, FirePass, icontrol, TMOS, and VIPRION are trademarks or registered trademarks of F5 Networks, Inc. in the U.S. and in certain other countries.