Gulf Breeze Discusses Application Monitoring Using IBM APM and Log Analysis. Anthony Segelhorst Gulf Breeze Software Partners Expo Booth: 169

Transcription

1 Gulf Breeze Discusses Application Monitoring Using IBM APM and Log Analysis Anthony Segelhorst Gulf Breeze Software Partners Expo Booth: 169

2 Agenda Overview of Application Performance Management (APM ITM v8) Overview of IBM Operations Analytics - Log Analysis Website Monitoring on the Cloud using Selenium How to tackle Application Monitoring Keys to success Real word examples Agent Builder Question and Answer 1

3 Overview of Application Performance Management (APM ITM v8)

4 What is APM Performance and availability for complex applications Supports: Data Centers Public Cloud Hybrid Combinations Provides: Status overview of Applications Health of the Application Components Quality of the End User Experience Current Version (as of 1/15/16)

5 APM Architecture Overview Agents Monitor systems, subsystems, application and collect data Communication between the Agents and Performance Management Server HTTPS communication to the Performance Management Server Agent is the client side of the communication Agents push data to the server every 60 seconds 4

6 APM Architecture Overview Performance Management Server Receives and processes monitoring information from all agents in the infrastructure Provides the Performance Management Console Uses DB2 and MongoDB Data Storage Stores data for all the values sent to the Server for 7 days Key information is stored for longer periods 5

7 Offerings IBM Application Performance Management Advanced Includes: IBM Monitoring IBM Application Performance Management IBM Application Diagnostics IBM Application Performance Management All Agents no Diagnostics views This offering is for end user experience, transaction tracking, and resource monitoring of all your application components. 6

8 Offerings IBM Monitoring Includes: Linux OS, Microsoft.Net, Node.js, Ruby, Unix OS WebSphere Applications, WebSphere Infrastructure Manager, Windows OS All Agents except IBM Integration Bus, SAP, and WebSphere MQ no Diagnostics views This offering is for resource monitoring of infrastructure, application components, and cloud workloads IBM Application Diagnostics This offering is for code level visibility into your applications and the health of your application servers 7

9 Integration Packages Netcool/OMNibus You can forward events from APM to Netcool/OMNibus Operations Analytics - Log Analysis Tivoli Monitoring Hybrid Gateway 8

10 Features Dashboards Thresholds Resource Groups Integrated analytics for predictive outage avoidance Search through log files View KPIs from ITM6 and APM in the same dashboard Historical Metrics Create your own charts and table Reports Role Based Access Control Transaction Tracking 9

11 Key Ports 80 - HTTP port for the Agent Subscription Facility (ASF) and Central Configuration Server HTTPS port for the Agent Subscription Facility (ASF) and Central Configuration Server HTTP port for the Performance Management console user interface HTTPS port for the Performance Management console user interface role-based access control Port for DB2 10

12 IBM Provided Agents Cisco UCS DataPower DB2 HMC Base HTTP Server IBM Integration Bus Microsoft Cluster Server Microsoft SharePoint Server Oracle Database JBoss Linux KVM Linux OS Microsoft Active Directory Microsoft Exchange Server Microsoft SQL Server Microsoft Hyper- V Server Microsoft IIS Microsoft.NET MongoDB MySQL Node.js PHP PostgreSQL Python Response Time Monitoring Ruby SAP Tomcat Unix OS Vmware VI WebSphere Applications WebSphere Infrastructure Manager WebSphere MQ Windows OS 11

13 Overview of IBM Operations Analytics - Log Analysis

14 What is IBM Operations Analytics - Log Analysis Provides a semi-structured data analytics solution Reduce problem diagnosis and resolution time, helping you to manage your infrastructure and applications more effectively 13

15 Different Editions Entry and Standard Editions Entry edition is Free You can load up to 2 GB of data per day No warranty or support on Entry Edition Entry edition can not be used in Production 14

16 Different Editions Edition Features Both Search Dashboards Role-based access control (RBAC) user authentication Insight Packs Statistical functions (min, max, sum, avg, and count) Standard Only Alerts Hadoop integration Additional statistical functions (missing, sumofsquares, stddev, and percentile) 15

17 Components Log Analysis UI Use the Log Analysis UI to create the data model, search, and visualize data. Indexing Engine The Indexing Engine use Apache Solr to index log file records. You can install the Indexing Engine on the same server or on dedicated servers. Event Integration Facility (EIF) Receiver The EIF Receiver is used to load data into Log Analysis. You can install it on the same machine as Log Analysis or you can install it on a remote machine for performance reasons. IBM Tivoli Monitoring Log File Agent 16

18 Components Logstash integration You can also use logstash to load data into Log Analysis. Data storage You can integrate IBM Tivoli Monitoring Log File Agent with Hadoop for long term data storage. Insight Packs Use Insight Packs to customize how log records are indexed and annotated. You can use the out of the box Insight Packs or you can create your own. Custom Search Dashboards REST API Log Analysis tooling 17

19 Benefits Analytics Data archiving Insight Packs Linking Scalable data streaming Search 18

20 Integrations Logfile Agent Omnibus Custom integration NOI (Network Operations Insight) Out of the Box Integration APM 19

21 Resources Great Log Analysis Resource Available Insight Packs Performance Tuning Virtual Appliance for a Trial 01.ibm.com/marketing/iwm/iwm/web/pick.do?source=SCALA&S_TACT =109HJ84W&lang=en_US 20

22 Website Monitoring on the Cloud using Selenium

23 Overview SaaS only offering as of 1/15/16 Released November of 2015 Playback can occur in different locations Can be used for external facing websites Use Selenium-IDE to record a synthetic script based on your web application Support synthetic transaction variables Useful for passwords that need to change every x months Useful for variable data Easier to record transactions in Selenium than Rational Performance Tester 22

24 Key Links IBM Demo Download Selenium IDE Firefox plugin You can download it here: Click on the link. Then open the file via Firefox (File, Open) to install the plugins. Other YouTube videos

25 How to tackle Application Monitoring

26 General Rules Top management support You are going to pull technical resources from their typical day to day tasks. Implementation via a Phased Approach Start with specific application Avoid the all Windows, Unix, and/or Linux servers Avoid negative press on the initial roll-out 25

27 General Rules What is the ultimate business need Clear vision on the problem you are trying to solve. Using technology for its own sake Example: Using web services to make a database call when a simple JDBC/SQL will perform the same task Demos of incremental success Hold people accountable to the timelines 26

28 Planning Key Resources Management Support APM Team RDBMS Design and Support Teams Operating System Support and Design Teams Application Subject Matter Experts Network Resources 27

29 Planning Pre-Requisites Hardware Operating Systems Operating System pre-requisites Application pre-requisites RDBMS Support Firewall Rules Small Successes 28

30 What to Avoid Working in a Vacuum Forcing standards Download and Install!= Success Not having proper training Not engaging the RDBMS Design and Support Teams Not engaging the Operating System Support and Design Teams 29

31 What to Avoid Cont d Forgetting about the application Subject Matter Experts Forgetting about Pre-Requisites Hardware Operating Systems Operating System pre-requisites RDBMS Support Large and Complex milestones Not having at least a non-prod system for testing 30

32 Keys to Success Work Plan Commitment from required teams Sandbox environment Install the application on an Operating System that you can run the application on Don t install it on Linux if you don t have any UNIX/Linux skills Install the Agent Builder on a separate Windows VM Customers lose the source code all the time 31

33 Keys to Success Command Line Configure DR/FO Read Blogs and Social media posts for new features, functions, and tips Build a process to maintain agents Know the documentation and technotes 32

34 Agent Installs Pre-requisites Document the agent pre-reqs for the SA and Application teams Determine a standard for the Agent Install Directory Verify and Test that the SA teams have a process in place to complete all pre-reqs on the system Document the supported operating systems and applications Silent Install Build a silent install process for the agents Leverage your in-house Software Distribution tool set already in place 33

35 Agent Deployments Determine the problem you are trying to solve Determine if an IBM out of the box agent meets the requirements Determine if a custom solution is required Review the vendors documentation for monitoring standards Pick 3-5 key metrics for visualization, alerting, and historical data gathering IBM agent provides thousands of metrics and users can get lost in the data volume 34

36 Agent Deployments Cont d Agent Install and documentation Build visualizations, alerting scenarios, and historical configurations TEST and TEST some more Performance and Load testing Data Quality Deployment Review deployment and make changes as required 35

37 Visualizations Have the SME focus on 3-5 key metrics for visualization. Wrong answers include: All of them I don t know Give me the best practices Right answers include: What do you look at when your boss asks you to look at an issue What metrics are used to help make decisions at 2 AM What commands or data do you use today when looking into issues Ask the vendor What tools are you using today 36

38 Visualizations Cont d How to keep the SME engaged Review the data availability in the Manuals Deploy the agent to a test/sandbox environment Explain how the data can be used Alerting Visualization Historical Data Keep working sessions between minutes at a time Schedule 3-5 working sessions with the SME 37

39 Visualizations Cont d Views Views should be built for the following data points What is happening now? What happened in the last 24 hours? What happened in the last 7 days? 38

40 Integrating APM with Event Management

41 Event Flows Every event needs an action What happens to events without an action Are you wasting Event Bus Cycles 40

42 Sample Event Flow Start Matching Enrichment No Set default values SysContact Set End Yes Yes Ops Visibility set? Yes Set OPS_Visibility=YES Is PROD Yes Tivoli-Admin Disable Page Disable Ticket OPS_Visibility=YES No No No SysContact Set No Is QA Yes Tivoli-Admin Disable Page Disable Ticket OPS_Visibility=NO Yes No Impact process calculates CR or NC based on current time and SysContact value Is PROD Yes Use Enrichment vales as specified Is TEST or PREPROD Yes Disable All Notification and ticketing, expire event after 10 minutes No Is QA Yes Downgrade P2->P3 P3->P3 Disable Page allowed No End No Is TEST or PREPROD Yes Disable Ticket Disable Page allowed No 41

43 On-Boarding Events Process How many people have experienced the following statements or questions? This is not what I wanted Nobody asked me What am I supposed to do with this event? This is way to many tickets, events, or s. Turn it off Why did I get 4 tickets, I only wanted 1? When did this get turned on? I have not slept in 5 days, can you just turn that off? 42

44 On-Boarding Events Process Gather requirements from stakeholders Document and review requirements Gain approval of requirements Build and review design documents Build and review QA/Test Plan Build and review Run Docs Perform a QI on the documentation Develop a solution 43

45 On-Boarding Events Process Perform QA testing Review QA testing results Deploy in an /log only format Deploy into a Ticketing/NOC deployment 44

46 Event Enrichment Getting alerts to the EventBus is typically a simple task, but enhancing them and getting them to perform the correct action can be harder Typical Scenarios include: Event Routing Ticketing, Paging, and/or ing Summary Cleanup Formatting the event Summary to something meaningful to the SME Impact Analysis Not all servers/events are the same 45

47 Real world examples

48 Sample Integrations Customer 1 External facing customer web site Using a different vendor for enterprise monitoring. GBS and IBM software has been brought back in 4 times in 2015 to expand application monitoring. IBM and GBS able to meet and exceed requirements on budget each time Apache, Tomcat, JBOSS, WebSphere, IBM Integration Bus, SAP, and others Customer 2 Internal and External facing web site IBM HTTP Server, WebSphere, and Oracle Synthetic transactions to verify availability, response time, and SLAs 47

49 Sample Integrations Customer 3 Migration from competitor to APM stack Customer 4 Managed Services Provide technical assistance on a case by case basis Move the customer from 80% success to 100% This is one of the most popular GBS offerings 48

50 Sample Integrations Customer 5 Existing IBM customer Assist with deployments of WebSphere Application Server, Jboss, Tomcat, Web Response Time, Synthetic Transactions Integration with ServiceNow Dashboards using Jazz, Omnibus, Impact, and SNOW 49

51 Sample Dashboards 50

52 Agent Builder

53 What is it Eclipse Based tool Wizard Driven Building custom agents that look and feel like IBM agents Create and modify custom agents that extend the monitoring capabilities of an IBM Tivoli Monitoring or IBM Performance Management environment One of my favorite APM features 52

54 Data Sources Process and service availability Network system availability (using ICMP ping) Command return codes Script output Windows Event Log Windows Management Instrumentation (WMI) Windows Performance Monitor (Perfmon) 53

55 Data Sources Simple Network Management Protocol (gets and traps) HTTP Availability and response time SOAP JDBC JMX CIM Log files AIX binary logs (errpt) Socket 54

56 Source Code Do not lose the Source Code Do not lose the Source Code Do not lose the Source Code Do not lose the Source Code 55

57 Questions and Answers

58 Contact Information Phone: Website Expo Booth:

59 Thank You Your Feedback is Important! Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your smartphone, laptop or conference kiosk.