A Ranger4 Guide to. Application Performance Management. Ranger

Transcription

1 A Ranger4 Guide to Application Performance Management Ranger

2 Contents 1.0 What is Application Performance Management? 1.1 APM and DevOps 2.0 Why should you do it? 3.0 What you should look for in a vendor 3.1 End-user experience monitoring 3.2 Application topology discovery and visualization 3.3 User-defined transaction profiling 3.4 Application component deep dive 3.5 IT operations analytics 3.6 Questions to ask 4.0 Considerations at implementation time 4.1 Cultural Change 4.2 Process Change Educate and Evangelise 4.3 SAAS or On-Premise 4.4 Verification 4.5 Discovery and Configuration 4.6 Alerts 5.0 Ranger4 Recommendations Ranger

3 1.0 What is Application Performance Management? Application Performance Management (APM), is the monitoring and managing of performance and availability of software applications. APM strives to detect and diagnose application performance problems to maintain an expected level of service. APM is the translation of IT metrics into business meaning (i.e. value). Five functional dimensions are required for successful APM (as defined by Gartner in the latest 2013 Magic Quadrant for APM): 1. End-user experience monitoring 2. Application topology discovery and visualization 3. User-defined transaction profiling 4. Application component deep dive 5. IT operations analytics 1.1 APM and DevOps The DevOps movement has been born out of the increasing conflict between IT development and operations teams as a result of businesses need to deliver more innovation to their customers via applications and infrastructures of ever-increasing complexity. Where development are all about change, operations are driven to control the stability of the environments they manage and deliver the highest possible uptime to the business - and change puts all of that at risk. APM helps development and operations work better together, collaborate more effectively, as it enables rapid problem analysis allowing the teams to quickly understand where the issue is with the software released and resolve it. Many organisations have typically set up war-rooms when there is a system outage where the team responsible for the application (developers, system administrators, DBAs, architects, security, testing etc) have been caught in a cycle of denial and blame whilst they manically try and identify what broke. The only truly important thing is to have the system running properly again in the shortest possible timeframe, but without effective diagnostic tooling this can be very hard to do. APM accelerates the time to recovery so developers can carry on innovating and operations can achieve system stability again - without wasting time arguing and blaming each other for the problem. Ranger

4 2.0 Why should you do it? High level benefits from using an APM solution include: - Reduced MTTR* - Increased uptime - Reduction in lost transactions/business profits - Improved ability to provide proactive support - Less time spent troubleshooting - More time spent innovating and developing - Fix problems before they impact business - Reduced operational running costs (hardware, support etc) - Improved user experience ** The MTTR is the Mean Time to Repair, Resolve or Resolution - each of the definitions means the same and can be used interchangeably Our customers tell us of scenarios where applications have failed and they first they have known of the outage or performance degradation has been when they have been told by a customer. Not only are they losing money while customers cannot transact, they are losing credibility, reputation and probably losing their customers to their competition. All of our customers are under pressure to continually innovate in order to compete and most businesses we work with are conducting more and more of their business online. Ranger

5 3.0 What you should look for in a vendor At a high level, APM tools must be selected with the five following strategic end goals in mind: 1. Simple to install and use 2. Able to go wide 3. Able to go deep 4. Able to understand business transactions 5. Built for cloud environments Your APM tool must be able to cope with the change your business constantly experiences. Remembering from section 1.0, Gartner s five functional dimensions of APM, let s explore each a little further. 3.1 End-user experience monitoring This is the capture and presentation of data about how end-to-end application availability, browser, network and server latency, execution correctness and quality appear to the end user. IT operations should be able to compare end user experience across geographies and browser types and advise development accordingly. We believe business transactions are the only true measure of application performance and you should look for a tool that provides real-time business transaction monitoring with the capability to measure and track all transactions: - Response Time - Load - Error Rate - Slow Rate - Stalls 3.2 Application topology discovery and visualization You should look for a tool that handles the discovery of the software and hardware infrastructure components executed by the application, and the possible ways in which these components communicate to deliver the application and makes it possible to visualize and manage entire applications. Look for tools that make it possible to auto-discover and map application tiers and services, and see the relationships between them, thus eliminating Ranger

6 blind spots. You tool should enable you to follow the flow of your transactions across all tiers and services (even in a highly distributed environment) so you can identify bottlenecks super fast. 3.3 User-defined transaction profiling The tracing of user-grouped events, which comprise a transaction as they occur within the application as they interact with components discovered in the second dimension (application topology discovery and visualization); this is generated in response to a user's request to the application. 3.4 Application component deep dive Your APM tool of choice should provide fine-grained monitoring of the resources consumed by, and events occurring within, the components discovered in the application topology discovery and visualization dimension. These may include both server and client side components, devices and interfaces. You should want to be able to identify root cause with complete code diagnostics and have full visibility into code execution in production. Your APM tool should add very little overhead and require little or no manual configuration. 3.5 IT operations analytics The burden of overseeing and managing applications has increasingly moved away from application architects and developers. Now, it is IT operations and infrastructure professionals who are often shouldering the responsibility for ensuring application health and performance. This is particularly true with respect to distributed applications. A fundamental change in the way that applications are built, deployed, and scaled has caused web applications to shift from being monolithic in nature to many components distributed over multiple tiers and nodes, all connected through service-oriented architectures. This trend has contributed to IT operations and infrastructure professionals being the ones whose are alerted when applications fail to meet business SLAs. You chosen tool should use a number of techniques including event processing, statistical pattern discovery and recognition, unstructured text indexing, search and inference, Ranger

7 topological analysis, and multidimensional database search and analysis to discover meaningful and actionable patterns in the typically large datasets generated by the first four dimensions of APM. It should enable you to detect business impact and performance spikes and your tool should learn "normal" performance behavior and "normal" code execution paths for all Business Transactions and Application Services, at all hours, days and weeks of the year - so anomalies can be detected automatically. Your tool of choice should ship with pre-built reports for your line of business people, dashboards for the operations teams and have a release comparison feature so that the development teams can see and understand the impact of their releases. 3.6 Questions to ask Here are some questions to ask when talking to APM vendors: 1. What is the installation process up front? 2. What kind of post-install work is required? 3. How does the tool map the application? 4. How does it display that mapping? 5. Does it keep up with the needs of agile development? 6. How much manual instrumentation is needed for complete visibility? 7. Does your tool have insight into the entire distributed environment? 8. Are there any blind spots in your current architecture? 9. If it does map the environment, how does that data come back to you? Is it simple to view? 10. Can your APM tool go deep as well as wide? 11. How does it isolate problem areas at the code level? How much overhead does it add? Less than 2%? 12. How do you define business transactions? 13. How does your tool represent them? 14. How does your tool speak the business language of the application? 15. Does the tool learn and react from performance baselines dynamically? If not, how are they set and maintained? 16. Is your tool built for a cloud environment ground up, or has it been retrofitted? 17. Can the tool manage application performance with respect to virtual and clouds environment? If so, how? How many nodes can the tool monitor in the cloud? 18. How does it adapt to the dynamic nature of the cloud? 19. Can it dynamically provision resources across the cloud? 20. What cloud-ready environments has it monitored in the past? What are the customer names? Ranger

8 Ranger

9 4.0 Considerations at implementation time Implementing APM and DevOps requires change - and change can be painful and is often resisted. Here are some pointers on how to make your APM implementation project go smoothly. 4.1 Cultural Change You ll probably find that the stakeholders in IT and business are delighted to hear that they will have reports available on system performance and you already have a number of evangelists, however, the key to perceived success is often in expectation, so we recommend you have a clear, documented set of requirements upfront from the team describing what they want to see in the reports and how they want them delivered. It s a good idea at this point to have a view on your maturity level when it comes to APM - how are you finding out there s a problem with your apps? Are your customers telling you or are you getting alerts from other tooling? What is the other tooling (make an inventory and classify the existing tools (monitoring for database, network etc) and how many alerts are you getting? How easy is it to interpret them? Who is currently using them and how? What do you like / not like about them? Perform a gap analysis based on the capabilities described in section Process Change It s unlikely your project s going to get off the ground without a solid business case so find and identify one or two (probably business critical) applications that are being significantly impacted by outages / performance issues - if you can calculate the revenue/profits lost during downtime all s the better. Additionally, you will, we hope, have baselined a number of quantifiable key metrics (MTTR, volume/severity of outages, number of people/time spent troubleshooting etc) and as part of your business case and project plan have some stated goals to achieve. Some of these may directly impact your SLAs and require a change in process - certain alerts for example may mean that your developers receive the pager call at 2am to tell them a severity one outage has occurred that requires their immediate attention. Everyone needs to be ready to respond to what your APM system can tell your organization. Here are some questions to ask yourself as you prepare and deploy your APM solution: Have you defined the alerts routing to the people defined in your support and recovery process? Ranger

10 Has everyone who will be receiving or has visibility of the new alerts been made aware of what they are and how to interpret them? If someone hears about your new APM solution and wants to receive alerts from it, how do they request this? What is your process for servicing this request? Is there a process to swiftly on-board a new application? Is the APM tool integrated with other corporate systems? Educate and Evangelise Your vendor may provide training materials, train-the-trainer or onsite/online training courses on your new APM tool. You ll need training material for: - Basic usage - Advanced concepts (memory leaks, policies, dashboard creation, etc) - Operations (alerts/events) training - Reports interpretation for line-of-business and development You need to make sure the people who will touch the product or consume the data have the information they need to be successful. Their success drives your success - you are all in this together. Evangelise your new tool - broadcast your success and, wherever possible, quote quantified metrics (a percentage improvement in uptime, the new average MTTR etc). Your vendor should be able to and want to work with you to monitor, record and socialise the success of the tool. They may even ask you to be a reference - supporting the tools you love helps ensure their longevity and is great for your career development. For every problem you solve with your new APM tool take a few minutes to document the success and make it available to the team and stakeholders. Collect the following information: Problem Description: Which application, what happened, what was the impact? Resolution: What was the root cause and how was it resolved? (Include screenshots) Business Impact: What was the time to resolution and how did that compare to the baselined MTTR? What was the quantified business impact and how did that compare to life before APM? 4.3 SAAS or On-Premise One of the first decisions you ll need to make at implementation (and you ll more than likely have already performed a POT/POC so may already have discussed this at some length with Ranger

11 your chosen vendor) is whether to host your own controller or use the vendor controlled SAAS. It s easier (less work for you) to go the SAAS route since the vendor provides and manages the platform but you may have considerations and requirements (around security in particular) that mean you would rather run the infrastructure yourself. Check your server procurement and set-up processes early if this is the case to avoid your project slowing down. Once the APM solution is up and running it s time to install the agents. 4.4 Verification After you ve deployed your agents (whether straight into production or advancing through your route-to-live) and you have started used the monitored application, you ll want to look at the user interface to see if the information contained within looks correct. Look at your application flow map and try to identify any missing application components Check the business transactions - are all the expected transactions there and reporting metrics? Are your end user experience metrics displaying? Do you have transaction snapshots showing your custom code executing in the run time? Send out test alerts to see if they make it to their destination If things don t look right you ll need to work out why. Maybe your application is different than you had initially conceived, or perhaps there a problem with the monitoring. Resolve any issues you see before declaring deployment and configuration victory. NOTE: Production Load Cannot Be Simulated Exactly To realize the most value from your APM purchase you must run it in production. No matter how good your Quality Engineering team is they cannot possibly code all of the weird and wonderful things your users will try to do in production. It can also be very difficult to duplicate your application environment in production. Example, you have 5000 JVMs spread across multiple cloud provider data centers. Replicating that environment would be time consuming and really expensive. 4.5 Discovery and Configuration If you ve picked the right tool, this bit should be easy. Your APM tool should not require more than a series of simple steps in order to be installed and should be able to perform auto-discovery. It should be up, running, and instrumenting Ranger

12 the distributed application within hours or even minutes. The process looks something like this: the end user installs the agents on all managed servers and virtual machines, installs the controller, and re-starts the application. At that point, the tool itself should be able to handle the challenge of mapping all the databases, tiers, and nodes within the distributed application, as well as displaying those relationships in an intuitive and visual way. 4.6 Alerts Do you like alerts? Not many people do. Why? Because it s difficult to set meaningful ones and ones that preempt real system issues. When you re setting your alerts, set your mind to thinking about business impact - not IT infrastructure resource utilisation. The first question to ask is: How can alerting be done the right way without spending more time and money than it costs to develop and run my applications? Your most important and actionable alerts should be based off metrics that are directly associated with business impact. Here are some examples: End user response time (good indicator of regional issues) Business transaction response time (good indicator of systemic issues) Business transaction throughput rate (are we seeing the same amount of traffic as usual?) Number of widgets sold (is there a problem preventing users from buying?) And remember, you might not always get all of this right first time and what s right today might not be right tomorrow. But your APM tool, used properly, will enable you to deliver higher quality innovation, faster. Ranger

13 5.0 Ranger4 Recommendations At Ranger4 we recommend AppDynamics for APM. In Gartner s 2013 Magic Quadrant (released December 2013) AppDynamics was recognised as a leading vendor in this market with the following strengths highlighted: - AppDynamics delivers analytics-driven root cause analysis - Corrective actions can be applied through run book automation assuring high availability - AppDynamics has capability across all 5 dimensions and DB monitoring too - A powerful visual user interface We also like Splunk for additional log-file analysis and for use with Big Data to make value-based business transformation decisions. We can make recommendations for you based on what we learn about your profile and work through a proof of technology or concept with you. We are also well-practised in writing business cases with our customers wherever they start on the maturity scale and measuring progress through and beyond the project. We often find that APM is one solution of several that will help an organisation benefit from a full-blown DevOps implementation and have developed our DevOps Maturity Assessment which baselines and scores an organization s current capabilities on our DevOps Maturity Index and provides a fit-assessed roadmap to a desired future state. To find out more, please visit Ranger