Measuring the End-User Experience of Your Enterprise Applications. Jeff Budge OneNeck IT Solutions

Transcription

1 Measuring the End-User Experience of Your Enterprise Applications Jeff Budge OneNeck IT Solutions

2 Agenda Entering the Application Performance Monitoring (APM) space Expose the limitations of Data Center monitoring to measure end user experience. Discuss the impact of application complexity towards application level monitoring. Getting Our Arms Around APM What is meant by the term APM. The value and return of APM. POC: Putting APM into Action What did we do concerning APM? What did we learn? 2

3 About the Speaker: Jeff Director of Enterprise Applications Strategy Prior to OneNeck: Over 20 years of enterprise applications and technology solutions experience, with a career-long focus on Oracle. Global CTO for CSC s Oracle Implementation Practice, Global Portfolio Lead for Oracle platform, virtualization, and technology North American CTO for Deloitte s Oracle ERP Implementation Practice Director of the Oracle Center of Excellence at Computer Data Systems, Inc. Multiple certifications across Oracle s hardware, software and business application spaces Bachelors in Electrical and Computer Engineering from the University of Tennessee, Knoxville. 3

4 The Beginning Molly is an experienced E-Business Suite (EBS) super user, and acts as the Liaison between the Supply Chain team and IT for her company. Today, she received a call from one of her users. Apparently the very critical EBS MRP runs are processing slowly causing cascading forecasting delays and potential big $ impact. She promises the user (her customer) to get on it quickly, and hurries off to talk to the experts, and to find out what their monitoring tools tell them is causing the delays. 4

5 Let s Check In with The Server Team How are the SERVERS?

6 The Server Team Says My boxes are pristine, finely tuned machines. Check with the Network guys 6

7 Hmmm 7

8 What Does the Network Team Think? Nope, my SLAs are all good and its PINGABLE this has to be DBA 8

9 What Do You Say DBA? Are you kidding me? I just ran stats, plus I have no blocking locks I told storage my I/O is a problem! 9

10 Uh Oh Are you questioning my SAN?? 10

11 Obviously, this is not the insightful, collaborative response she expected from the team. After all they have the most visibility and access into the environment! 11

12 So What is Molly s Response? Although she sees clearly that our overall monitoring was incapable of proactively or reactively helping with this issue, her hands are tied with the information she received from her team. She has no choice but to go back to her user with a Thumbs Up! 12

13 Molly s Customer is Not Impressed. 13

14 Molly s Monitoring Solution Failed. But Why?

15 Before: Simple Mainframe/Client-Server Architecture User Tech Stack Close proximity between user and monitor Administrator 15

16 Now: Complex, Highly Distributed Architecture User Distance between user & monitor is wide Tech Stack Administration Teams 16

17 The Logical Complexity of Applications 17

18 Communication Across Silos Before: Not uncommon that one person did it all. They were the IT department and could holistically piece it all together. Now: Teams (often very large) are now clearly segregated and often leverage providers or contractors. What does this mean? They don t easily see the other s domain They (often) don t care much about the other s domain They don t know how to interpret results from other domains. Result = Minimal, if any, communication lines and therefore severe limitation to solving problems. 18

19 Traditional Metrics and SLAs The traditional UPTIME Service Level Agreement (SLA) provides almost no value. SLAs should represent accountability towards the availability and performance of the Application and Functions within. Quality of Service (QOS) reporting should be application function/transaction centric. Component level metrics are still needed, but more for activities like capacity planning. 19

20 Enter Application Performance Monitoring. But What Is It?

21 Application Performance Monitoring (APM) APM is the translation of IT metrics into business meaning/value Larry Dragich, April 2012

22 Gartner s 5 Dimensions of APM End-user experience monitoring (EUM) the capture of data about how the end-to-end latency, execution correctness and quality appeared to the real user of the application. A secondary focus on application availability may be accomplished by synthetic transactions simulating the end user. Application topology discovery and visualization- the discovery of the software and hardware infrastructure components involved in application execution, and the array of possible paths across which these components communicate to deliver the application. User-defined transaction polling the tracing of user-grouped events, which comprise a transaction as they occur within the application as they interacts with components discovered in the second dimension; this is generated in response to a user s request to the application. Applications component deep dive the fine-grained monitoring of resources consumed and events occurring within the components discovered in the application discovery and visualization dimension. This includes server-side components of software being executed. IT Operational Analytics (ITOA) the combination or usage of the following techniques: 1. Complex operations event processing 2. Statistical pattern discovery and recognition 3. Topological Analysis 4. Multidimensional database search and analysis Magic Quadrant for Application Performance Monitoring, October 2014 Gartner, Inc. 22

23 Additional APM Benefits and Value Single Source of the Truth Drill down / consolidate up / spin sideways Session forensics the end user session DVR Baselining and meaningful SLA definition 23

24 Learnings from an APM Proof of Concept

25 OneNeck IT Solutions at a Glance We provide end-to-end IT Solutions to mid-market and enterprise companies Solution line up includes: cloud and hosting solutions, colocation services, managed services, ERP application management, professional services, IT hardware resale and world-class data center facilities Backed by the Fortune 500 strength of Telephone and Data Systems 650+ Employees 450+ Technical Resources 7 Data Centers Proven client focus Headquartered in Scottsdale, Arizona 25

26 Our Oracle Capabilities Host, manage, upgrade and maintain Oracle applications including E-Business Suite, JD Edwards, Hyperion and OBIEE Hosting Oracle since 1998; Currently manage more than 15,000 Oracle end users in 39 different countries End-to-End Oracle EBS 11i/12 hosting, administration, technical and functional support Certified, world-class Application DBAs and technical resources 26

27 Long Term Goals OneNeck has targeted 6 key application platforms: 1. Oracle E-Business Suite ( and later versions) 2. Microsoft Dynamics AX (AX2009(5.0) or later) 3. Oracle Demantra ( and later) 4. Oracle Agile PLM (PLM for Process v6.1 or later) 5. SAP (ECC6.0 release pack 4 and later) 6. Planview PPM Oracle Database and Microsoft SQL Server are in scope given their role in the above applications Red Hat Linux, Oracle Linux, Oracle Solaris and Microsoft Windows Server are the in scope operating systems 27

28 POC: Scope Short Term Focus The initial POC was focused on proving each vendor s solution against a single application environment: Oracle E-Business Suite Oracle Database Oracle ias rollup 5 Database tier O/S: Solaris bit Application/Web tier O/S: Red Hat Linux Release 4 Update 8 32-bit 28

29 POC: Vendors Evaluated Computer Associates APM (Wily Introscope and Nimsoft Monitor) Dell Software (formerly Quest Software) Foglight Idera/Precise Software Precise TPM (Transaction Performance Management) Oracle Enterprise Manager 12c, Application Management Suite for EBS and Real User Experience Insight Riverbed/Opnet AppInternals Xpert and AppResponse Xpert 29

30 POC: Approach Six month evaluation process including vendor on-site engagements. Applied against a meaningful customer test environment with production-similar architecture. Created weighted detailed scoring criteria according to APM definitions for objective evaluation. Held frequent checkpoints to evaluate vendor/tool progress 30

31 POC: Findings The majority of tools employed an architecture which includes: Packet capturing span port appliance Server deployed agents/collectors Data repository and management server Agents required moderate configuration for instrumentation Tools generally fell into 2 camps: Generic, web-application framework (JEE,.NET) centric Application and supporting technology focused. 31

32 POC: Findings (Continued) No native support for proprietary, higher-port technologies such as Oracle Forms. All tools provided both active (synthetic) and passive transaction monitoring. Tools varied in the representation of EUE (total round trip time) to identify infrastructure bottlenecks. Reporting and publishing capabilities varied greatly amongst vendors. 32

33 POC: Considerations The cost of tool overhead needs to be factored into the equation. 3% is a common value we heard. Security considerations are important; especially in multi-customer environments Packet capture appliances see ALL traffic Data at rest and in transit implications Don t neglect the work to define a proper governance model. APM tools handled traditionally = Component Monitoring. Proprietary protocols/ports not readily supported. Result is gaps in the overall picture. Need to compensate for this. 33

34 POC: Considerations (Continued) Almost always a cultural shift will be required Sponsorship and evangelization is needed at many levels Not everyone will be convinced initially Don t take on too much all at once POC specific: Don t underestimate the commission/decommission activity Be prepared to work through CM processes to account for risk Choose your target environment carefully. Balance is needed between having a sufficient transaction footprint and introduced risk. 34

35 POC: The Results Your Mileage May Vary Identify, Revisit and Reconfirm your APM priorities What is most important to your company? End User Experience Business Event Correlated Tracing Broad and Deep Event Analysis Define your own selection criteria and weighting Perform sufficient due diligence, including cultural and technology fit We chose Dell Software (formerly Quest Software) Foglight Our initial technical implementation is complete, we are in the process of identifying the true benefits in multiple domains (IT operations, process specific, business level, etc.) We plan to present updates on our implementation at conferences later this year. 35

36 Thank You 36

37 Citations Gartner s 5 Dimensions of APM Full Slide Jonah Kowall, Will Cappelli Gartner Magic Quadrant for Application Performance Monitoring, 28 Oct 2014 The Logical Complexity of Applications Diagram Don Jones The Five Essential Elements of Application Performance Monitoring Defining APM Quoted Definition Larry Dragich The Anatomy of APM; APM Digest April

38 Appendix

39 OneNeck IT Solutions at a Glance We provide end-to-end IT Solutions to mid-market and enterprise companies Solution line up includes: cloud and hosting solutions, colocation services, managed services, ERP application management, professional services, IT hardware resale and world-class data center facilities Backed by the Fortune 500 strength of Telephone and Data Systems 650+ Employees 450+ Technical Resources 7 Data Centers Proven client focus Headquartered in Scottsdale, Arizona 39

40 Data Center and Office Locations 40

41 Our Solution Set Cloud & Hosting Solutions Managed Services ERP Application Management Professional Services IT Hardware Resale Cloud Servers Private Clouds Hybrid Clouds Cloud Storage Desktops in the Cloud (DaaS) Colocation Managed Applications Managed Databases Managed Networks Managed Servers End User Support Disaster Recovery as a Service (DRaaS) Oracle Microsoft Infor SAP IT Assessments Design Migrations & Implementations IT Roadmaps and Planning Technology Consulting Cisco EMC HP VMware Citrix F5 NetApp Security & Compliance Communication & Collaboration 41

42 Recognition We have a long history of being recognized for significant achievements Gartner, Inc. OneNeck IT Solutions named to the Visionaries quadrant in Magic Quadrant for Oracle Outsourcing, North America CRN s Tech Elite OneNeck IT Solutions named to CRN s 2012 List of Tech Elite 250 J.D. Power and Associates OneNeck IT Solutions named J.D. Power and Associates 2011 Customer Service Champions The Black Book of Outsourcing OneNeck IT Solutions selected No. 1 ERP Management Outsourcer, No. 1 Mid-Tier Service Provider of Data Center Management and one of the Top 50 global outsourcing service providers 42

43 Customer Landscape Delighting Customers on Six Continents 250+ Colocation Customers 300+ Cloud Customers 75+ ERP Application Management Customers 500+ Hardware resale and professional services customers 15,000+ Devices Under Managed Services alone 43

44 Our Value Proposition Our core competencies, financial strength, and strong customer service heritage positions OneNeck IT Solutions like no other. OneNeck IT Solutions Financial strength of a Fortune 500 company Capital resources available for technology investments Local and regional focus to emphasize customer intimacy Our commitment to service excellence Customer Benefit This creates vendor stability and dependability for you Stay current with a rapidly evolving marketplace We will know your business and its requirements Meeting our commitments to you is in our DNA A recognized culture of trust, integrity, service discipline, and process orientation 44

45 What does OneNeck IT Solutions stand for? Our Purpose: To deliver world-class enterprise IT solutions with unrivaled service. Our Mission: At OneNeck IT Solutions we live our name. That is, we provide our own combination of cleverness and serious expertise in the delivery of IT solutions that fit each and every customer s needs. We operate as an extension of our customer s teams, anticipating their requirements. This is what makes us trusted, genuine and approachable in an industry that can be cold or calculated. We take pride in knowing what is best for our customers, often before they do. We are about providing business and technology solutions, being a trusted partner along the journey, and injecting a great deal of ourselves in the process. Because we understand what s at stake. That means putting our neck on the line every day. World Class Expertise Trusted Partner Outstanding Service Our tone: we speak with knowledge and expertise never preachy or above-it-all always approachable, down-to-earth, respectful and trusted. We add value by solving IT issues, not by promoting our own products. We are the skilled companion customers rely on and like to work with. 45

46 Our Parent Company- TDS Telephone and Data Systems, Inc. [NYSE: TDS], provides wireless; cable and wireline broadband, TV and voice; and hosted and managed services to approximately six million customers nationwide through its business units, TDS Telecommunications Corp., U.S. Cellular, OneNeck IT Solutions LLC, and TDS Baja Broadband LLC Founded in ,700+ employees $5.3B in revenue A Fortune 500 company A focus on long-term value creation Strong balance sheet, ample liquidity and investment-grade debt ratings 46

47 Why are we different? We Have an Impeccable Track Record of Satisfied Clients Since 1997 we started providing outsourcing services in 1997, we maintain a 98% contract renewal rate compared to an outsourcing industry average of 85% We are Highly Flexible We support a variety of hardware platforms, connectivity, and bandwidth options to provide a tailored environment for our clients We host/manage many 3 rd party applications such as: , CRM, SCM, Reporting We offer holistic DR solutions We manage desktops, phone systems, mobile devices, and other infrastructure We are Acutely Responsive We have strict incent responsiveness SLAs We are an extension of your IT team We are a Customer Intimate Organization We understand the Principles of Outsourcing Success.it s woven into the fabric of OneNeck s existence. We practice the principles daily. We are focused Providing hosting and managed services is all that we do

48 Our History 48