Mobile application performance management

Transcription

1 Mobile application performance management Xavier Chevalier Solution Consultant Benelux March, 2014 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

2 Start 60B Apps downloaded from Apple AppStore The mobile revolution 25% By 2017, enterprises will have an enterprise app store 4:1 By 2016, mobile app projects will outnumber traditional app projects

3 The Mobile revolution 3

4 A poor app gets worse by Srma131 No longer able to book flights on the update. No support for this app by alicereed Airline does not have any support for this app for issues or improvement suggestions. Still can't view my insurance card by JJR27 Would be a neat feature but after a lot of time they never fix it. So pretty useless app. Poor App Ratings 4

5 How can end user experience be assured in a highly fragmented Mobile value delivery chain? OS Browser Device Carrier Back-end Increased Complexity! 5

6 Visualize Isolate Predict Mobile performance issues Treat your app like a V I P How to get there! 6

7 Visualize App performance for best mobile end user experience Across any mobile app type Real User Monitoring and Proactive Monitoring 360 view across end users and infrastructure 7

8 Isolate Isolate mobile app performance issues To a specific app version, device, carrier, etc.. To a specific user flow all the way to the line of code in the backend To a specific 3 rd party service 8

9 Predict Predictive analytics for mobile app performance Early warnings for service outages Early catch of issues (e.g.: based on volume drop) No manual thresholds required 9

10 APM Mobile solution overview

11 HP provides insight into the entire mobile value chain Treat your app like a VIP monitor it end to end! The HP solution Web Native Hybrid 11

12 Collect data from a robot executing scripts emulator Real user monitors Visualize and isolate enduser experience Collect end user experience data from the mobile device Collect application data from the network HP Business Process Monitor HP Real User Monitor 12

13 Web Server App Server Isolate with HP Diagnostics Pinpoint root cause of poor mobile application performance within your application Monitor transaction latency Isolate issues across app servers, databases & middleware 13

14 Constantly compare expected and actual user behavior Predictive analytics Alert on a performance regression after a new release Catch unexpected declines in typical user activity 14

15 APM Mobile where to begin?

16 Mobile app monitoring maturity model Proactive monitoring only Advanced analytics Diagnostics RUM Value Helps determine baseline performance for fixed transactions Proactive monitoring to find issues even if real users are not accessing the app Leverage VuGen scripts from dev/test to get you started quickly. BPM Maturity 16

17 APM Mobile BPM overview

18 BPM architectural components NOTE: monitored apps do not have to be inside your data-center. BPM can monitor your cloud service providers or your apps hosted in public, hybrid clouds. 18

19 Native mobile apps recording methods Capture File Source Advantages Disadvantages Mobile device VuGen with configured hot spot wireless access point Server with cellular network card Mobile device emulator Record the real traffic that comes from the device Easy and intuitive Admin permissions not required Record real traffic (both WiFi and cellular network traffic) No device required No 3G needed Requires administrator permissions or rooted device Not all devices support ad-hoc wireless access points. Captures WiFi only (does not capture cellular network traffic) Requires administrator permissions Requires installation of software on the server Requires locating and installing device emulator 19

20 TruClient mobile recorder 20

21 Record scripts, configure schedule/locations 21

22 Automatic distribution of scripts and schedule 22

23 Monitor from your data center 23

24 Monitor from an external location 24

25 Visualize, isolate and predict! 25

26 Visualize with HP BPM and Service Level Management 26

27 Isolate mobile carrier related issues Identify specific mobile carrier issues using BPM agents on machines with 3G/4G card connecting using different mobile carriers 27

28 Mobile app monitoring maturity model Proactive + real user monitoring Advanced analytics Diagnostics RUM BPM Incremental value Capture performance and availability of your app for real users Supports BYOD initiatives Isolate issues based on carrier, device type, OS version and app version Provide real user flows to dev/test Capture 3 rd party API/URL impact Maturity 28

29 APM Mobile RUM overview

30 Gateway Firewall EC2 Gateway Firewall Mobile RUM architecture Client Monitor Probe only DMZ Mobile gateway Web server Optional cloud deployment RUM CM probe RUM CM probe RUM engine BSM Client Monitor and Network Probe 3 rd party cloud services DMZ Mobile Gateway Web server Duplicated traffic App servers RUM data (encrypted) RUM CM probe RUM NW probe RUM engine BSM 30

31 Real user monitoring for mobile apps Collect Mobile App performance from native and hybrid Apps running on any Android or ios mobile device Instrument in 5 min No code change required Collect all HTTP network traffic, all the time: TX response times, errors, events, volume 31

32 No code changes required Post-build instrumentation Two click Android instrumentation App from Google Play, instrument it No developers intervention required Applicable for both native and hybrid apps Supports Android 2.2 and up 32

33 No code changes required Simple ios instrumentation Requires developer/build manager to add 3 frameworks and a properties file to the project s build (also hosted on CocoaPods) Applicable for both native and hybrid Supports ios 5 and up 33

34 RUM Client Monitor Probe Target app Strengths Considerations Browser based apps Instrumented native mobile apps (Android/ ios) Captures full user experience. Includes measurements for all 3 rd party URL s. No data center side network devices required. Captures browser type and OS name. Captures full user experience. Includes measurements for all 3rd party URL s. No data center side network devices required. Captures WiFi, mobile carrier connection details, device type, device OS version, native app version. Sees virtual public IP end-points, not backend servers IPs. Minor change required to web-application to inject JavaScript instrumentation. Single tier (end-user) perspective. Each probe supports 2k pages per second. Sees virtual public IP end-points, not backend servers IPs. Requires instrumenting native application (no coding!). Single tier (end-user) perspective. Each probe supports 2k pages per second. 34

35 RUM Network sniffer Probe Strengths Captures full request/response contents and actual servers serving application. Captures TCP/IP level detail for network health. Supports DB, web services, streaming apps and other non-web protocols including extending for custom protocols. Captures performance at multiple tiers (front-end, load balancer, back end servers, DB and other back end resources) Easy to configure alerts based on full user session data. Considerations Requires network tap or switch SPAN port and access to the actual network infrastructure in the data center. Captures the server side times not the full browser rendering time nor any 3 rd party URLS used by the application. Probe capacity varies based of usage of features like pattern matching, SSL, snapshot on error. 35

36 Visualize with HP Real User Monitor - locations Gain visibility to mobile app performance & user activity Understand usage and experience from different geographies 36

37 Visualize with HP Real User Monitor app versions Visualize app version adoption and regressions 37

38 Visualize with HP Real User Monitor device and OS Surviving the mobile continuous delivery release cycles New devices and OS versions are released frequently, making it impossible to track and test everything understand how your application is keeping up with change 38

39 Best of both worlds - combine both probe types Get end-to-end visibility by deploying both probe types Shown here : Mobile device App server Database 39

40 User Flow Analytics Link to auto-develop scripts and track common pathways as transactions Most common pathways 40

41 Analytics - RUM User Flows Analytics Target customers are the existing APM customers. Different use case per persona. Business/Marketing (App Owner, Architect) Monitoring (App support, Monitoring team) Testing (PCOE) PAL Understanding the common patterns of user behavior in the site Discovering most common user transactions so they can be defined as business transaction for monitoring. Generating load scenarios based on real user flows from production 41

42 Mobile app monitoring maturity model Proactive + real user monitoring + deep diagnostics Advanced analytics Diagnostics RUM BPM Incremental value Easily isolate end users problems caused by backend issues Isolate issues if within your complex app farm Isolate to a specific server, app instance, SQL query, web service and line of code Fully integrated with BPM/RUM for end to end visibility Maturity 42

43 APM Mobile Diagnostics overview

44 A very simplified view of your complex apps 44

45 No RDBMS Software Required Diagnostics deployment architecture BSM Servers Diagnostics Commander HP Diagnostics Agent Collection of metrics, call profiles and topology from ASP.NET apps deployed to IIS HP Diagnostics Server HTTP/HTTPs App Specific Protocol Collection of metrics, call profiles and topology from Java,.NET and Python based apps Diagnostics Mediator(s) Diagnostics Collector(s) Agentless collection of metrics and topology from MS SQL, Oracle DB/RAC, SAP ABAP, WebMethods, IBM MQ Series, Tibco EMS. VMWare vsphere 45

46 Isolate with HP Diagnostics Quickly identify problematic application tier where was most time spent? 46

47 Isolate with HP Real User Monitor and HP Diagnostics Detect end users experiencing low performance via HP RUM and pinpoint root cause with HP Diagnostics 47

48 Mobile app monitoring maturity model Ultimate monitoring leveraging advanced predictive analytics Advanced analytics Diagnostics RUM BPM Incremental value No manual setting of alert trigger thresholds Detect issues earlier and correct them before impacting the business. Alert on anomalies like: Performance regression after a new release Unexpected decline in usage activity Maturity 48

49 APM Mobile Predictive Analytics overview

50 How Service Health Analytics works Collects Data Creates Baselines Detects Anomalies Sends Event Investigate Remediate Adaptive Baseline detect metrics seasonality, trend and create dynamic baselines Topology Analysis correlate metrics of related CIs across domains Temporal Analysis differentiate between spikes and real anomalies Historical Analysis compare with similar anomalies from the past Statistical Learning learn normal system behavior and suppress noise Statistical Self Learning Historical Analysis Baseline SHA Runtime Core Temporal Analysis Topology Analysis 50

51 Feature: adaptive baselines Automatically detect trend and seasonality On-going learning to adapt to changing conditions Transaction response time and its expected baseline sleeve over 2 days 51

52 Thank you

53 APM Mobile sample use cases

54 Visualize with HP Real User Monitor - locations Gain visibility to mobile app performance & user activity Understand usage and experience from different geographies 54 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

55 Visualize with HP Real User Monitor app versions Visualize app version adoption and regressions 55 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

56 Visualize with HP Real User Monitor device and OS Surviving the mobile continuous delivery release cycles New devices and OS versions are released frequently, making it impossible to track and test everything understand how your application is keeping up with change 56 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

57 Visualize with HP Real User Monitor segment your users Surviving the mobile continuous delivery release cycles Understand Understand the the exact exact profile profile your of your users users in order in order to design, to design, develop develop and and test what test matters according most to real life scenarios. Device OS Version Application Version Mobile Carrier Connection Type 57 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

58 Isolate with HP Real User Monitor ext service provider How does 3 rd party service providers affect your mobile app performance? Unlock performance of all external cloud services your apps depend upon Is it an external performance issue or is the issue within your domain? 58 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

59 Isolate with HP Real User Monitor full user session Detect and isolate Find a specific user session and investigate what yielded bad experience 59 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

60 Visualize with HP Real User Monitor detect & alert Real-time detection Receive real time alerts once Response time rises Availability drops Application experiences abnormal volume 60 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

61 APM Mobile BPM use cases

62 Visualize with HP BPM and Service Level Management 62 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

63 Isolate mobile carrier related issues Identify specific mobile carrier issues using BPM agents on machines with 3G/4G card connecting using different mobile carriers 63 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

64 HP SaaS over-the-air (OTA) mobile monitoring Insight into wireless / carrier performance & availability Problem it solves: Root cause analysis is it the app or the wireless service? Visibility into mobile end user performance & availability Time to value and resource constraints in IT operations Extends on-premise APM environment on mobile and to external locations Mobile Network Response Time & Availability Data Center Benefits: Visibility into mobile app performance broken out by location, carrier & transaction, e.g.; log-in, transfer funds; Lower mean to isolation quick root cause analysis Insight in minutes via HP SaaS No Capex investment required 64 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Restricted.

65 BPM over-the-air performance monitoring Isolate performance problems WAN vs. wireless 65 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

66 BPM over-the-air availability monitoring Proactively monitor the mobile end-user experience Identify top problems based on transaction location 66

67 APM Mobile Diagnostics use cases

68 Visualize with HP Diagnostics Assure legacy backend succeeds serving a growing number of mobile users accessing it in different usage patterns compared to traditional applications - frequent logins, shorter sessions = load bursts 68

69 Isolate with HP Diagnostics Quickly identify problematic application tier where was most time spent? 69

70 Isolate with HP Real User Monitor and HP Diagnostics Detect end users experiencing low performance via HP RUM and pinpoint root cause with HP Diagnostics 70

71 APM Mobile customer references

72 What our customers say We use both HP BPM and RUM to monitor our mobile applications. Using those two applications we can monitor and track our applications from end-users perspective. Hakan Karamanli Operations and Customer Service Manager Garanti Bank Goal! a quarter (25%) of our revenue is driven through mobile we can identify exactly what the performance is based on devices. Victor Miller Principal Architect for Tools and Monitoring, United Airlines 72

73 Listen to Garanti Bank Listen to United Airlines 73

74 Improving mobile app experience Health Care Provider with 2m members Providing better mobile app experience Powered by the HP Software HP Real User Monitor Mobile Experience Monitoring Availability improved from 80% to 99.61% Capacity planning process improved Better user segmentation DevOps cycle shortened 74 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

75 Thank you! Questions? Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

76 End-to-end mobile performance management Mobile app performance, availability, user analytics App Carrier Service Web Native Hybrid Baseline real user performance; Assure and protect the brand equity. Performance views by time, place, carrier & transaction; Lower mean to isolation, insight in minutes. Understand transaction popularity and response times; Dev/Ops script reuse. Proactive (synthetic) 76 Real user

77 HP Business Process Monitor Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Restricted.

78 New mobile recording capabilities in LoadRunner/Vugen HP LoadRunner Mobile Recorder app Android app available from Google Play Store Enables recording of all network traffic of your native, browser-based or hybrid mobile web applications to a capture file. The file is then used to generate a LoadRunner script Creates a.lrpcap file on the device which can be ed to the machine running Vugen Can be recorded on a 3G or WiFi interfaced device Scripts can be run inside or outside your network, from your own and/or HP hosted locations Works on any rooted device

79 New mobile recording capabilities in LoadRunner/Vugen Proxy Recording Mode No need to install anything on the device Can be used for desktop O/S that do not support Vugen e.g. Linux, Mac etc. Works for web and native mobile applications that use HTTP(s) as the transport How to get started? Select the new option Proxy Recording when start recording Configure your mobile device to use the Vugen recording proxy Open the native app/browser on the device to access the application and all web traffic will be recorded Turn off the proxy setting to stop sending traffic to Vugen

80 Visualize with HP Business Process Monitor Mobile BPM Architecture Option 1 80

81 Visualize with HP Business Process Monitor Mobile BPM Architecture Option 2 81

82 Visualize with HP Business Process Monitor Mobile BPM Architecture Option 3 BPM (Wireless) Wireless Carrier/ISP BPM location can be configured with access to a Wireless Carrier (with a local 3G card for example) Internet BPM (WAN) Script can be replayed over Internet from local location or remote location. Ideally, network throughput is limited in run-time settings. Business Application BPM (LAN) 82

83 Visualize with HP Business Process Monitor Mobile BPM Scripting Mobile TruClient for Web App emulate mobile browser using Firefox, get an approximation of end to end user experience. Simulate device by setting User- Agent, connection settings and screen size. -> Support web applications recording only Mobile App (HTTP/HTML) transport layer emulation, get an insight into server performance. Create VuGen script from packet capture (PCAP) files, emulator recording, or new proxy recording method to support secure communication -> Support native & web application 83

84 Visualize with HP Business Process Monitor Mobile BPM Scripting Monitoring solution Protocols BPM installation Real wireless carrier network monitoring Mobile TruClient Mobile App (HTTP/HTML) Add to the BPM machine a 3G card to emulate real carrier network connection User OTA monitoring provided by Performance anywhere SaaS service Proactive backend (over the WAN) side Monitoring MobileTruClient Mobile App (HTTP/HTML) BPM inside monitoring, and over the WAN monitoring 84

85 Mobile TruClient network diagram Script Recorder VuGen Installed use TruClient Protocol Script is recorded here. Internet 1) Install VuGen as normal on a supported platform. 2) Record VuGen script using Mobile TruClient Protocol after specifying the proper mobile device settings. Business Application Servers 85

86 Proxy recording for a mobile device application Wireless Carrier/ISP 1) Configure your mobile device to use the VuGen Recording Proxy Proxy Internet Full VuGen Script Recorder Business Application Servers NOTE : The device and Vugen machine must be within the same network 86

87 Recording with a device emulator (Android only) Device Emulator Internet Script Recorder VuGen * Business Application Servers 1) pcap file is captured here from traffic generated by a Mobile Device emulator. 2) VuGen script is then generated here. * - In the case of an iphone, the Mac-based device emulator can be used with the VuGen Sniffing agent directly (installed on MacOS) to create a recording without the need to create a pcap file. 87

88 HP Real User Monitor Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Restricted.

89 Real User Monitoring for mobile, cloud and web All Apps, All Locations, All the Time - Single Data Store Data Center Network Probe Server Collector Client Mobile Apps RUM Browser Hybrid Network Real User Monitor Browser Real User Monitor Browser Real User Monitor Server Collects data from network, measure inside data center, monitor frontend and backend tiers Mobile Collect data from client, No Network settings required! Mobile Server Network Combines data from client and network for true end to end visibility Network All Apps Discovery Mobile & Web Apps User Analytics End to End User Experience 89 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Restricted.

90 VDI Mail Backend /Middleware Frontend Various Increasing real-time end-user coverage RUM Protocols, Templates and Protocol SDK Application Protocol Application Protocol Custom web Flash Flex SAP Oracle Forms NCA Siebel HTTP/HTTPs AMF0, AMF3 HTTP/S, SAP GUI Over HTTP/TCP HTTP/S FTP LDAP ISO 8583 FTP LDAP VISA Base I, MasterCard CIS PeopleSoft HTTP/s SOAP IBM WebSphere MQ HTTP/s MQ Streaming/VoIP UDP Oracle DB Oracle DB.NET WCF MS SQL MS SQL MySQL MySQL IMAP IMAP DB2 DB2 SMTP POP3 SMTP POP3 Citrix Xen Citrix ICA 90

91 Feature: client monitoring and web tier breakdown Web Tier Breakdown feeding from multiple network perspectives Send Message response time is broken by time spent in each tier 91

92 HP Diagnostics

93 Isolate with HP BPM and HP Diagnostics Detect and Isolate Investigate server performance problem detected from a threshold breached for a synthetic user transaction 95

94 Isolate with HP Real User Monitor network sniffer Detect and Isolate Expand isolation span and capabilities by monitoring not only the application running on the mobile device but also the web and backend tiers of the application using RUM network sniffer 96

95 Isolate with HP Diagnostics Detect and Isolate 97

96 HP Service Health Analyzer

97 Feature: adaptive baselines Automatically detect trend and seasonality On-going learning to adapt to changing conditions Transaction response time and its expected baseline sleeve over 2 days 99 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

98 Feature: anomaly detection Metrics from different monitoring sources. Users can drag and drop in chart for more details Metric mini charts allow to visually correlate behaviours 100 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. SHA Anomaly timeframe

99 Feature: anomaly investigation UI View Service topology as captured by RTSM View planned and discovered changes (before and after the anomaly started) Reveal additional related CIs outside of the applications service model View abnormal metrics, events and planned downtime windows Observe metric behavior vs. expected baseline Visually or automatically correlate metric trends to help pinpoint problems Get to the root cause faster! 101 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

100 Online gaming provider: using SHA for business metrics SHA detected an issue 1 hour before it was reported by an end user 102 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.