Mobile Performance Management Tools Prasanna Gawade, Infosys April 2014

Transcription

1 Mobile Performance Management Tools Prasanna Gawade, Infosys April 2014 Computer Measurement Group, India 1

2 Contents Introduction Mobile Performance Optimization Developer Tools Purpose and Overview Mobile Web Performance Best Practices Verification Tools Mobile Web Performance Debugging Tools Android Performance Debugging Tools Android Memory Profiling Tools ios Performance Debugging and Memory Profiling Tools Windows Performance Debugging and Memory Profiling Tools Mobile Performance Optimization Techniques and Tools Mobile Debugging Tools Mobile Performance Optimization Developer Tools v/s Technology View Mobile Performance Testing Tools Mobile Application Performance Management Tools Computer Measurement Group, India 2

3 Introduction GOAL - Assess & identify best-in-class tools that measure & optimize performance of online mobile applications -for increased customer satisfaction, and superior delivery on devices & the network. Mobile Performance Optimization Developer Tools Performance Testing Tools Application Performance Management Tools to assist in developing & fine-tuning optimized mobile apps Tools that assess performance from a userexperience point of view (e.g., round-trip for X function takes Y seconds). Instrumentation and Monitoring to assess application performance at a granular level to deduce bottlenecks, trends & areas of focus Computer Measurement Group, India 3

4 Mobile Performance Optimization Developer Tools Computer Measurement Group, India 4

5 Purpose and Overview Tools to assist in developing & fine-tuning optimized mobile apps Check Adherence to Performance Best Practices Ensure that the application meets the stated performance goals Identify and resolve performance bottlenecks Implement Performance Optimization techniques Generate actionable feedback to improve performance Computer Measurement Group, India 5

6 Mobile Web Performance Best Practices Verification Tools Tools that help you speed up your mobile site s page load times by showing you exactly what s slowing them down DEVICE & BROWSER SUPPORT YSlow YSlow is available as a browser plugin in Internet explorer, Chrome, Firefox, Safari and Opera. It is also available as a bookmarklet which is ideal for analyzing mobile sites. PageSpeed PageSpeed Plugins are available for chrome and Firefox. Page Speed Insights allows you to easily analyze and optimize your site for mobile performance. KEY BENEFITS YSlow grades web page based on one of three predefined ruleset or a user-defined ruleset. It offers suggestions for improving the page s performance. When you profile a web page with Page Speed, it evaluates the page s conformance to a number of different rules. These rules are general front-end best practices you can apply at any stage of web development. Page Speed and YSlow generally offer the same service. Each service analyzes a page using a set of rules that they believe are most relevant to page speed and performance. Its best to run both tools in conjunction and take the recommendations that will work best with your site design. Computer Measurement Group, India 6

7 Mobile Web Performance Debugging Tools Tools to assist in detecting performance bottlenecks in the mobile web application DEVICE SUPPORT BROWSER SUPPORT KEY BENEFITS COST AT&T APPLICATION RESOURCE OPTIMIZER (ARO) ios, Android and Windows Phone Firefox, Safari, Internet Explorer and Chrome AT&T ARO monitors Mobile app s battery life, CPU, response times, and network data apart other browser specific data Open Source HTTPWATCH ios only Firefox, Safari, Internet Explorer and Chrome Easy to use, accurate and comprehensive tool that provides automated recommendation to improve performance Free version is available but has restricted features. FIDDLER ios, Android and Windows Phone Firefox, Safari, Internet Explorer and Chrome Facilitates performance bottleneck detection by capturing response time, caching, compression and network statistics Open Source Computer Measurement Group, India 7

8 Mobile Web Performance Debugging Tools : Evaluation Criteria Criteria Performance Metrics Browser Level Performance Profiling features Other features Description Ability to measure & report: Request Level response time break-up, Page Level response time break up, Network Time break up, status code, warnings, errors and compression metrics Capability to provide automated recommendation to improve performance, java script profiling, caching details and cookie information Ability to easily integrate, support advance filtering, extensible customization and generate easy to read reports. Each criteria was given a priority rating from 1 to 3 ( 1 Low, 2 Medium, 3 High) based on criticality. Each criteria was then evaluated on the scale of 1 to 5 ( 1 Very Poor, 2 Poor, 3 Good, 4 Very Good, 5 Excellent). The total score was calculated by taking sum of the product of priority rating and evaluation rating. Computer Measurement Group, India 8

9 Mobile Web Performance Debugging Tools: Evaluation Results ARO Httpwatch Fiddler 0 Performance Metrics Performance Profiling Other features Total ARO captures most of the performance metrics and provides detailed recommendations on how to fix persistent performance problems but tool usage is complex and time consuming. Captures device specific metrics along with the other browser specific metrics for performance. HTTPwatch (licensed version) is a user friendly tool that can help us detect performance bottlenecks with ease. Does not support android devices. Fiddler is a free, cross-platform tool that helps to identify performance bottlenecks on all the browsers/ devices. Less features as compared to ARO and HTTPwatch. Computer Measurement Group, India 9

10 Android Performance Debugging Tools Android SDK Tools: DDMS and TraceView KEY BENEFITS The DDMS is an invaluable tool for checking the memory usage of Android app, drilling down into errors, simulating a range of real-world conditions, and much more. TraceView is a graphical viewer to see logs created by an Android application. Using, TraceView you can measure the performance of your application to identify performance problems Little eye Little Eye allows monitoring of all apps built for Android 2.3 or later, including system apps! Once our device is plugged in, Little Eye automatically detects all installed apps on your device - all we need to do is to select an app and start monitoring. COST DDMS and TraceView are free and shipped with android sdk Single user license costs upto $500. Computer Measurement Group, India 10

11 Android Performance Debugging Tools: Evaluation Criteria Criteria Performance Metrics Performance Profiling Simulation and Reporting Others Description Ability to measure & report: Response times, Network Statistics, Resource(CPU, Disk, Memory) Usage Statistics, status code, warnings, errors and power consumption Ability to provide automated recommendation to improve performance, method level timing, line level timing and decompiling third party code. Ability to emulate different types and speed of network, save profiling data for future viewing and generate easy to use reports Advance Filtering, capability to interface with IDE s and user intuitive Each criteria was given a priority rating from 1 to 3 ( 1 Low, 2 Medium, 3 High) based on criticality. Each criteria was then evaluated on the scale of 1 to 5 ( 1 Very Poor, 2 Poor, 3 Good, 4 Very Good, 5 Excellent). The total score was calculated by taking sum of the product of priority rating and evaluation rating. Computer Measurement Group, India 11

12 Android Performance Debugging Tools: Evaluation Results Performance Metrics Performance Profiling Simulation and Reporting Other features DDMS and TraceView Little eye DDMS should be used for android application profiling and ARO should be used to capture performance metrics such as CPU, Power and Network data. Little eye does not provide response time data and its breakup at method or line level. Little captures performance metrics such as CPU, Memory, Network and Disk I/O, which are also captured by ARO except Disk I/O. Use the lint tool for static code analysis. The Lint checks your Android project source files for potential bugs and optimization improvements in correctness, security, performance, usability, accessibility and internationalization Computer Measurement Group, India 12

13 Android Memory Profiling Tools The Android SDK provides two main ways of profiling the memory usage of an app: DDMS Allocation Tracker Allows you to track, in real time, where objects are being allocated when you perform certain actions in your application The Allocation Tracker is useful when you want to get a sense of what kinds of Memory allocation are happening over a given time period Heap dump Analysis using Eclipse Memory Analyzer (MAT) A heap dump is a snapshot of an application's heap, which is stored in a binary format called HPROF. Observing changes in the VM heap is a good starting point to detect Memory leaks MAT can open the HPROF file and generate default reports like Dominator Tree, Histogram, leak suspects, etc. which can help you to you find memory leaks and high memory consumption issues. Computer Measurement Group, India 13

14 ios Performance Debugging and Memory Profiling Tools Instruments is a performance, analysis, and testing tool for dynamically tracing and profiling OS X and ios code Key Features: Shows time line of any event occurring in the application, such as CPU activity variation, memory allocation, and network and file activity, together with graphs and statistics. The Instruments application can be used to find potential memory leaks and the allocations in both OS X and iphone applications Captures device diagnostics regarding energy usage, CPU Activity, Network Activity, Display Brightness, Sleep/Wake, Bluetooth, Wi-Fi, and GPS instruments. Reports I/O events, File Operations and application activities In addition, you can use Xcode analyzer for static code analysis and identify problems such as logic flaws, memory management flaws, dead store flaws and API usage flaws. Computer Measurement Group, India 14

15 Windows Performance Debugging and Memory Profiling Tools The Windows Phone Application Analysis tool is a monitoring and profiling tool to evaluate and improve the quality and performance of Windows Phone apps. To analyze the performance of drawing visual items and method calls in the code To analyze object allocation and detect application Memory leak Captures key metrics such as: Launch time Responsiveness Network data transfer Battery consumption Windows Phone Application Analysis comes with windows phone SDK and integrates seamlessly with Visual Studio Computer Measurement Group, India 15

16 Mobile Performance Optimization Techniques and Tools The following tables provides standard performance optimization techniques that are implemented in web based projects and recommended tools in each of the category Key Function Key Benefits Uglify JavaScript Minification and Compression YUI Compressor CSS Minification and Compression Sprite Generator Sprite Generation AdvanceCOMP Lossless Image Compression Reduces network overhead and response time by reducing size of JavaScript files. Reduces network overhead and response time by reducing size of CSS files Reduces network overhead by lowering the number of http requests by combining multiple images into a single image file Improves user experience by reducing Image size Note: Uglify and YUI compressor do not concatenate files in isolation but when integrated with build scripts such as ant, grunt can concatenate JS and CSS files. Computer Measurement Group, India 16

17 Mobile Debugging Tools The following tables provides tools used primarily to resolve functional issues. Key Function Key Benefits Batarang AngularJS Debugging and Profiling WeInRe Remote Web Debugging Inspects Scopes, Models, DOM Elements and methods Displays response times for expressions attached to the scopes Remotely inspect and edit HTML DOM elements Captures performance metrics such as response time Firebug Debugging and Monitoring Eclipse ADK Android Debugging tool Visual Studio Debugger Windows Debugging tool Firebug provides options to edit, debug, and monitor CSS, HTML, and JavaScript in web pages Eclipse ADK allows us to debug currently running threads, set breakpoints and view system log files in real time Windows Phone application debugging tool that can allow us to edit and recompile code on the fly Computer Measurement Group, India 17

18 Mobile Performance Optimization Developer Tools v/s Technology View Performance Development Tools Mobile Web Device Performance Debugging Performance Best Remote Practices Code Debugging Verification Code Debugging Script Minification (JS) Script Minification (CSS) Image Optimization Sprite Compressi on All Devices and Browsers Device ARO* Fiddler* HttpWatch* Key Functions Programming languages YSlow* PageSpeed* WeInRE Firebug Batarang** Native App UGLIFY YUI Compressor AdvanceComp Sprite Generator (CodePlex) Performance Debugging Code Debugging Memory Profiling ANDROID ios WINDOWS Java Objective C & Objective C++ C#, VB.Net & C++ Traceview ARO Eclipse ADK Lint MAT Instruments XCODE Analyzer Instruments Windows Phone Application Analysis Tool Visual Studio Debugger Windows Phone Application Analysis Tool * Run this tool on all the browsers supported by the application ** Only for AngularJS applications Computer Measurement Group, India 18

19 Tools for Mobile web Development Waterfall Model Tools / Techniques Design Development Testing Sprite Generator AdvanceComp YUI Compressor Uglify Firebug / Batarang* WeInRe YSlow ARO Fiddler * Only for AngularJS applications Computer Measurement Group, India 19

20 Tools for Mobile Web Development Agile Model Tools Phases Frequency Sprite Generator Design Every Sprint AdvanceComp Design Every Sprint YUI Compressor Development Every Sprint Uglify Development Every Sprint WinRE Development Every Sprint Firebug/ Batarang* Development Every Sprint YSlow Testing Every alternate Sprint ARO Testing Every alternate Sprint Fiddler Testing After every two Sprints Each Sprint is assumed of 2 weeks. The above table is just a high level recommendation that can be used as a reference. The actual process of when to use these tools in agile model of development should be defined based on project requirements. Computer Measurement Group, India 20

21 Tools for Native App Development Waterfall Model Eclipse ADK Lint ARO MAT TraceView Xcode Analyzer Instruments Visual Studio Debugger Window Phone Application Analyzer Design Development Testing Android Development Tools ios Development Tools Windows Development Tools Computer Measurement Group, India 21

22 Tools for Native App Development Agile Model Tools Phases Frequency Android Native Application Development Eclipse ADK Development Every Sprint Lint Development Every alternate Sprint ARO Development Every alternate Sprint MAT Development Every Sprint TraceView Development Every Sprint ios Native Application Development XCode Analyzer Development Every Sprint Instruments Development Every alternate Sprint Windows Native Application Development Visual Studio Debugger Development Every Sprint Window Phone Application Analyzer Development Every alternate Sprint Each Sprint is assumed of 2 weeks. The above table is just a high level recommendation that can be used as a reference. The actual process of when to use these tools in agile model of development should be defined based on project requirements. Computer Measurement Group, India 22

23 Mobile Performance Testing Tools Computer Measurement Group, India 23

24 Mobile Performance Testing Tools SUPPORTS MOBILE WEB SUPPORTS NATIVE SUPPORTS TESTING FROM REAL MOBILE DEVICES REMARKS HP LOADRUNNER Does not support automated testing of mobile apps. NEOLOAD Neoload does not support Mobile device performance measurement SOASTA Touch Test and Cloud Test PERFECTO MOBILE Perfecto Mobile is suitable for Functional Test Automation but relies on other tools like HP LoadRunner/JMeter for performance testing Computer Measurement Group, India 24

25 Mobile Application Performance Management Tools Computer Measurement Group, India 25

26 Mobile APM Tools COMPUWARE GOMEZ AND DYNATRACE SUPPORTS MOBILE WEB SUPPORTS NATIVE REMARKS Compuware Gomez & DynaTrace together provide the most extensive drill-down capability including application code-level metrics. It has ability to measure and report all performance metrics such as response times, bytes and also reports all crashes & errors. NEW RELIC New Relic analyzes metrics like response times, http errors - However it lacks crashreporting capability and ability to drill down to the application level root cause. SOASTA MPULSE mpulse has a very basic capabilities for mobile performance measurement Computer Measurement Group, India 26

27 Thank you Computer Measurement Group, India 27

28 APPENDIX A: YSLOW Grades Recommendations Each Rule is graded from A to F, where F grade indicates very poor compliance with the rule and A grade indicates that the rule has been effectively implemented within the page Computer Measurement Group, India 28

29 APPENDIX B: Fiddler To Start Fiddler, Launch the shortcut of Fiddler from Start > All Program. Open the web page to be analyzed for performance in a browser of your choice. URL Bytes Sent Bytes Received Protocol type HTTP Status Estimated Performance Computer Measurement Group, India 29

30 APPENDIX C: ARO Computer Measurement Group, India 30

31 APPENDIX D: TraceView Timeline Panel Profile Panel Computer Measurement Group, India 31

32 APPENDIX E: DDMS Allocation Tracker STEP 1: Start DDMS and select the process to be analyzed. Click Update Heap button. STEP 4: To detect memory leaks check whether the 'Allocated' value keeps increasing after several GC calls. STEP2: Click the Cause GC button STEP 3: Click on an object type in the list to see a bar graph that shows the number of objects allocated for a particular memory size in bytes Computer Measurement Group, India 32

33 APPENDIX F: Instruments Computer Measurement Group, India 33

34 APPENDIX G: TouchTest Ref: Mobile-Test-Labs-Solutions-Brief.pdf Computer Measurement Group, India 34