Sales. Guide. for Channel Partners

Transcription

1 Sales Guide for Channel Partners

2 Table of Contents 2015 SmartBear Confidential TABLE OF CONTENTS SECTION I: How To Use This Guide...3 SECTION II: Who is SmartBear?...4 The Leader in Software Quality Tools for the Connected World...4 SmartBear Products...5 SECTION III: SmartBear Value Guides...7 SmartBear Development Tools...7 Collaborator Discovery Guide...7 Collaborator Differentiator...11 AQTime Discovery Guide...14 SmartBear Test Tools...16 Test Discovery Guide...16 Test Differentiator...21 API Readiness Tools...25 Ready! API Discovery Guide...25 Ready! API Differentiator...29 ServiceV Pro Discovery Guide...33 ServiceV Pro Differentiator...36 SmartBear s Monitoring Tools...39 AlertSite UXM Discovery Guide...39 AlertSite UXM Differentiator...45 SECTION IV: SmartBear s Channel Management Team

3 2015 SmartBear Confidential How to Use This Guide SECTION I: HOW TO USE THIS GUIDE Dear Partners, This sales guide has been prepared for use by Partners and is a CookBook for both Sales personnel and Consultants (if you are a service Partner) on how to position SmartBear s products in your customer base. Section I: Who is SmartBear? Resources: There are two diagrams to illustrate the SmartBear Tools portfolio. The first diagram displays where each tool fits in the software lifecycle. The next diagram is a quick reference guide and the Products and Solutions overview is a helpful summary of core capability of each of the Smart- Bear tools and who the target users and also buyers of the tool will be. Section II: SmartBear Value Guides Each SmartBear tool set also has a Value Discovery Guide which comes in two parts: Discovery Guide is a positioning document, which makes no reference to any of our tools, but helps draw out the pain points the clients may be experiencing and allow the Salesperson or Consultant understand which tool(s) best fit the customers need and also assess what additional services / solutions they may require. Differentiator Guide is a comparative document showcasing the feature set of each SmartBear tool and how it can solve the client s pain points. It also includes sample references from existing client deployments to support the argument for choosing SmartBear tools, as well as a summary of SmartBear tool strengths & weaknesses against other competitor offerings. Section III: SmartBear s Channel Management Team details on your Channel Management Team. Sincerely, The SmartBear Partner Team 3

4 Who is SmartBear? 2015 SmartBear Confidential SECTION II: WHO IS SMARTBEAR? The Leader in Software Quality Tools for the Connected World Nearly all applications today are connected to other applications through APIs, to mobile devices, and the cloud. The Connected World drives tremendous value for application providers, but introduces significant challenges to effectively build, test, and monitor the applications and the connections between them. SmartBear helps companies build and deliver the world s greatest applications through collaborative development, functional testing, performance testing and performance monitoring across API, mobile, web, and desktop. 4

5 2015 SmartBear Confidential SmartBear Products SMARTBEAR PRODUCTS SmartBear Products What the Product does Business Value Value Add Partner Products Additional Business Value Offered by Partner Product Add-On SmartBear and/or Partner Products that combine to create Solutions Used By Development Manages Document and Code Reviews. Integrates with 16 industry leading Software Configuration Management Tools. Integrates with Eclipse & Visual Studio IDEs. Supports multiple document formats. Code Performance Profiling (C, C++,.Net, Java, Silverlight, Delphi, VBScript, JavaScript). Memory profiling (allocation & leaks). Code coverage analysis (% tested vs used). Improved governance. Better quality software shipped. Demonstrate compliance to Regulatory and Industry Standards. Improved user experience (performance /response times). Better quality software shipped. SCM & IDE Integrations with: AccuRev, CMVC, CVS, Git, Mercurial, MKS Integrity, Perforce, Rational ClearCase Rational Synergy, Rational Team Concert, Subversion, StarTeam, Surround SCM, TFS, Vault, VSS. Integrated Development Environment. Whole Project Team. Developers Test Automation Eng Testing Manage test cases in central location. Manage test hosts and schedules. Create and manage defects. Trace Tests to User Stories. Report on test execution. Improved governance / traceability. Better quality software shipped. Demonstrate compliance to Regulatory and Industry Standards. OpsHubs Integrators including: AccuRev, Jira, Bugzilla, CVS, Git, HP ALM, HP Quality Center, Rational ClearCase, ClearQuest, Team Concert, TFS, Rally, TestTrack, Serena Business Manager, Subversion, Trac. Integrated Development Environment. With TestComplete & OpsHub integrators. Test Manager Testers Test Automation Eng Test automation tool. Supports Desktop, Mobile & Web testing. Test Recording and Playback. Supports scripting using VB- Script, JScript. Integrates with Agile and CI tools. Mobile support Native apps Hybrid apps Xamarin apps Faster time to market. Better quality software shipped. Test Automation Frameworks(TAF) (supplied by various Partners) Faster time to market Increased productivity With QAComplete (Solution: Provides integrated test management and automated testing). Test Manager Testers Test Automation Eng 5

6 SmartBear Products 2015 SmartBear Confidential SMARTBEAR PRODUCTS Web load testing tool. Supports Cloud, RIA and Mobile load testing (Ajax, Flex, ASP.Net & Silverlight). Automates Web load testing. Comprehensive Load Test reports. Improved user experience (performance / response times). Better quality software shipped. Performance Engineer Testers API Readiness API functional testing tool. Supports SOAP, REST, HTTP, JMS, AMF, JDBC. Supports Virtualization & API Discovery. Plugin architecture allows for extensions. API Readiness Platform - SoapUI NG Pro, LoadUI NG Pro, ServiceV Pro. Better quality software shipped. Apiary - API Blueprint IBM - API Manager MuleSoft - RAML Reverb Technologies - Swagger. Full API documentation. Allows users to browse Mule- Soft s APIHub directory for APIs using RAML or Swagger definitions. With LoadUI Pro (Solution: Provides API functional & load testing). Testers Test Automation Eng API load testing tool. Supports distributed API load testing. Automates API load testing. Comprehensive Load Test reports. Improved user experience (performance / response times). Better quality software shipped. With SoapUI Pro (Solution: Provides API functional & load testing). Performance Engineer Testers User Experience / Application Performance Monitoring Website speed & availability monitoring. Mobile Monitoring. API monitoring. SLA monitoring. Transaction tracing. Improved user experience (performance / response times / outage reduction). Better quality software shipped. Faster time to find / isolate code defects & performance hotspots. Perfecto Mobile. With SoapUI Pro & LoadUI Pro (Solution: Provides End to End Web, application performance monitoring and API functional & load testing) Perfecto Mobile (Solution: Users will be able to monitor their applications across real devices in the Perfecto Mobile cloud, using preexisting scripts) IT Manager CIO NOC Manager 6

7 2015 SmartBear Confidential Collaborator Discovery Guide SECTION III: SMARTBEAR VALUE GUIDES SmartBear Development Tools Collaborator Discovery Guide Value Driver Improve Code Design and Quality Improve Agility through Collaboration Across Functional Teams and Locations Generate Metrics to Improve Process and Verify Compliance Accelerate Team Member Onboarding Discovery Questions (Current State) Describe how successful your team has been at finding defects during development (rather than QA or the customer). Describe how your distributed team members collaborate. Describe what you are currently doing to ensure that each developer has some understanding of the overall code base. If someone were to leave, what does that mean for your team? Describe how you prepare for a compliance audit. Tell me about the development tools you use. Tell me how product management, development, and QA collaborate on user stories, code, and test plans and ensure they stay in sync. Describe how your distributed team members collaborate. Describe what you are currently doing to ensure that each developer has some understanding of the overall code base. If someone were to leave what does that mean for your team? Describe for me how you prepare for a compliance audit. What information is typically necessary when preparing for an audit? How do you locate the materials pertinent to an audit? Describe what you are currently doing to ensure that each developer has some understanding of the overall code base. Tell me about the process you use to get new members of the development team up to speed on their responsibilities. If a developer on your team leaves, describe how you ensure that the rest of the team can take over his or her work. Before Scenarios Defects found by QA/customer rather than development. Lack of collaboration between members of the development organization particularly those away from HQ. Lack of understanding of the overall code base. Tedious manual effort to document comp-liance. Multiple standalone tools for development. Not agile due to lack of communication between product management, deve-lopment and QA. Lack of collaboration between members of the development organization particularly those away from HQ. Lack of understanding of the overall code base. Tedious manual effort to document compliance. Materials spread out across multiple systems, don t talk to each other. Lack of understanding about what s needed and increased time spent finding the appropriate materials. Developers know their own code, but not what others are working on. Team members must learn on their own rather than from each other. When a developer leaves, the other developers have little idea what that developer was wor-king on and how to pick up his or her work. 7

8 Collaborator Discovery Guide 2015 SmartBear Confidential Value Driver Improve Code Design and Quality Improve Agility through Collaboration Across Functional Teams and Locations Generate Metrics to Improve Process and Verify Compliance Accelerate Team Member Onboarding Negative Consequences High re-work costs and damage to reputation. Inability to quickly respond to changing customer and market dynamics. Inefficient development process (higher cost). Intellectual capital rests with individuals rather than team. High cost of preparation efforts and potential fines. Inability to quickly respond to changing customer and market dynamics. Inefficient development process (higher cost). Intellectual capital rests with individuals rather than team. Slower, error prone development. High cost of preparation efforts and potential fines. Slower, error prone development. Increased costs due to time spent finding the appropriate materials. Inefficient development process (higher cost). Intellectual capital rests with individuals rather than team. Slower, error prone development. Slower, error prone development. Discovery Questions (Future State) Describe the role you would like developers to play in driving quality. When market dynamics change, tell me about how you would like your extended development team to be made aware and respond. Describe how your teams could work together more effectively. How would capturing the knowledge of each developer help you? Describe how you might be able to access information more quickly. Explain how your development environment could be improved. Describe the role you would like developers to play in driving quality. When market dynamics change, tell me about how you would like your extended development team to be made aware and respond. Describe how your teams could work together more effectively. How would capturing the knowledge of each developer help you? Explain how your development environment could be improved. Describe how you might be able to access information more quickly. Explain how your development environment could be improved. Describe how you might prove compliance more easily. When market dynamics change, tell me about how you would like your extended development team to be made aware and respond. Describe how your teams could work together more effectively. How would capturing the knowledge of each developer help you? Explain how your development environment could be improved. More defects found by development, before QA and the customer. More defects found by development, before QA and the customer. Easily document and report on compliance. Extended development team collaborates and reacts to change. After Scenarios Extended development team collaborates and reacts to change. Greater collaboration among developers and sharing of best practices. Greater knowledge of the overall code base among developers. Easily document and report on compliance metrics. Development tools used all talk to each other. Extended development team collaborates and reacts to change. Greater collaboration among developers and sharing of best practices. Greater knowledge of the overall code base among developers. Development tools used all talk to each other. Development tools used all talk to each other. Greater collaboration among deve-lopers and sharing of best practices. Greater knowledge of the overall code base among developers. Development tools used all talk to each other. 8

9 2015 SmartBear Confidential Collaborator Discovery Guide Value Driver Improve Code Design and Quality Improve Agility through Collaboration Across Functional Teams and Locations Generate Metrics to Improve Process and Verify Compliance Accelerate Team Member Onboarding Positive Business Outcomes Cost savings through reduction of rework. Faster, more nimble development. Faster onboarding, greater ability to address changes to code base. Minimize risk due to resource departure. Cost savings through reduction of rework. Faster, more nimble development. Faster onboarding, greater ability to address changes to code base. Minimize risk due to resource departure. Cost savings through reduction of rework. Faster, more nimble development. Quickly respond to audit requests. Cost savings through reduction of rework. Faster, more nimble development. Faster onboarding, greater ability to address changes to code base. Quickly respond to audit requests. How would you enable developers to find more defects? How would you enable developers to find more defects? How would you enable developers to find more defects? How would you enable developers to find more defects? Discovery Questions (What s Required to get there?) Describe how you would make the process of collaboration easier. How do you ensure that collaboration happens among developers? How would you facilitate that knowledge transfer? What would the system look like? Describe the integrations you would like your peer review tool to have. Describe how you would make the process of collaboration easier. How do you ensure that collaboration happens among developers? Describe how you would make the process of collaboration easier. How do you ensure that collaboration happens among developers? How would you facilitate that knowledge transfer? What would the system look like? Describe how you would make the process of collaboration easier. How do you ensure that collaboration happens among developers? How would you facilitate that knowledge transfer? Required Capabilities Efficient means to review code and track fin-dings. Central location to manage documents and facilitate peer code review. Forum for developers to share code and insight on code. Central location for reporting on code comp-liance. Ability to integrate with SCMs and IDEs currently in use. Efficient means to review code and track findings. Central location to manage documents and facilitate peer code review. Efficient means to review code and track findings. Central location to manage documents and facilitate peer code review. Forum for developers to share code and insight on code. Central location for reporting on code compliance. Efficient means to review code and track findings. Central location to manage documents and facilitate peer code review. Forum for developers to share code and insight on code. Metrics # defects found by development and defect density. Adoption rate and delivery improvements. Speed of delivery. # of participants in each review, overlapping reviewers. # of reporting elements around accountability. Speed of delivery. Speed to productivity (for new hires). Decrease in rework. # defects found by development and defect density. Speed of delivery. # of participants in each review, overlapping reviewers. # of reporting elements around accountability. Speed of delivery. Speed to productivity (for new hires). # of participants in each review, overlapping reviewers. 9

10 Collaborator Discovery Guide 2015 SmartBear Confidential Value Driver Improve Code Design and Quality Improve Agility through Collaboration Across Functional Teams and Locations Generate Metrics to Improve Process and Verify Compliance Accelerate Team Member Onboarding Peer Review Peer Review Peer Review Peer Review How We Do It (Satisfy the Required Capabilities) Integration with SCM/IDE. Easily upload materials. Invite the appropriate people to review and capture their conversations. Defect tracking. Metrics and Reporting Easily upload materials. Invite the appropriate people to review and capture their conversations. Defect tracking. Invite the appropriate people to review and capture their conversations. Defect tracking Metrics and Reporting Capture the appropriate metrics and render them in reports. Invite the appropriate people to review and capture their conversations. Defect tracking. Capture the appropriate metrics and render them in reports. Peer Review Peer Review Peer Review Peer Review How We Do It Better Integration with 11 SCMs and the major IDEs. Create users and review groups, leveraging existing structures like LDAP. Synchronous and asynchronous conversations tied to places in materials. Structured workflow to ensure resolution of identified defects. Create users and review groups, leveraging existing structures like LDAP. Synchronous and asynchronous conversations tied to places in materials. Structured workflow to ensure resolution of identified defects. Adaptive workflow, templates and fields. Synchronous and asynchronous conversations tied to places in materials. Structured workflow to ensure resolution of identified defects. Adaptive workflow, templates and fields. Metrics and Reporting Synchronous and asynchronous conversations tied to places in materials. Structured workflow to ensure resolution of identified defects. Adaptive workflow, templates and fields. Adaptive workflow, templates and fields. Metrics and Reporting Automatically gather appropriate. review metrics for reporting. Automatically gather appropriate review metrics for reporting. Electronic signature. Customizable reports. Customizable reports. Proof Points Cisco Problem: Inefficient manual review process Solution: Tool assisted code review Results: More defects found, lower cost of development (less rework), faster onboarding. Intuit Problem: Needed to make it easy to promote the best practice of reviews and inspections, while capturing valuable process data Solution: Implemented Collaborator to track metrics and drive best practices Results: Cost savings through reduction of rework and faster, more nimble development. USAA Problem: Could not be Agile because they are worldwide and could not collaborate effectively Solution: Used Collaborator across development, product management, and testing teams inserting user story, code, and test plan into a single review Results: Increased collaboration across extended development team, now consider themselves much more Agile. Intuit Problem: Needed to make it easy to promote the best practice of reviews and inspections, while capturing valuable process data Solution: Implemented Collaborator to track metrics and drive best practices Results: Cost savings through reduction of rework and faster, more nimble development. Intuit Problem: Needed to make it easy to promote the best practice of reviews and inspections, while capturing valuable process data Solution: Implemented Collaborator to track metrics and drive best practices Results: Cost savings through reduction of rework and faster, more nimble development. 10

11 2015 SmartBear Confidential Collaborator Discovery Guide Value Driver Improve Code Design and Quality Improve Agility through Collaboration Across Functional Teams and Locations Generate Metrics to Improve Process and Verify Compliance Accelerate Team Member Onboarding Proof Points GE Healthcare Problem: Proving compliance Solution: Implemented Collaborator for reporting and electronic signatures for accountability Results: Responded more quickly to audit requests. HP Problem: Measure return of investment on peer review Solution: Initiate and track peer review costs and time savings Results: Return on investment is 10-to-1 and saved $21.4 million per year. Intuit Problem: Needed to make it easy to promote the best practice of reviews and inspections, while capturing valuable process data Solution: Implemented Collaborator to track metrics and drive best practices Results: Cost savings through reduction of rework and faster, more nimble development. GE Healthcare Problem: Challenge with proving compliance Solution: Implemented Collaborator for reporting and electronic signatures for accoun-tability Results: Responded more quickly to audit requests. HP Problem: Needed to measure return of investment on peer review Solution: Initiate and track peer review costs and time savings Results: Return on investment is 10-to-1 and saved $21.4 million per year. HP Problem: Needed to measure return of investment on peer review Solution: Initiate and track peer review costs and time savings Results: Return on investment is 10-to-1 and saved $21.4 million per year. Collaborator Differentiator Type (U,C,H) (U,C,H) Differentiator Customer Value Defensibility Qualifications Unique Peer Review Code and Documents User stories. Test plans. Code. Revenue Ability to speed development and react to changes in specs. Decrease rework time. USAA Solution: Use Collaborator for document review Results: A faster, more nimble Agile team that can react to changing customer information (user stories). IBM Solution: Peer review Results: Each hour of peer review during development saved 20 hours of testing and 82 hours of rework. How is the extended development team Agile? Describe what is preventing them from being more Agile. How is the development team made aware of changes to user stories after initiating a project? Tell me about how you review test plans and user stories today. Collaboration across extended development team reduces lack of knowledge transfer between teams. Comparative Customized Workflow Custom fields (participants, review). Customizable review templates. Groups/review pools. Quicker, more efficient reviews. Eliminate the need to change current process. Intel Solution: Custom fields and templates Results: By allowing/forcing users to enter a value for time spent on code prior to review, Intel saved time in calculating this metric and reduced risk by handling all time metrics with one tool. Explain the procedures that are unique to your company that could prevent you from collaborating through peer review. What specific metrics do you want to capture during reviews? Describe how you manage different review types across projects and groups. 11

12 Collaborator Differentiator 2015 SmartBear Confidential Type (U,C,H) (U,C,H) Differentiator Customer Value Defensibility Qualifications Comparative Metrics Collection LOC reviewed. Defects found. Time spent in review. Revenue Increase speed of development. Optimize the team s ability to find bugs in development. Ability to determine the correct processes and allocation of resources. ebay Solution: Utilizing metrics and reporting Results: ebay is reviewing metrics and making changes based on them, which has increased speed and quality, and reduced risk. What metrics do you collect today around code review? What data do you use to make process changes? Tell me about the amount of time you spend in code review today and how you can confirm it s optimal. Comparative Scalability # of clients accessing the server simultaneously. Ability to handle multiple locations. Revenue Quickly ramp up development community. Scale to the enterprise on one server. Proven enterprise solution. Alcatel Lucent Solution: Single global solution for peer review Results: Running 800 concurrent users across the globe on a single instance of Collaborator. Describe how you can ensure the solution you choose can scale to your current and future enterprise requirements. Tell me about how you are currently supporting globally dispersed teams. How do you ensure that your review data is easily accessible anywhere in the world? Comparative Multiple Integrations # of SCMs. # of IDEs. No need for additional review systems. KLS Tencor Solution: A peer review system that could integrate with their multiple SCMs and IDEs Results: Integration with multiple SCMs and IDE (Git and RTC) allowed them to get up to speed quickly. How are you able to support multiple SCMs and IDEs? How would the peer review system you choose support multiple SCMs and IDEs? Remove manual steps and human intervention. Holistic Best in Class Support Phone support. Utilize development as third tier support. Revenue Customers get up to speed quickly and can maintain their environment. Support mitigates higher price paid for product. GE Healthcare Solution: Looking for new best practices and support assistance Results: Acted as trusted advisor, got them up to speed quickly, helped manage their integrations. Tell me about how you are going to manage the installation and maintenance of your peer review system. How readily available is the support from other vendors that you re considering? Customers get up to speed quickly and can maintain their environment. 12

13 2015 SmartBear Confidential Collaborator Differentiator Competitor SmartBear Strengths and Weaknesses How They Compete Against Us Our Strategic Response Crucible Gerrit Customizable workflows. Metrics/reporting. UI. Lower price up to 5 users. Integration with Jira. Good enough. - Graphical reports. + Customizable workflows. Free tool. + Reporting/metrics. Google lineage. - Price. Good enough. - Lack of Eclipse/Git Integration. You get what you pay for (i.e.: customization, integrations) and we offer strong support. We offer a limited integration with Jira in addition to many other far more important features. You get what you pay for (i.e.: customization, integrations) and we offer strong support. Long-term cost of unsupported free tools. Customer Objection Our Response Proof Points Price Our price is cheap relative to the value of document review, metrics and reporting, integrations. Schlumberger Problem: Know they want code review but were having difficulty justifying the price Solution: Selling Collaborator internally by positioning it as a necessary collaboration tool across their three main geographies. Code review takes too much time The time is worth the value gained from a better code base. Kronos Problem: Did not believe code review would be worth it Solution: Collaborator Result: Post-implementation, they found code review to be a worthwhile investment with a strong ROI. Code review is not Agile (too rigid) Code review helps enterprises become more Agile. USAA Problem: Wanted to be Agile but could not, given their size and distributed nature of the teams Solution: Collaborating through peer review Result: Enabled the teams to be more nimble and more aware of changes during the development process. Not a priority Go higher in organization in order to learn more about the customer s value drivers. Schlumberger Problem: Know they want code review but it s not a priority for the overall company Solution: Schlumberger UK has a major advocate for peer review that is selling the product first in his location, but wants to eventually sell it throughout corporate (UK, Houston, France). Other solutions are good enough Have you considered the total cost of ownership for open source? PlantCML Problem: They were looking for a way to eliminate the manual steps of a code review and gather metrics Solution: Collaborator Result: Eliminated many of the manual steps in their code review process and allowed them to generate custom reports to gather specific types of metrics. 13

14 AQTime Discovery Guide 2015 SmartBear Confidential AQTime Discovery Guide Value Driver Discovery Questions (Current State) Improve Application Performance What impact do performance bottlenecks within your applications have on end user satisfaction? How do you assess your application for possible memory leaks? How do you know what the resource allocation looks like for your applications? How do you ensure adequate code coverage when testing for memory leaks? Before Scenario Negative Consequence No tools used to find performance bottlenecks in applications. No ability to do memory profiling to identify sources of leaks. New applications are released that perform inefficiently. Applications are unstable and overly prone to crash. Developers can t quickly assess applications to understand how efficiently they use resources. New applications released without knowing whether code coverage is sufficient to look at all parts of the application. Discovery Questions (Future State) How would you like to go about detecting memory leaks and getting more visibility into resource consumption of your applications? What do you need to be more confident that new application releases will be as stable as possible and perform to end user expectations? How valuable would it be to know whether tests cover the entire application and whether additional tests are required? After Scenario Applications are designed to be as efficient as possible in consumption of memory and other computing resources. Confidence that applications will be stable and less likely to crash. Realistic tests that accurately simulate users and activities, getting as close to real life as possible. Positive Business Outcome Better end user experience and higher satisfaction. Applications work according to original business goals. Discovery Questions (What s Required to get there?) What is necessary for you to quickly find performance bottlenecks in your applications and scripts? How much more confidence would you have in your applications if you could detect different types of memory and resource leaks? How would you like to be able to perform code coverage analysis to understand whether tests are looking at every part of the application? 14

15 2015 SmartBear Confidential AQTime Discovery Guide Value Driver Improve Application Performance Required Capabilities Accomplish performance profiling of the entire or specific parts of the application. Find and monitor memory leaks. Ability to do continual code coverage analysis to assess which parts of the application are fully tested, partially tested, or not tested at all. Metrics Memory consumption rates and performance times. Total memory usage, memory consumption by category, and resource types. Percent coverage rates and lines of code covered or uncovered. Performance hot spots. Function level and line level hit counts and memory allocations. How We Do It (Satisfy the Required Capabilities) How We Do It Better Find performance bottlenecks in 32-bit and 64-bit C/C++, Delphi,.NET, Java, and Silverlight applications and scripts. Start with a quick performance inspection using the lightweight Sampling profiler and then drill deeper into the hot spots using the more accurate Performance profiler. See application s memory and resource allocations in real time to detect excessive memory and resource usage. Monitor total memory usage, memory usage by category (heap, reserved virtual memory etc). AQtime provides code coverage analysis for C/C++,.NET, Silverlight, Java and Delphi applications. AQtime can be used to measure code coverage during manual application runs, as well as integrated into your automated regression tests for continuous coverage analysis. Code coverage analysis shows which source files, functions and lines of code have been covered by tests, are untested or only partially tested. This helps you know what additional tests to create and how to improve your existing tests. Comprehensive yet easy-to-understand reports that allow developers to easily locate performance bottlenecks, memory and resource leaks and other issues in applications, and optimize the source code appropriately. Summary and detailed reports with ability to visualize critical paths. Use as a standalone performance profiler, or integrated into Microsoft Visual Studio and Embarcadero RAD Studio IDEs for improved productivity. Focus on specific parts of source code and select the exact source files, classes or methods to performance profile or exclude from profiling; start and stop profiling any time during the application run or start and stop profiling automatically when the application execution enters or exits a specific method. Compare profiling results of different application runs to easily view performance difference before and after optimizations; view the difference as exact values or as a percentage. Merge code coverage results from multiple test runs to get aggregate coverage statistics for your entire project. This is useful for large projects where testing is performed in multiple environments. AQtime can also merge results automatically after each coverage session to save you time and allow you to focus on the end results. AQtime includes a Failure Emulator to simulate faults during your application run and monitor their impact on the application behavior to evaluate the effectiveness of application s built-in fault detection and recovery code, determine the performance loss due to failures, and identify dependability bottlenecks that may need to be addressed using fallback solutions. Work smarter and optimize your code in Microsoft Visual Studio or Embarcadero RAD Studio. Find performance issues, memory and resource leaks right from your IDE. 15

16 Test Discovery Guide 2015 SmartBear Confidential SmartBear Test Tools Test Discovery Guide Value Driver Improve Quality Across Mobile, Web and Desktop Increase Speed to Delivery Ensure Predictable Performance Improve Flexibility and Ease of Use Across All Skill Sets Discovery Questions (Current State) What is the impact of doing mostly manual testing? Walk me through how your tools work together. Describe how well your current tools adapt if development is releasing new product initiatives (i.e., web to mobile migration). Explain how your testing is affected when new versions of supported technologies are introduced. What is the impact of doing mostly manual testing? Walk me through how your tools work together. Describe how well your current tools adapt if deve-lopment is releasing new product initiatives (i.e., web to mobile migration). Explain how your testing is affected when new versions of supported technologies get introduced. Explain how you on-board a team member and how long it takes. Walk me through how your tools work together. What is the impact of doing mostly manual testing? Explain how you on-board a team member and how long it takes. How do you ensure team members with limited scripting abilities can complete testing within time and budget? Describe how your solution adapts to the needs of team members who know scripting and those who don t? Before Scenarios No automation/performance tools and more tests than time to run Patchwork of tools that don t meet current business needs Current testing tools don t support new product initiatives (web/mobile/new code) Current tools do not support evolving 3 rd party releases (i.e., new version of IE, Chrome, 3 rd party controls) No automation/performance tools and more tests than time to run Patchwork of tools that don t meet current business needs Current testing tools don t support new product initiatives (web/ mobile/new code) Current tools do not support evolving 3 rd party releases (i.e., new version of IE, Chrome, 3 rd party controls) Tools difficult to use, steep learning curve for team members No automation/performance tools and more tests than time to run. Patchwork of tools that don t meet current business needs. Difficult to use, steep learning curve for team members Doesn t create robust test cases for people with limited skill sets Solution provides limited extensibility; it does not meet specific testing needs of experienced testers Negative Consequences Defects or performance issues released in production software, increasing support and development costs. Poor metrics and reporting. Lost pace in the market due to time consuming testing process, lagging development and missed release dates. Lost QA time spent maintaining tests based on new internal builds or third party releases. Defects or performance issues released in production software, increasing support and development costs. Poor metrics and reporting. Lost pace in the market due to time consuming testing process, lagging development and missed release dates. Lost QA time spent maintaining tests based on new internal builds or 3 rd party releases. Costly and difficult to hire, longer ramp time, lower productivity and greater cost of turnover. Defects or performance issues released in production software, increasing support and development costs. Poor metrics and reporting. Costly and difficult to hire, longer ramp time, lower productivity and greater cost of turnover. Poor release quality. Employee frustration due to inability to use tools effectively. 16

17 2015 SmartBear Confidential Test Discovery Guide Value Driver Improve Quality Across Mobile, Web and Desktop Increase Speed to Delivery Ensure Predictable Performance Improve Flexibility and Ease of Use Across All Skill Sets Discovery Questions (Future State) Describe what you need to get through tests in time. How do you envision your testing tools working together effectively? Explain what you need to effectively support new initiatives. Tell me what you would need to effectively support 3 rd party releases. Describe what you need to get through tests in time. How do you envision your testing tools working together effectively? Explain what you need to effectively support new initiatives. Tell me what you would need to effectively support 3 rd party releases. Describe what you need to get through tests in time. How do you envision your testing tools working together effectively? Describe what you need to get through tests in time. Explain what you need to ensure team members with different skills sets are able to test effectively. Tell me what you need to ensure the solution meets the requirements of both new and experienced testers. After Scenarios Tools enable test completion in the allocated time. Easily create, manage and run tests. Support for all technologies/platforms (desktop, mobile and web). Confidence that vendor stays current with market changes. Tools enable test completion in the allocated time. Easily create, manage and run tests. Support for all technologies/platforms (desktop, mobile and web). Confidence that vendor stays current with market changes. Tools enable test completion in the allocated time. Easily create, manage and run tests. Products support different skill sets and responsibilities Employees with different skill sets are able to create robust test cases Solution scales to different needs easily and quickly Positive Business Outcomes Reduced support and development costs. Efficient and predictable testing process speeds up development. Test maintenance is not time consuming or cumbersome. Reduced support and development costs. Efficient and predictable testing process speeds up development. Test maintenance is not time consuming or cumbersome. Reduced support and development costs. Efficient and predictable testing process speeds up development. Test maintenance is not time consuming or cumbersome. Efficient and predictable testing process speeds up development. Test maintenance is not time consuming or cumbersome. Quick ramp up time, higher productivity, decreased cost of turnover. Discovery Questions (What s Required to get there?) Describe what you need from a tool to complete tests on time. Explain the level of integration you would like among your tools. What platforms/technologies do you need to support? How do you judge if the vendor will stay current with market changes? Describe what you need from a tool to complete tests on time. Explain the level of integration you would like among your tools. What platforms/technologies do you need to support? How do you judge if the vendor will stay current with market changes? Describe what you need from a tool to complete tests on time. Explain the level of integration you would like among your tools. Tell me how you think products can support different skill-sets. Describe how you ensure if emp-loyees with limited coding skills can create robust tests. Describe how you know if a tool can be extended to provide specific functionality. 17

18 Test Discovery Guide 2015 SmartBear Confidential Value Driver Improve Quality Across Mobile, Web and Desktop Increase Speed to Delivery Ensure Predictable Performance Improve Flexibility and Ease of Use Across All Skill Sets Required Capabilities Automate tests across required environments and configurations. Well-integrated tools. Flexible tools that can adapt across new technologies/platforms. Timely support for evolving 3 rd party releases. Automate tests across required environments and configurations. Well-integrated tools. Flexible tools that can adapt across new technologies/platforms. Timely support for evolving 3 rd party releases. A central place to manage testing efforts. Automate tests across required environments and configurations. Well-integrated tools. A central place to manage testing efforts. A central place to manage testing efforts. Easily create custom extensions. Create robust tests without writing scripts. A central place to manage testing efforts. Metrics Hours spent testing per release. Delta between planned and actual testing release date (# of missed deadlines). Traceability matrix, test coverage. Technology/Platform support (checklist). Track record of vendor supporting new technology; Reusability of test cases. Time to support case resolution. # of defects found post deployment (by customer or internally). Delta between planned and actual testing release date (# of missed deadlines). Traceability matrix, test coverage. Technology/Platform support (checklist). Hours spent testing per release. Track record of vendor supporting new technology; Reusability of test cases. Time to support case resolution. Technology/Platform support (checklist). Hours spent testing per release. Track record of vendor supporting new technology; Reusability of test cases. Time to support case resolution. # of defects found post deployment (by customer or internally). Cost associated with rehire, ramp time for a tester. Delta between planned and actual testing release date (# of missed deadlines). Hours spent fixing defects, directly impacting revenue. # of missed testing deadlines due to long UAT cycles, which impacts revenue. How We Do It (Satisfy the Required Capabilities) Automate Tests (A Central Place to Manage Testing Efforts) Easy to create tests that can run on different platforms, environments, devices. Record and playback allows people to create automated tests without deep technical knowledge. Scripting, keyword driven test and extensive set of checkpoints for advanced users. Distributed testing. Well Integrated Tools QAComplete and TestComplete integration provides a single interface for managing, running and analyzing all kinds of automated and manual tests. Full Traceability across manual and automated Tests allows QA Managers to run traceability reports to ensure proper test coverage exists for requirements. Automate Tests (A Central Place to Manage Testing Efforts) Easy to create tests that can run on different platforms, environments, devices. Record and playback allows people to create automated tests without deep technical knowledge. Scripting, keyword driven test and extensive set of checkpoints for advanced users Distributed testing. Automate Tests (A Central Place to Manage Testing Efforts) Easy to create tests that can run on different platforms, environments, devices. Record and playback allows people to create automated tests without deep technical knowledge. Scripting, keyword driven test and extensive set of checkpoints for advanced users. Distributed testing. 18

19 2015 SmartBear Confidential Test Discovery Guide Value Driver Improve Quality Across Mobile, Web and Desktop Increase Speed to Delivery Ensure Predictable Performance Improve Flexibility and Ease of Use Across All Skill Sets Flexible Tools that Can Adapt Across New Technologies/Platforms Well Integrated Tools How We Do It (Satisfy the Required Capabilities) Test native, web and hybrid android applications and take advantage of enhanced functionality like Visualizer and Geolocation on IOS 8. Products have common UI rather than partnership. Timely support for Evolving 3rd Party Releases Rapid release schedules. QAComplete and TestComplete integration provides a single interface for managing, running and analyzing all kinds of automated and manual tests. Full traceability across manual and automated tests allows QA Managers to run traceability reports to ensure proper test coverage exists for requirements. How We Do It Better Automate Tests (A Central Place to Manage Testing Efforts) TestComplete includes images of the tested application captured during the test run and originally recorded screenshot; Particularly helpful in spotting the difference visually in case a test case fails. Cross browser testing. No need to re-record the test case in case of change. More scripting language support. Faster test execution. Distributed testing. Well Integrated Tools Common UI for testing web/desktop/mobile applications; integrated tools for scheduling, running, managing and reporting on all manual and automated test cases at a lower cost. Automate Tests (A Central Place to Manage Testing Efforts) TestComplete includes images of the tested application captured during the test run and originally recorded screenshot; Particularly helpful in spotting the difference visually in case a test case fails. Cross browser testing. No need to re-record the test case in case of change. More scripting language support. Faster test execution. Distributed testing. Automate Tests (A Central Place to Manage Testing Efforts) TestComplete includes images of the tested application captured during the test run and originally recorded screenshot; Particularly helpful in spotting the difference visually in case a test case fails. Cross browser testing. No need to re-record the test case in case of change. More scripting language support. Faster test execution. Distributed testing. Selenium integration. Flexible Tools that Can Adapt Across New Technologies/Platforms Superior object recognition. Access to device data (i.e., orientation, geolocation, Wi-Fi and locale). Consistent gestures. Timely Support for Evolving 3rd Party Releases Well Integrated Tools Common UI for testing web/desktop/mobile applications; integrated tools for scheduling, running, managing and reporting on all manual and automated test cases at a lower cost. Selenium integration. Management s commitment to release early. 19

20 Test Discovery Guide 2015 SmartBear Confidential Value Driver Improve Quality Across Mobile, Web and Desktop Increase Speed to Delivery Ensure Predictable Performance Improve Flexibility and Ease of Use Across All Skill Sets Industry Awards Industry Awards Industry Awards Customer References/Case Studies Proof Points 2014 Dr. Dobb s Award Received award for excellence in automated testing Customer References/Case Studies Numara Software Problem: Limited time, limited resources and issues with test coverage Solution: TestComplete Results: The automated scripts created using TestComplete can run in several hours instead of manually in a few days G3 Technologies Problem: Improve software quality and enhance productivity Solution: TestComplete, QAComplete Results: Seamless integration with TestComplete provided insight and control over manual and automated testing efforts. At the average American company, worker productivity improves maybe 2 to 3 percent per year. At G3T, we have to do even better than that so we can continue to thrive and grow profitably Dr. Dobb s Award Received award for excellence in automated testing Customer References/Case Studies Numara Software Problem: Limited time, limited resources and issues with test coverage Solution: Test- Complete Results: The automated scripts created using TestComplete can run in several hours instead of manually in a few days 2014 Dr. Dobb s Award Received award for excellence in automated testing Analyst Reports/Studies Gartner Magic Quadrant: SmartBear has a full life cycle view of software quality, including application performance management, as well as 3 rd party integration and a strong developer community program. SmartBear provides solid support for migrating assets from HP solutions into its products. It offers an attractive pricing model, with price points that address different markets. Customer References/Case Studies Numara Software Problem: Limited time, limited resources and issues with test coverage Solution: TestComplete Results: The automated scripts created using TestComplete can run in several hours instead of manually in a few days QualityMine Corporation Problem: Switching between different tools to automate tests Solution: TestComplete Results: A lot of tools limit users with the record and playback paradigm. For more serious test automation tasks, TestComplete provides the flexibility to implement automation frameworks that are highly reusable and maintainable across builds and different applications that can quickly boost automated test coverage and maintain existing automated test suites with ease. 20