Artificial Intelligence and Testing. Kishore Durg AccentureTechnology June 2016

Size: px

Start display at page:

Download "Artificial Intelligence and Testing. Kishore Durg AccentureTechnology June 2016"

Marjorie Poole
7 years ago
Views:

3 What is intelligent automation? A constellation of automation technologies that, when used together, spell success. Building blocks for intelligent automation success More proven Increased intelligence Foundation Robotic Integrated analytics Artificial Intelligence Project level, ad-hoc automation for areas like scripts Standardization of scripts and reduced manual efforts Use of analytical tools to predict and recommend Self-learning systems and self-evolving tools Transactional Judgement-driven Copyright 2016 Accenture. All rights reserved. 3

Intelligence Project level, ad-hoc automation for areas like scripts Standardization of scripts and reduced manual efforts Use of

4 State of the market: Automation is awesome! GROWING EXECUTIVE AWARENESS 85% of execs use automation to improve their business process. 1 POWERFUL RESULTS 43% of execs achieved cost savings of greater than 15% from intelligent process automation in the past 2 years. 2 INCREASED INVESTMENT AND INNOVATION AI startups in the US alone have increased 20-fold in the past four years. 3 1, 2 Accenture Technology Vision 2016 Survey. 3 Artificial Intelligence Startups See 302% Funding Jump in 2014, CB Insights, February 10, Copyright 2016 Accenture. All rights reserved. 4

2 INCREASED INVESTMENT AND INNOVATION AI startups in the US alone have increased 20-fold in the past four years.

5 Intelligent Automation: Exponential business potential Doing things differently Doing different things

6 But business leaders have questions about intelligent automation: When is the right time to invest? Where do I start? Can I trust it to run critical business processes? Can I trust it to give me reliable insight? Copyright 2016 Accenture. All rights reserved. 6

Can I trust it to run critical business processes?

8 A Few examples of A.I. across all of Accenture businesses ATA for PharmacoVigilance ATA for Credit Services ATA for Sentiment Analysis Unstructured Cyber Threat Detection This drug doesn t harm the body at its best Copyright 2016 Accenture Technology Lab (Bangalore). All rights reserved.

Credit Services ATA for Sentiment Analysis Unstructured Cyber Threat

10 A.I. in Application Management Automation Themes Cognitive Mechanism Benefit Big Data based operational log analysis Log Analysis, and Ticket Prevention Prevention and closure of common / repetitive errors at source. Automated resolution of Incidents Use NLP and Machine Learning to discover similar, co-related incidents Faster Resolution through cognitive Ticket Solving Agent Knowledge Engineering support for Problem resolution NELL Create knowledge graph on application domain and data. Guided problem resolution / support through Knowledge Search Agent Copyright 2016 Accenture Technology Lab (Bangalore). All rights reserved.

Automated resolution of Incidents Use NLP and Machine Learning to discover similar, co-related incidents Faster Resolution through cognitive Ticket Solving

11 A.I. in Testing Services Accenture Technology Labs (ATL) A.I. and Digital Testing Open Innovation Niche Cognitive Vendor US Research Institute

12 Degree of Complexity Degree of Complexity Degree of Complexity Robotic Humanoid Expert Assistant (Say hello to R.H.E.A.!!!) Cognitive Capability Stack Software Engineering Application Areas Sense Computer Vision ASR NLU (Text) Testin g Savant Comprehend Question Answering Deep Learning Case Based Reasoning Knowledge Graph Dialog Strategy Enablers (ACE+ OS) Application Management Act and Recommend BOTS Process Recommendation NLG Social Impact Learn Active learning / reinforcement learning ACE Copyright 2016 Accenture Technology Lab (Bangalore). All rights reserved.

!!!) Cognitive Capability Stack Software Engineering Application Areas Sense Computer Vision ASR NLU (Text) Testin g Savant Comprehend

14 Accelerating Test Automation Platform (ATAP) Challenges : Author the automation test scripts using tools. The amount of manual effort it takes to maintain an automation script. ATAP allows a tester to author the automation in simple English like sentences. These sentences are then automatically converted to JAVA code using Selenium. TestScripts in ATAP WebObjects in ATAP Copyright 2016 Accenture Technology Lab (Bangalore). All rights reserved.

ATAP allows a tester to author the automation in simple English like sentences.

the test scripts for an application Benefits: The tool showed over 70% accuracy and reduced the manual

15 A.I. in Testing Services : Addressing vendor transitions and Knowledge transfer - Accenture Test Coverage Analyzer Test Scripts from Client Generate a Application Functionality Diagram (AFD) from all the test scripts for an application Benefits: The tool showed over 70% accuracy and reduced the manual effort needed to 20% of original. Copyright 2016 Accenture Technology Lab (Bangalore). All rights reserved.

16 A.I. in Testing Services: Pretotype to Test Case Generator Asset Benefits to Accenture Automatically generates test cases and executable test scripts from visual requirements. Improve productivity. Benefits to Clients Superior quality software Faster time to market Customer satisfaction

test scripts from visual requirements. Improve productivity.

A.I. in Testing Services: Accelerated Accenture Test Repository Reuse (Accelerated ATR Reuse) New Requirement or Change Request as Search Query Accenture

significantly reduce the amount of manual effort required to identify and reuse testing artifacts from ATR for generic testing engagements.

17 A.I. in Testing Services: Accelerated Accenture Test Repository Reuse (Accelerated ATR Reuse) New Requirement or Change Request as Search Query Accenture Test Repository (ATR) Search ATR to identify all testing artefacts meeting search criterion and rank them in order of relevance Benefits: The tool would significantly reduce the amount of manual effort required to identify and reuse testing artifacts from ATR for generic testing engagements. Accelerated ATR Reuse PoC has been piloted with Banking CoE. The tool showed around 80% accuracy and could reduce up to 70% of the manual effort during reuse process.

18 Regression testing A.I. in Testing Services: Start Up Ecosystem Input Solution Output Benefits Test Planning Phase End user Test cases & test scripts Numerous Defects identified Cognitive Computing Solution Grouping of the defects Regression test planning De-duplication of defects Assign appropriate tester / developer to the defect Risk based Testing -Execution limited to co-related test cases Cost Saving Risk based testing for regression System builds co-relation between production, test cases & defects Faster time to market System Testing Phase High priority test cases Application modules with more defects

Cognitive Computing Solution Grouping of the defects Regression test planning De-duplication of defects Assign appropriate tester / developer to the

19 A.I. in Testing Services : Test Data Repository Generation and Reuse

20 A.I. in Testing Services : Digital Testing and Intelligent Execution analytics Customer Employee Partner Test across all online and offline channels X-Industry Enablers Services Applications Online channels WEB MOBILE SMART TV SOCIAL Offline channels POS Retail platform Web front end Database Consolidated & shared master data Cognitive Analytics Integration Full control on retail infrastructure through internal hosting Test on all key back end services Analytics, integration and data management Mobile App Test Lab Web Browser Test Lab Connected World Testing Digital Broadcast Testing Platforms Digital Crowd Test Platform KIOSK TELE MARKET STORE DELIVERY BROCHURES Omni channel testing OCTAVE Data Management Dedicated test infrastructure easily adaptable to dynamic business case changes Security Test Platform Omni channel Test & Validation Engine Products Compatibility Testing Interoperability Testing Automation Libraries Functional Testing Framework APIs Security Testing Performance Testing Conformance Testing Accenture Digital standard test practices [reusable]. End to end automation capability Digital Optimization Device Test Analytics Copyright 2016 Accenture All rights reserved

infrastructure through internal hosting Test on all key back end services Analytics, integration and data management Mobile App Test Lab Web Browser Test Lab Connected World Testing Digital Broadcast

21 Data Inputs The Touchless test Automation platform Legend Analytics Guided Test Cockpit Accenture s IP Data Probes Data Probes Data Probes Production logs and execution traces (masked) Production tickets Development logs and execution traces defects SEARCH Log Mining based System Usage Modeling & Failure Modeling Engine CAST Program Profiler Usage Model & Failure Model inputs ConformiQ Model Generator Test Scripts DISCOVER & LOCALIZE DEFECT LOCALIZATION 21 ENGINE Multi-layer Test Automation API & Services testing Cognitive Learning Engine - test execution Performance Testing Defect deduplication Defect Management System Code and code change metrics Authorship metrics Module structure graphs Program Analysis Data Accenture Test Repository Machine Learning based Defect Discovery & Localization Engine Defect Prediction Heat Maps Multi-layer Automation Defect patterns and bug profiler, Dig-deeper-orsearch-elsewhere decision rules Java Platform Testing TDM SVT Production System

Augmented Search for Software Testing

Augmented Search for Software Testing For Testers, Developers, and QA Managers New frontier in big log data analysis and application intelligence Business white paper May 2015 During software testing cycles,