Continuous Integration (CI)
|
|
|
- Delphia Griffin
- 10 years ago
- Views:
Transcription
1
2 Introduction A long standing problem for software development teams has been to maintain the stability of an application while integrating the changes made by multiple developers. The later that integration is delayed the more potential there is for risk and failure in the integration process. Continuous Integration (CI) tries to mitigate this risk by frequently integrating small changes into an evolving code base. This white paper discusses the CI process and shows how we at CC Pace practice it. Intended Audience This document is aimed at: Developers who will incorporate the CI phase of application development process Lead programmers and data architects who need to complete the groundwork tasks required before the CI phase Information Services (IS) managers who will provision the infrastructure required to support the CI phase Project managers who need to understand what tasks must be completed before the CI phase kicks off and the best practices to be incorporated as a team culture What is it? CI is the process designed to ensure that your software is always working, and that you get comprehensive feedback in a few minutes as to whether any given change to your system has broken the functionality. It exposes integration problems as early as possible. It also aims to have a built, tested, and potentially releasable build at every stage of a project. As defined by Martin fowler: Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. 2
3 So, how do we start? We start by setting up the CI server components described below and illustrated in Figure 1. Developer workstations host the chosen IDE, source/version control client, and have access to a per-workstation sandboxed database schema. Source repository server hosts the Version Control System (VCS).It is the central location for all the source code. It allows multiple developers to collaborate projects simultaneously. VCS must be set up to allow nonexclusive checkouts with possible merging i.e., two features may affect the same managed file at the same time. No exclusive checkout locks should be configured, as this may prevent a developer from finishing a task. Integration Server hosts the continuous integration build agent, which constantly monitors the source control system for check-ins. When checkins is detected, the build agent initiates a complete automated rebuild of the system, and runs the complete suite of automated regression tests, if any. Alerts will be sent to the team if the build fails for any reason. UAT/Demo Server is dedicated to the use of the business team to verify product increments. 3
4 Can I have some examples of CI configurations? CC Pace has incorporated the following CI configurations in the delivery of JAVA and.net solutions. Table 1 Various CI configurations What is the process? Ideally, CI is a completely automated process that interferes as little as possible with a developer s productivity. To start the process, developers are required to do three things when integrating their code with the VCS: 1. Update their local code base with the latest changes from the VCS. 2. Merge any conflicts this is a manual process if different developers have changed the same lines of code. 3. Build and run tests locally. 4. Commit the local code base to the VCS only after your local build is successful. Never commit broken code. Once the code is committed to the VCS, the CI server detects the change and pulls the latest code and dependencies from the VCS. The code is rebuilt and automated tests are run to verify that nothing has broken. Other tasks in the build process are also run, such as database schema updates and test data setup. The CI server provides ongoing status updates as each step in the build process is completed. The developer can check this status and react to any failed builds by making the necessary changes and checking in again. 4
5 The end result of a successful integration is a compiled and tested artifact that can be promoted to another environment, such as UAT. How often does the build run? 5
6 What are the barriers for CI adoption? Key highlights All the code, build scripts, test scripts etc should be versioned in a single version control repository. Successful build should depend on passing all unit tests Separate and isolated Integration Server Frequent incremental commits Speed up builds by - Designing independent and parallel build-agents for separate code modules. This may require separate database instances for each build-agent if they run in parallel. - Using a mock testing framework to limit the interactions of tests with the database. Rapidly communicate status (success, failure or in-progress) of a build 6
Software Construction
Software Construction Martin Kropp University of Applied Sciences Northwestern Switzerland Institute for Mobile and Distributed Systems Learning Target You can explain the importance of continuous integration
Continuous integration End of the big bang integration era
Continuous integration End of the big bang integration era Patrick Laurent Partner Technology & Enterprise Applications Deloitte Mario Deserranno Manager Technology & Enterprise Applications Deloitte The
DRUPAL CONTINUOUS INTEGRATION. Part I - Introduction
DRUPAL CONTINUOUS INTEGRATION Part I - Introduction Continuous Integration is a software development practice where members of a team integrate work frequently, usually each person integrates at least
Leveraging Rational Team Concert's build capabilities for Continuous Integration
Leveraging Rational Team Concert's build capabilities for Continuous Integration Krishna Kishore Senior Engineer, RTC IBM [email protected] August 9-11, Bangalore August 11, Delhi Agenda What
Continuous Integration: Improving Software Quality and Reducing Risk. Preetam Palwe Aftek Limited
Continuous Integration: Improving Software Quality and Reducing Risk Preetam Palwe Aftek Limited One more title Do you love bugs? Or Are you in love with QC members? [Courtesy: Smita N] Agenda Motivation
Nick Ashley TOOLS. The following table lists some additional and possibly more unusual tools used in this paper.
TAKING CONTROL OF YOUR DATABASE DEVELOPMENT Nick Ashley While language-oriented toolsets become more advanced the range of development and deployment tools for databases remains primitive. How often is
Delivering Quality Software with Continuous Integration
Delivering Quality Software with Continuous Integration 01 02 03 04 Unit Check- Test Review In 05 06 07 Build Deploy Test In the following pages we will discuss the approach and systems that together make
Continuous Integration. CSC 440: Software Engineering Slide #1
Continuous Integration CSC 440: Software Engineering Slide #1 Topics 1. Continuous integration 2. Configuration management 3. Types of version control 1. None 2. Lock-Modify-Unlock 3. Copy-Modify-Merge
The Importance of Continuous Integration for Quality Assurance Teams
The Importance of Continuous Integration for Quality Assurance Teams Without proper implementation, a continuous integration system will go from a competitive advantage for a software quality assurance
Continuous Integration
Continuous Integration Collaborative development issues Checkout of a shared version of software ( mainline ) Creation of personal working copies of developers Software development: modification of personal
Continuous integration for databases using Red Gate tools
Whitepaper Continuous integration for databases using Red Gate tools A technical overview Continuous Integration source control develop Dev Dev Dev build test Automated Deployment Deployment package Testing
SOFTWARE DEVELOPMENT BASICS SED
SOFTWARE DEVELOPMENT BASICS SED Centre de recherche Lille Nord Europe 16 DÉCEMBRE 2011 SUMMARY 1. Inria Forge 2. Build Process of Software 3. Software Testing 4. Continuous Integration 16 DECEMBRE 2011-2
Developing Software in a Private workspace - 4.01 PM PMS
SBCH06.fm Page 67 Friday, October 4, 2002 4:01 PM 6 Private Workspace A government clerk s room, showing a desk with books, telephone and directory, and a desk lamp on it. Washington, D.C., 1939. Photo
Continuous Delivery. Alejandro Ruiz
Continuous Delivery Alejandro Ruiz True reality How the customer explained it How the project leader understood it How the analyst designed it How the programmer wrote it What the customer really needed
Kevin Lee Technical Consultant [email protected]. As part of a normal software build and release process
Agile SCM: Realising Continuous Kevin Lee Technical Consultant [email protected] Agenda What is Continuous? Continuous in Context As part of a normal software build and release process Realising Continuous
Continuous Delivery Workshop
Continuous Delivery Workshop deployment pipelines Workshop materials created by Jez Humble, Martin Fowler, Tom Sulston, & Neal Ford deployment pipelines tests, synergistic practices, incremental deployment
Software Configuration Management Best Practices for Continuous Integration
Software Configuration Management Best Practices for Continuous Integration As Agile software development methodologies become more common and mature, proven best practices in all phases of the software
Practicing Continuous Delivery using Hudson. Winston Prakash Oracle Corporation
Practicing Continuous Delivery using Hudson Winston Prakash Oracle Corporation Development Lifecycle Dev Dev QA Ops DevOps QA Ops Typical turn around time is 6 months to 1 year Sprint cycle is typically
Pipeline Orchestration for Test Automation using Extended Buildbot Architecture
Pipeline Orchestration for Test Automation using Extended Buildbot Architecture Sushant G.Gaikwad Department of Computer Science and engineering, Walchand College of Engineering, Sangli, India. M.A.Shah
Implementing Continuous Integration Testing Prepared by:
Implementing Continuous Integration Testing Prepared by: Mr Sandeep M Table of Contents 1. ABSTRACT... 2 2. INTRODUCTION TO CONTINUOUS INTEGRATION (CI)... 3 3. CI FOR AGILE METHODOLOGY... 4 4. WORK FLOW...
Software Configuration Management. Context. Learning Objectives
Software Configuration Management Wolfgang Emmerich Professor of Distributed Computing University College London http://sse.cs.ucl.ac.uk Context Requirements Inception Elaboration Construction Transition
Meister Going Beyond Maven
Meister Going Beyond Maven A technical whitepaper comparing OpenMake Meister and Apache Maven OpenMake Software 312.440.9545 800.359.8049 Winners of the 2009 Jolt Award Introduction There are many similarities
Improving database development. Recommendations for solving development problems using Red Gate tools
Improving database development Recommendations for solving development problems using Red Gate tools Introduction At Red Gate, we believe in creating simple, usable tools that address the problems of software
WHITEPAPER. Improving database development
WHITEPAPER Improving database development Introduction At Redgate, we believe in creating simple, usable tools that address the problems of software developers and technology businesses. In considering
Continuous integration for databases using
Continuous integration for databases using Red Wie Sie Gate die tools Microsoft SQL An overview Continuous integration for databases using Red Gate tools An overview Contents Why continuous integration?
Source Control Systems
Source Control Systems SVN, Git, GitHub SoftUni Team Technical Trainers Software University http://softuni.bg Table of Contents 1. Software Configuration Management (SCM) 2. Version Control Systems: Philosophy
Software configuration management
Software Engineering Theory Software configuration management Lena Buffoni/ Kristian Sandahl Department of Computer and Information Science 2015-09-30 2 Maintenance Requirements System Design (Architecture,
The Value of Adopting Agile & CI for OBIEE
The Value of Adopting Agile & CI for OBIEE KEVIN MCGINLEY medium.com/@kevin_mcginley @kevin_mcginley linkedin.com/in/kevinmcginley www.redpillanalytics.com [email protected] @RedPillA 2014 RED
Boosting Agile Methodology with webmethods BPMS
Boosting Agile Methodology with webmethods BPMS Sami Morcos Chief Solution Architect, Software AG Enterprise Architect Office Rupinder Singh Director, Software AG Global Consulting Services Topics Agile
Software Configuration Management. Visiting Lecture Tero Kojo
Software Configuration Management Visiting Lecture Tero 1 About the lecturer Lectured Software Configuration Management for three years at TKK T-76.614 SCM Also a year as the course assistant Practical
Continuous Integration in Kieker
28. November 2014 @ Stuttgart, Germany Continuous Integration in Kieker (Experience Report) Nils Christian Ehmke, Christian Wulf, and Wilhelm Hasselbring Software Engineering Group, Kiel University, Germany
Agile Software Factory: Bringing the reliability of a manufacturing line to software development
Agile Software Factory: Bringing the reliability of a manufacturing line to software development Today s businesses are complex organizations that must be agile across multiple channels in highly competitive
Continuous integration for databases using Redgate tools
Continuous integration for databases using Redgate tools Wie Sie die Microsoft SQL Server Data Tools mit den Tools von Redgate ergänzen und kombinieren können An overview 1 Continuous integration for
Automated testing and continuous integration
Technical white paper Automated testing and continuous integration Build and test your software at the same time Table of contents Introduction 2 Who should read this document? 2 Build managers 2 Developers
Software Continuous Integration & Delivery
November 2013 Daitan White Paper Software Continuous Integration & Delivery INCREASING YOUR SOFTWARE DEVELOPMENT PROCESS AGILITY Highly Reliable Software Development Services http://www.daitangroup.com
Software Configuration Management
Software Configuration Management 1 Software Configuration Management Four aspects Version control Automated build Change control Release Supported by tools Requires expertise and oversight More important
Continuous Integration Just another buzz word?
Continuous Integration Just another buzz word? Brad Appleton, Steve Konieczka, Steve Berczuk September 2003 Last month we wrote that we would be addressing some questions and concerns raised by readers
Introducing Continuous Integration
Chapter 2 Introducing Continuous Integration Commit Code Frequently Don t Commit Broken Code Fix Broken Builds Immediately Write Automated Developer Tests All Tests and Inspections Must Pass Run Private
Enabling Continuous Delivery by Leveraging the Deployment Pipeline
Enabling Continuous Delivery by Leveraging the Deployment Pipeline Jason Carter Principal (972) 689-6402 [email protected] Pariveda Solutions, Inc. Dallas,TX Table of Contents Matching
Continuous Integration
LT-QAI Torch Talk: Continuous Integration Marc @dfki.de DFKI Language Technology Lab, Saarbrücken and Berlin, Germany What do I mean by Torch Talk? Idea to increase the frequency of
Global Software Change Management for PVCS Version Manager
Global Software Change Management for PVCS Version Manager... www.ikanalm.com Summary PVCS Version Manager is considered as one of the leading versioning tools that offers complete versioning control.
Getting started with API testing
Technical white paper Getting started with API testing Test all layers of your composite applications, not just the GUI Table of contents Executive summary... 3 Introduction... 3 Who should read this document?...
Beginners guide to continuous integration. Gilles QUERRET Riverside Software
Beginners guide to continuous integration Gilles QUERRET Riverside Software About the speaker Working with Progress and Java since 10 years Started Riverside Software 7 years ago Based in Lyon, France
ALM2013VS_ACC: Application Lifecycle Management Using Visual Studio 2013
ALM2013VS_ACC: Application Lifecycle Management Using Visual Studio 2013 Description This three-day, instructor-led course provides students with the knowledge and skills to effectively use the Application
Continuous Integration on System z
Continuous Integration on System z A Proof of Concept at Generali Deutschland Informatik Services GmbH Enterprise Modernization GSE Frankfurt, 14th October 2013 Markus Holzem, GDIS-AS mailto: [email protected]
Modern practices 2.3.2015 02.03.2015 TIE-21100/21106 1
Modern practices 2.3.2015 1 Today s lecture Learn what some modern SW engineering topics are about A peek to some research topic of our department 2 3 4 5 6 How the lectures continue? 02.03 Modern practices
Continuous delivery Release software on-demand, not on Red Alert
Continuous delivery Release software on-demand, not on Red Alert Have it all. Ahead of the competition Value In a world where customers expect a mobile and connected 24x7 experience, businesses must adapt
The Benefits of Utilizing a Repository Manager
Sonatype Nexus TM Professional Whitepaper The Benefits of Utilizing a Repository Manager An Introduction to Sonatype Nexus TM Professional SONATYPE www.sonatype.com [email protected] +1 301-684-8080 12501
An Introduction to Continuous Delivery
An Introduction to Continuous Delivery rolf russell continuous delivery practice lead 2011 All rights reserved. conan the deployer getting it in front of users quickly http://code.flickr.com small feature
<Insert Picture Here>
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment
Introduction. Software Development and Change Management Recommendations
Introduction This document was created by a joint working group of the Futures Industry Association ( FIA ) Principal Traders Group and the FIA European Principal Traders Association (collectively FIA
Effective Team Development Using Microsoft Visual Studio Team System
Effective Team Development Using Microsoft Visual Studio Team System Course 6214A: Three days; Instructor-Led Introduction This three-day instructor-led course provides students with the knowledge and
CONTINUOUS INTEGRATION
CONTINUOUS INTEGRATION REALISING ROI IN SOFTWARE DEVELOPMENT PROJECTS In the following pages we will discuss the policies and systems that together make up the process called Continuous Integration. This
Software Process and Project Plan
Software Process and Project Plan for Trillium Health - Grant Management Version: 1.2 Prepared by: Akshay Karnawat Date: November 26, 2014 (updated) Team Members: Shannon Trudeau ([email protected]) Matt
Application Lifecycle Management Using Visual Studio 2013 (SCRUM)
Course Code: QAALMS13 Vendor: Microsoft Course Overview Duration: 3 RRP: 2,009 Application Lifecycle Management Using Visual Studio 2013 (SCRUM) Overview This three-day, instructor-led course provides
GECKO Software. Introducing FACTORY SCHEMES. Adaptable software factory Patterns
Introducing FACTORY SCHEMES Adaptable software factory Patterns FACTORY SCHEMES 3 Standard Edition Community & Enterprise Key Benefits and Features GECKO Software http://consulting.bygecko.com Email: [email protected]
We (http://www.newagesolution.net) have extensive experience in enterprise and system architectures, system engineering, project management, and
We (http://www.newagesolution.net) have extensive experience in enterprise and system architectures, system engineering, project management, and software design and development. We will be presenting a
Continuous Integration, Delivery and Deployment. Eero Laukkanen T-76.5613 - Software Testing and Quality Assurance P 20.11.2015
Continuous Integration, Delivery and Deployment Eero Laukkanen T-76.5613 - Software Testing and Quality Assurance P 20.11.2015 System Integration In engineering, system integration is defined as the process
Continuous Integration with Jenkins. Coaching of Programming Teams (EDA270) J. Hembrink and P-G. Stenberg [dt08jh8 dt08ps5]@student.lth.
1 Continuous Integration with Jenkins Coaching of Programming Teams (EDA270) J. Hembrink and P-G. Stenberg [dt08jh8 dt08ps5]@student.lth.se Faculty of Engineering, Lund Univeristy (LTH) March 5, 2013 Abstract
SharePoint Services: Using Workflows
SharePoint Services: Using Workflows Table of Contents INTRODUCTION TO WORKFLOWS... 1 WHAT ARE WORKFLOWS?... 1 WORKFLOWS THAT ARE INCLUDED IN OFFICE SHAREPOINT SERVER 2007... 2 ABOUT ADDING A WORKFLOW
Continuous Integration
Continuous Integration WITH FITNESSE AND SELENIUM By Brian Kitchener [email protected] Intro Who am I? Overview Continuous Integration The Tools Selenium Overview Fitnesse Overview Data Dependence My
Developing Solutions for Microsoft Dynamics AX in a Shared AOS Development Environment
Microsoft Dynamics AX 2012 Developing Solutions for Microsoft Dynamics AX in a Shared AOS Development Environment White Paper This document provides guidance for developing solutions when multiple development
Introduction to Programming Tools. Anjana & Shankar September,2010
Introduction to Programming Tools Anjana & Shankar September,2010 Contents Essentials tooling concepts in S/W development Build system Version Control System Testing Tools Continuous Integration Issue
Continuous Integration: Aspects in Automation and Configuration Management
Context Continuous Integration: Aspects in and Configuration Management Christian Rehn TU Kaiserslautern January 9, 2012 1 / 34 Overview Context 1 Context 2 3 4 2 / 34 Questions Context How to do integration
The Deployment Production Line
The Deployment Production Line Jez Humble, Chris Read, Dan North ThoughtWorks Limited [email protected], [email protected], [email protected] Abstract Testing and deployment
Atomate Development Process. Quick Guide
Development Process Quick Guide METHODOLOGY Every project is unique You know your business inside out. You have thought and planned your ideas carefully and are keen to see it live as soon as possible.
Using Oracle9i SCM for Software Configuration Management. An Oracle Technical White Paper July 2002
Using Oracle9i SCM for Software Configuration Management An Oracle Technical White Paper July 2002 INTRODUCTION As organizations strive to build larger, more complex, fully integrated software applications
DATABASE VIRTUALIZATION AND INSTANT CLONING WHITE PAPER
DATABASE VIRTUALIZATION AND INSTANT CLONING TABLE OF CONTENTS Brief...3 Introduction...3 Solutions...4 Technologies....5 Database Virtualization...7 Database Virtualization Examples...9 Summary....9 Appendix...
Test Driven Development Part III: Continuous Integration Venkat Subramaniam [email protected] http://www.agiledeveloper.com/download.
Test Driven Development Part III: Continuous Integration Venkat Subramaniam [email protected] http://www.agiledeveloper.com/download.aspx Abstract In this final part of the three part series on
DevOps for the Mainframe
DevOps for the Mainframe Rosalind Radcliffe IBM Distinguished Engineer, Enterprise Modernization Solution Architect [email protected] 1 Please note IBM s statements regarding its plans, directions, and
Continuous Integration - An Efficient Quality Assurance Tool
Continuous Integration - An Efficient Quality Assurance Tool Dr. Thomas Wehner Managing Partner/Consultant Software Engineering Dr. Wehner, Jungmann & Partner January 08, 2009 About Myself 12+ years IT-experience
Redbooks Paper. Deploying Applications Using IBM Rational ClearCase and IBM Tivoli Provisioning Manager. Introduction
Redbooks Paper Edson Manoel Kartik Kanakasabesan Deploying Applications Using IBM Rational ClearCase and IBM Tivoli Provisioning Manager Introduction This IBM Redpaper presents a simplified customer environment
SOE. managing change in system development projects: configuration management
SOE managing change in system development projects: configuration management 2 3 understanding the problem of change change is one of the most fundamental characteristics in any software development process
Efficient Automated Build and Deployment Framework with Parallel Process
Efficient Automated Build and Deployment Framework with Parallel Process Prachee Kamboj 1, Lincy Mathews 2 Information Science and engineering Department, M. S. Ramaiah Institute of Technology, Bangalore,
Fast Feedback: Jenkins + Functional and Non-Functional Mobile App Testing Without Pulling Your Hair
Fast Feedback: Jenkins + Functional and Non-Functional Mobile App Testing Without Pulling Your Hair Uzi Elion, Tecnology Director Carlo Cadet, Director, Technical Evangelists Tuning the Process for Mobile
Tracking Database Schema Changes. Guidelines on database versioning using Devart tools
Tracking Database Schema Changes Guidelines on database versioning using Devart tools Table of Contents Introduction...3 Typical Development Environments...3 Database versioning problems...4 Improving
Delivery. Continuous. Jez Humble and David Farley. AAddison-Wesley. Upper Saddle River, NJ Boston Indianapolis San Francisco
Continuous Delivery Jez Humble and David Farley AAddison-Wesley Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London Munich Paris Madrid Cape Town Sydney Tokyo Singapore
Quality Cruising. Making Java Work for Erlang. Erik (Happi) Stenman
Quality Cruising Making Java Work for Erlang Erik (Happi) Stenman 2 Introduction Introduction I will talk about automated testing, and how to make Java do that work for you. 2 Introduction I will talk
Continuous Testing with ElectricCommander. Electric Cloud, Inc. 2009
Continuous Testing with ElectricCommander Electric Cloud, Inc. 2009 Continuous Testing with ElectricCommander Drive quality earlier in the development process with continuous testing. Large development
Agile Development with Jazz and Rational Team Concert
Agile Development with Jazz and Rational Team Concert Mayank Parikh [email protected] Acknowledgements: Thanks to Khurram Nizami for some of the slides in this presentation Agile Values: A Foundation
Software Configuration Management and Continuous Integration
1 Chapter 1 Software Configuration Management and Continuous Integration Matthias Molitor, 1856389 Reaching and maintaining a high quality level is essential for each today s software project. To accomplish
Levels of Software Testing. Functional Testing
Levels of Software Testing There are different levels during the process of Testing. In this chapter a brief description is provided about these levels. Levels of testing include the different methodologies
Software Configuration Management Best Practices
White Paper AccuRev Software Configuration Management Best Practices Table of Contents page Executive Summary...2 Introduction...2 Best Practice 1: Use Change Packages to Integrate with Issue Tracking...2
Continuous Integration
Continuous Integration Stefan Sprenger ([email protected]) Semesterprojekt Verteilte Echtzeitrecherche in Genomdaten 15. Dezember 2015 Motivation 2 How was software developed before CI?
SQL Server Training Course Content
SQL Server Training Course Content SQL Server Training Objectives Installing Microsoft SQL Server Upgrading to SQL Server Management Studio Monitoring the Database Server Database and Index Maintenance
2405 - Using Git with Rational Team Concert and Rational ClearCase in enterprise environments
2405 - Using Git with Rational Team Concert and Rational ClearCase in enterprise environments Bartosz Chrabski Executive IT Specialist WW Competitive Sales Team [email protected] Peter Hack ClearCase
SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS
SUCCESFUL TESTING THE CONTINUOUS DELIVERY PROCESS @huibschoots & @mieldonkers INTRODUCTION Huib Schoots Tester @huibschoots Miel Donkers Developer @mieldonkers TYPICAL Experience with Continuous Delivery?
Whitepaper. Continuous Integration Tools Applying Best Practices to the Toolchain
Whitepaper Continuous Integration Tools Applying Best Practices to the Toolchain Table of Contents Introduction... 3 Continuous Integration Tools... 3 Device Test Automation... 3 API / Web Test Automation...
