How To Achieve Continuous Delivery

Transcription

1 White Paper Overcoming Jenkins Sprawl: Going from CI to CD with ElectricFlow

2 Software is everywhere. And accelerating the delivery and quality of that software can mean the difference between merely surviving, and thriving. In the past few years, forward-thinking product teams have adopted Agile and Lean Development practices, along with Continuous Integration ( CI ) solutions to automate and accelerate the build and test phase of their application lifecycle. Many organizations are realizing that the low cost and simplicity of setting up a Jenkins server has lead to a proliferation of one-off Jenkins instances. These islands of innovation are the result of a get it done/ shadow IT mindset among quick-moving development teams. This decentralization of Jenkins CI activities is sometimes referred to as Jenkins sprawl, and presents organizations pursuing a Continuous Delivery model with many challenges, including: Jenkins, a CI solution with an MIT-style license, has been adopted by many product teams because of its free as in beer price tag, community-driven support model, and straightforward deployment options. As much as CI does to accelerate software projects, mature organizations have begun adopting Continuous Delivery ( CD ) practices to increase velocity beyond the build and test phase. CD is a development discipline where product is kept in a constant shippable state, enabling efficient release to market at any time, according to business need. Achieving Continuous Delivery is a long-term, complex and ambitious transformation, involving and impacting all dimensions of an R&D organization. The CD mindset is a way-of-working that requires collaboration, transparency, and continuous improvement in order to identify and eliminate bottlenecks in the delivery process. 1. No centralized build-test-deploy infrastructure Organizations often wish to centralize the infrastructure for building, testing and releasing new products. Large numbers of unique Jenkins servers makes it difficult to govern and manage CI activities from a single, centralized location. 2. Barriers to sharing and reusing scripts In many larger projects, reusable code components are shared between teams. If these libraries are being managed in a redundant fashion on multiple CI servers, the build scripts for those libraries begin to diverge. Such distribution of artifacts decreases the likelihood that code, tests and scripts can be effectively shared by all development teams. 3. Inefficient hardware utilization Larger, mature projects often require significant hardware resources for both building and testing. A decentralized build and test environment across multiple Jenkins instances makes it difficult to achieve the economies of scale necessary to efficiently utilize virtual machines and/or pooled resources (both hardware and software). 2

3 4. No obvious commercial support The open source community, a.k.a. forums and mailing lists can be credited for much of the strength of open source. However, a mailing list isn t very helpful when your mission critical server is down. A quick search online can help you find information about error messages and common mistakes, but all too often it turns out to be a bug that you must somehow work around. 5. Difficulty scaling build-test-deploy environment Each build environment contains its own unique set of configuration requirements, resulting in more complexity. This increases the effort required to scale these environments to meet the demands of growing and distributed teams with more complicated build scenarios. 6. Complex testing and resource administration As projects grow in complexity, the effort required to provide just-in-time assets and artifacts to geographically distributed teams grows. Lack of a centralized management layer to coordinate distributed build environments increases friction between developed products and the resources and human capital necessary to effectively test those products. Embrace and Extend To achieve Continuous Delivery requires continual optimization of every part along the path to value delivery. When faced with the challenges of Jenkins sprawl, organizations often consider consolidating all of their build and test servers into a centralized and managed CI environment. While there is surely value and economies of scale to be realized in this approach for IT professionals and senior management, it is likely to meet resistance by project leaders and software developers. This resistance is the result of the need for agility at the project and team level: when projects are in a rapid development phase, significant changes are being made both to the source code and the build and test infrastructure. Centralizing the access and management of continuous integration can have the unintended negative consequence of stifling agility. ElectricFlow: Enterprise Ready. Enterprise Friendly. 7. Bespoke environments across teams Providing every project team with the ability to create their own bespoke CI server means developer and QA team knowledge cannot be easily transferred from one environment to another. This increases the associated costs of moving development resources between teams. 8. Build tracking and cross-project reporting Specific CI server instances provide updates to a particular team, but aggregating status to a departmental or enterprise level remains a significant challenge. Correlating dependency and status across independent projects is often a complex manual process of merging build status data from multiple, independent CI sources. Ease of Use Self-service model Add new projects without scripting Targeted role-based alerting to log file errors Facilitate sharing of libraries of procedures CLI, API and SDK access to key functionality Resource Management Manage details of hardware resources and configurations Pool machines/resources and balance loads across them Scalability and Security Fault tolerance/reliability/scalability Secure the system with role-based access control or LDAP authentication Produce a build bill of materials 3

4 There is another way to achieve CD while still providing individual project teams the flexibility they need: integrate the existing disparate Jenkins environments into a single, centrally managed and easily governed instance of the ElectricFlow platform from Electric Cloud. This approach minimizes the risks of smothering agile project and development teams, while capitalizing on the significant enterprise capabilities of ElectricFlow. By adopting the ElectricFlow platform, organizations can: 1. Gain a centralized view of Jenkins instances in the enterprise 2. Prioritize and migrate projects to ElectricFlow to increase collaboration and sharing of artifacts across the enterprise 3. Accelerate organization-wide CD with a buildtest-deploy Center of Excellence to establish best practices and leverage pooled enterprise resources By leveraging an existing investment in Jenkins CI servers and applying the centralized governance possible with Electric Cloud, companies can increase visibility while reducing risk. Centralize Reporting for multiple Jenkins Instances The old adage, you cannot manage what you cannot measure was never more true than with the production of software. By establishing ElectricFlow as the system of record for the build-test-deploy status of all projects in the enterprise, project teams and management can achieve a single pane of glass to enable visibility into the status of multiple projects, at a glance. By simply placing an ElectricFlow Agent on each Jenkins server and monitoring all build events, a unified view of the entire enterprise development pipeline can be realized. Centralize Reporting with ElectricFlow Tag-based selection, reporting and management Automatically extract important data from logs Provide cross-product and trend reports Report on resource utilization rates Figure 1. Centralize Enterprise Build Reporting with ElectricFlow 4

5 This allows the ElectricFlow dashboard to become a centralized place where any stakeholder (development, management, quality assurance, and more) can evaluate the status of every project in the delivery pipeline. This is an important first-step towards establishing a center of excellence where all best-practices are fostered and shared. Migrate Jenkins Instances to ElectricFlow A fundamental principle behind the introduction of ElectricFlow to organizations using Jenkins is that, at first, it offers an additive level of visibility and management, not a disruptive one. Certain projects may be poor candidates for rip-and-replace migration from one CI server to another, but there is very little reason to limit the amount of reporting and visibility possible over every Jenkins instance across the organization. As with any iterative process, success is built on early positive feedback. As the results from centralized cross-project reporting become evident, individual projects may decide that they would benefit from the increased scalability and governance offered by a solution such as ElectricFlow. When determining which projects should be migrated first, there are a number of factors to be considered, including: 1. Project Maturity New projects, those with minimal code churn or those that are not integrating with late-cycle QA teams are good candidates for migration. 2. Reliance on Pooled Resources Projects that can benefit from access to pooled hardware and software resources such as virtual labs, QA infrastructure and build farms are good candidates for migration. 3. Need for Visibility Projects that create artifacts or product that other teams rely on for their build and test processes are good candidates for migration. Bringing the candidate projects into ElectricFlow provides benefit to any teams which would like to leverage shared capabilities / libraries / components without disrupting any existing build practices or project velocity. It also helps to improve the utilization of hardware that you already own by pooling it together and making it available to groups in an on-demand, elastic way. Figure 2. Prioritizing and Migrating Candidate Projects to ElectricFlow 5

6 A measured approach to migration helps prove the value of an enterprise-capable build-test-deploy system while making all build processes more efficient, transparent, auditable and repeatable for the entire organization. Accelerating CD with a Build-Test-Deploy Center of Excellence Once projects have been migrated off of Jenkins, organizations can fully leverage the enterprise capabilities within ElectricFlow. These capabilities include on-demand access to distributed and parallel job processing, increased governance and auditability, sophisticated artifact management, dynamic resource provisioning, target platform integration and validation and comprehensive analytics and reporting. ElectricFlow s unique information architecture groups items (steps, jobs, procedures, etc.) segments related items in virtual, access controlled projects and each item is tagged with properties that make it simple to create reports and identify assets for reuse. Distributed workspaces also help geographically distributed teams to share a common build and release platform. ElectricFlow collects pinpoint statistics (such as number of compilations, number of tests run, and number of test failures) and provides visibility into important productivity metrics such as trends in broken builds or test failures. The resulting build-test-deploy Center of Excellence gives developers, project managers and quality assurance professionals self-service access to the best builds for all platforms. Organizations may find it necessary to retain individual instances of Jenkins throughout their CI infrastructure. Because these islands of functionality are being managed and reported on as part of the entire Continuous Delivery process using the ElectricFlow platform, teams can maintain the flexibility they need and still operate safely and efficiently by leveraging enterprise resources on demand. Across all projects, visibility will be greatly improved. Build scripts will be more readily shared and extended for diverse target platforms. Figure 3. Creating a Centralized, Comprehensive Build-Test-Deploy Environment 6

7 Summary While not possible in every case, achieving CD through a centralized build-test-deploy environment gives the entire enterprise visibility and understanding of the complete software development process. Standardized and ubiquitous access to resources and artifacts enables developers to move easily between teams, and project managers to better capitalize on the work of others. IT professionals can allocate, configure and share platforms, resources and improve hardware utilization. Management and other constituents can understand, predict and trace the build-test-deploy cycle and gain visibility into promotion and product release decisions. About Electric Cloud Electric Cloud powers Continuous Delivery. We help organizations developing web, mobile, and embedded systems deliver better software faster by automating and accelerating build, test and release processes at scale. Industry leaders like Cisco, E*TRADE, Gap, GE, Huawei and Qualcomm use Electric Cloud solutions and services to boost DevOps productivity and Agile throughput. For more information, visit electric-cloud.com. Pursuing a phased approach to leveraging Jenkins, rather than blindly replacing it, gives all stakeholders an opportunity to leverage existing value without disrupting the velocity and agility of existing teams. For any organization, improvements in visibility and consistency in the build-test-deploy cycle will accelerate their journey to CD, and result in improved software quality, quicker time to market and better overall morale of the entire software delivery team. 7

8 Corporate Headquarters Electric Cloud, Inc. 35 S. Market St, Ste 100, San Jose, CA T: F: Electric Cloud China New City Center Plaza, No.70, Room 908, Tong Chuan Road, Shanghai, , China T: / china.info@electric-cloud.com Electric Cloud Europe 1650 Arlington Business Park Theale, Reading Berkshire RG7 4SA United Kingdom T: +44 (0) europe.info@electric-cloud.com Electric Cloud Japan KK 22F Shibuya Mark City West Dogenzaka, Shibuya-ku Tokyo Japan T: japan-info@electric-cloud.com Electric Cloud, Inc. All rights reserved. Electric Cloud, ElectricAccelerator, and ElectricFlow are trademarks of Electric Cloud, Inc. All other names are used for identification purposes only and are trademarks of their respective companies.