Enterprise DevOps. No more silos. March 2014 Dave van Herpen. Sogeti Nederland B.V White paper

Transcription

1 Enterprise DevOps No more silos March 2014 Sogeti Nederland B.V White paper

2 Contents 1 DevOps: no more silos DevOps defined DevOps relations Continuous Delivery DevOps in practice Enterprise DevOps... 7 Addendum: 10 Golden Rules of Enterprise DevOps...8 1/10

3 1 DevOps: no more silos Organizations worldwide have adopted the Agile way of working. This, however, does not say that these organizations actually bring new or adapted software to production with the required speed and frequency. Predominantly during this final step (often referred to as the last mile ) the delivery process hampers. The root cause? The (IT) organization has too many silos. The solution? DevOps, an organizational movement and way of thinking which enables continuous delivery. This sounds a lot easier than it is in real life. Who doesn t know them: IT departments where designers, developers, testers, support and operations live in splendid isolation from each other, with a minimal level of collaboration. The designers cherish their own requirements methodologies, developers working on their code (possibly in Scrum teams), after which the results are pulled through the test factory, in order to be thrown over the operations wall at the end. Products, as delivered by the development teams (Scrum has named these potentially shippable products ), pile up in front of operations doorstep. And when these product increments are eventually implemented in a large release, it takes unacceptably long before errors/bugs can be related to their source. Integration problems don t come to the surface before the tester is running the acceptance tests. And what about the user, who is constantly faced with latency in the availability of critical changes? 2/10

4 2 DevOps defined The time has come to get rid of these silos, which have triggered the compartmentalization of IT. The answer to this originates from (mostly) Belgium and the USA, where the DevOps movement experienced a boost. On a global scale, this movement has rapidly gained terrain and relevance ever since being given its name in Initiated by joining a number of ideologies and good practices from the worlds of development and operations, DevOps has catalyzed a true IT revolution. To illustrate this, it has enabled several organizations to deploy (software, configurations) to production up to 8000 times faster than traditional organizations. Although the name suggests only Development (Dev) and Operations (Ops) will more closely collaborate, the movement is much broader than just that. That is why bringing together Dev and Ops is referred to as DevOps Lite (after Patrick Debois), whereas true DevOps also entails the integration of crucial roles such as the business, testing/qa and security. This holistic thinking is the first principle of DevOps. Besides that, it is considered fundamental to DevOps to not only have (mostly Agile) development teams deliver potentially shippable products, but to have the target deployment environments available as well (provisioning). Clearly this is where DevOps takes Agile implementations one step further, thereby providing the IT organization more valuable feedback on the quality of the delivered products. Surely automation plays an essential role here. Without a high degree of automation, it is virtually impossible to provision and synchronize (DTAP) environments in a fast and standardized way. Probably the most fundamental shift which is part of the DevOps way of working, is the way errors and risks are dealt with. Traditional organizations tend to have a cultural heritage where errors are being punished, hence covered up. DevOps organizations assume that errors are excellent, as they improve the organization s resilience. It enhances the organizational capability to learn, moving these types of organizations towards a state of antifragility (Nicholas Taleb). They are known for their ability to absorb disturbances, even grow from them, and continuously adapt to changing circumstances. Source: Mike Kavis Antifragile Systems: Designing for Agility vs. Stability 3/10

5 3 DevOps relations The revolutionary aspect of DevOps is not about the individual components it touches. It is the contextual combination and application of these techniques, methods and ideologies. The following essential relations are identified in relation to DevOps: Agile: Many of the principles applied in organizations that have adopted DevOps, concur with the Agile principles. Think of short feedback loops, minimizing unit size and fast flow of planned work. Agile Lean Lean: The Lean way of thinking is not only applicable to the factory floor. Lean elements such as Voice of the Customer, Flow, Pull and Kaizen are used more and more in IT organizations. Waste is reduced and errors are identified and solved at the source ( no defects downstream ). ALM Cloud DevOps ITIL ToC Theory of Constraints (ToC): This methodology, related to Lean, is characterized by the elimination of bottlenecks. By consistently searching for essential limitations in your organization s product and service flows, these constraints (or bottlenecks) can be taken away adequately. A wonderful example of a bottleneck in an IT organization is found in the The Phoenix Project book (Behr, Kim, Spafford), where the IT organization is hampered by the heroism of one particular IT specialist (Brent), the incarnate bottleneck. ITIL: Without a doubt, ITIL also plays a significant role in DevOps organizations. If well applied, the introduction of Agile and Lean principles and instruments in the entire IT delivery chain (so including operations and support) account for faster and more flexible service management processes. Take Configuration Management, which is crucial in DevOps in sharing information between several roles and domains. Cloud: Many organizations have started their transformation to the cloud, either partly or full blown. Cloud technology enables fast provisioning, adjustment (scaling up/down) and synchronization of (DTAP) environments and in automating several build, integration, test or deployment tasks. Application Lifecycle Management (ALM): As said, DevOps is seriously slashing traditional silos in organizations. These silos often represent the stages in the application lifecycle, such as design, build, test and operations. In bringing these functions together in multidisciplinary teams, DevOps is directly related to the emergence of integrated ALM tooling. 4/10

6 4 Continuous Delivery The Continuous Delivery discipline has been of the primary contributors to the initiation of DevOps. The principles behind Continuous Delivery are virtually identical to those behind the DevOps movement. Although the name Continuous Delivery is actually much better in depicting the actual business need, the DevOps philosophy has a much more dominant position in the global market at this moment. Continuous Delivery is mostly aimed at joining all (semi-)finished IT products in the so-called deployment pipeline and automating as much processes as possible within this pipeline. DevOps also explicitly includes the cultural and organizational impact in order to truly deliver on this continuous delivery promise. Thus, Continuous Delivery is in fact a subset of DevOps, with primary components as Continuous Integration (continuously integrating and testing code which was built separately) and Continuous Deployment (continuously and automatically deploying all changes to production). The objective of Continuous Delivery is to build and maintain software in such a way, that it can be promoted to production at any given time. 5/10

7 5 DevOps in practice From its international context the DevOps movement has its origin mainly in start-ups and internet companies, such as Flickr, Amazon, Netflix, Twitter, Etsy, Facebook and Salesforce. Of course, this is strongly related to the innovative culture and dynamic business these companies are facing. In the past few years however, its adoption has migrated rapidly to other sectors. This is also very much the case in The Netherlands. Several banks, Insurance companies, logistic suppliers, industrial multinationals, energy providers, but also (semi-)government organizations have started adopting DevOps principles and practices, including the organizational and cultural consequences. Some organizations have rigourously changed their organizational model towards a DevOps collaboration model, others are still in the experimental stage. The problems these organizations are facing, can be related explicitly to the DevOps CAMS pilars (see below). Particularly the essential cultural (C) shift to a multidisciplinary and interdepartmental collaboration model is a really tough hurdle; to IT professionals, yet even more to their managers. CAMS DevOps is a lot more than just automating the IT delivery processes to a high degree. This notion is illustrated by the so-called CAMS pilars, which are fundamental to all successful applications of the DevOps philosophy: No matter to what extent an organization has applied IT process automation, as long as its professionals and managers do not have the required competences or show the right behavior, the silos will never truly disappear. This calls for a thorough change approach. Once processes have been standardized, they can be automated. Tools are an essential catalyzer in achieving continuous integration, in applying consistent version control and configuration management, automating deployments, or provisioning and monitoring environments. In order to improve, information is indispensable. By means of metrics, information can be offered to professionals and managers for decision making purposes, such as number of deployments per time unit or FTE, or the number of implemented improvements. Joining several roles in teams is fundamental to DevOps, preferably co-located and with adequate knowledge management tools. Sharing knowledge on requirements, development, test findings and operational issues contribute to the continuous feedback flows within these multidisciplinary teams. 6/10

8 6 Enterprise DevOps This almost sounds too good to be true. And in fact it is. Even though the DevOps movement is delivering numerous best and worst practices on a global scale, there is no such thing as reading the book or blog and start doing DevOps tomorrow. If you want to make a success out of your DevOps-related efforts, it is advisable to take into account the 10 Golden Rules of Enterprise DevOps (see addendum). The term Enterprise DevOps is synonymous to the conscious implementation of the DevOps (and Continuous Delivery) philosophy in your organization, with careful consideration of complex landscapes and the overall business needs. Practical documentation: - N.N. Taleb, Antifragile: Things That Gain From Disorder, G. Kim, K. Behr & G. Spafford, The Phoenix Project, J. Humble & D. Farley, Continuous Delivery, R. England, Plus! Standard + Case, 2013 is Management Consultant at Sogeti Netherlands, mainly active in the field of Agile, Lean, Operations and Service Management. 7/10

9 Addendum: 10 Golden Rules of Enterprise DevOps 1 Do not forget about your core systems It goes without saying that mostly start-ups and internet companies have adequately adopted the DevOps principles. After all, they did not have any legacy systems and historical complexity. The average (Dutch) organization, however, simply has these type of complex environments to deal with on a daily basis. At first it might not always seem logical to include these traditional environments while moving to DevOps. Resistance in these environments tends to be quite substantial. Moreover, the necessity to deploy and deliver in a frequent and fast manner may not be present. These situations ask for careful listening to actual business needs, thereby connecting to these new ways of working as much as desired. Even in these (eg. mainframe) environments, automated testing, integration and deployment scan add substantial value, just as much as more intensive collaboration between the disciplines. 2 Embrace the cloud Speed and flexibillity. It should not surprise you that the cloud is a core driver behind successful DevOps organizations. Platform-as-a-Service (PaaS) enables the prompt availability and adaptability of DTAP environments. This way, a developer does not need to wait 3 weeks for a test environment; instead it is built within a matter of minutes, and broken down when it is no longer needed. This not only saves time and money, but also effectively optimizes the quality of the synchronized environments by means of standardization. 3 Reward risk takers In achieving a DevOps organization, creating and nurturing a DevOps culture is an essential vehicle. A DevOps culture can be distinguished by its open en stimulating attitude with regard to risks. By decreasing the size of work units (changes) and intense automation, also in testing, risks are limited to a large extent and errors are prevented. Nonetheless, should errors occur, then foremost they need to be treated as learning points, instead of being hidden in fear of punishment. Only then a truly resilient (Taleb: antifragile ) organization can grow with future- and shockproof environments. In DevOps production environments we even see destructive agents (like Netflix Chaos Monkey) being launched in order to bring down random services, and then learn from this; all in order to optimize the organization s learning capability. 4 Not everything can be automated A strong discipline of continuous delivery within DevOps organizations require a high degree of automation. All activities that can be standardized are eligible for automation. You may think of integration, testing, deployments and provisioning. However, there will always remain a substantial part of the work that cannot be automated. It is helpful using the practical partition Rob England (The IT Skeptic) has introduced, where Standard requests are distinguished from Case requests. Standard requests are repeatable, predictable and eligible for automation. The process, solution and timeframe for Cases, however, cannot be determined up front, as they simply did not occur before. Every organization, DevOps or not, will always need to consider how these Cases are handled most effectively. 5 Remove bottlenecks Organizations that plan to adopt the DevOps philosophy, generally have this desire in order to better deal with all types of changes, internal and external. The related processes change as well. As a matter fact, process improvement is an essential part of daily work, or even more important than that. This results in a continuous quest for bottlenecks in work processes, so as to reduce and eliminate them. Central to this belief is that only improvements on the bottleneck are considered useful; all other improvements are some kind of waste. 8/10

10 6 Manage your portfolio Working in and with DevOps teams takes a great amount of discipline, as well as trust. In line with the Agile way of thinking, the DevOps philosophy is based on creating space and freedom in deciding how the required products are realized and maintained by the teams. Typically, this liberty leads to the overall business case (on program or portfolio level) not being picked up adequately by the different teams. The importance of management information and performance data is not decreasing if DevOps has been adopted. More than ever, it has become relevant for organizations to facilitate decision making across innovation (project portfolio) and the status quo (application portfolio). 7 Include IT support DevOps primarily concentrates on improving the collaboration between the (business and) IT domains and speeding up the delivery cycles. Providing support on the existing information systems and functionalities can implicitly be assigned to the teams, but in practice many DevOps adoption programs fail to include this vital function within the teams focus. Whereas particularly in handling information requests, support demands and other service requests, DevOps (eg. Agile, Lean) practices can be really beneficial. A great example if the use of kanban systems to visualize, handle and monitor (mostly Case) requests, such as problems, changes or improvements. 8 Use IM to involve your business The direct consequence of breaking down the traditional IT silos, is that teams start collaborating more closely in highly automated environments. The risk this might produce is that IT has lost sight of the customer (business, user) in an enthousiastic search for efficiency and collaboration within IT. In The Netherlands we should be proud of the professional position of Functional Management (Business Information Management),in representing the business interest in IT development and operations. Let s not squander this position and give the business information manager the (Product Owner) role it deserves, with regard to functional support, demand articulation and functionality management, also in DevOps cultures. 9 Facilitate multidisciplinarity Merely rebranding your current IT teams to DevOps teams is not exactly a panacea here. In order to really make the Agile and Lean principles work, the teams will need to be composed and managed on a multidisciplinary basis. This means that developers, testers and operations/support engineers should complement and help each other, as long as it supports the team result. This also implies no restrictions should be imposed in fulfilling this multidisciplinarity by means of rigid resource management or dogmatically chasing KPIs on management level. The team members should be given the freedom to expand their knowledge and competences outside their original domain. 10 DevOps engineers do not exist Now that the DevOps movement has gained significant ground from traditional methods, new function titles are growing out of all proportion. Some organizations adopting the DevOps philosophy have started their search for DevOps engineers. But these do not really exist. Surely, a multidisciplinary team demands developers to do testing, while operations engineers will write documentation together with the developer. But still that does not lead to pure generalists, or DevOps engineers. There will always be specialists, but with a multidisciplinary (DevOps) state of mind. 9/10