Enterprise DevOps Integrating Development and Operations to Deliver Continuous Value with Modern Business Applications
Contents Taking your business forward with Modern Business Applications and Modern ALM...3 Accelerating value delivery with Enterprise DevOps...5 Enterprise DevOps Practices...6 Key takeaways and the Microsoft Enterprise DevOps Solution...13 Appendix: Hybrid IT for Modern Business Applications...14 2
Taking your business forward with Modern Business Applications and Modern ALM The app phenomenon is steadily permeating industries and ecosystems the world over. What was largely a consumer and web start-up play in the initial years, has since evolved to becoming a relevant and high order priority for organizations across industries. The advances of the internet and surround technologies coupled with the proliferation of software powered devices, have surfaced new app enabled gateways for organizations to accelerate the expansion and reach of their cause. Business leaders in organizations the world over are gaining a strong awareness of such opportunities, and have increasingly high expectations for their IT organizations to deliver modern business applications at a continuous cadence in realizing them. Modern business software Modern business applications take advantage of the continuous advances in internet and cloud technologies, the proliferation of software-powered devices, and the growing affinity for users to be connected at all times. This enables new reach opportunities to further an organization s cause. Direct-to-consumer apps that expand audience reach and increase brand affinity, Supply chain modernization services that enable connected commerce across brick and mortar, web, mobile, and social channels, and Mobile workforce enablement apps that increase employee productivity delivering differentiated and location independent customer services, are examples of modern business application investment opportunities of relevance to organizations across industries and in all phases of development. 3
Realizing these new opportunities requires an organization to graduate their software applications and IT infrastructure from enterprise to internet scale. Modern business applications are internet scale apps. Internet scale apps are edge facing and are accessible anytime, anywhere, and on any platform or device. IT infrastructures that power internet scale apps need to be capable of also enabling an internet scale variety, velocity, and volume of usage, transactions, and data. Modern business applications present a tremendous opportunity for IT organizations to be a frontline leader and enabler of the business. To realize this opportunity, IT organizations need to re-imagine and evolve their practices across Application Lifecycle Management (ALM), Infrastructure Management, and Operations. In evolving application development, infrastructure management and operations practices, IT organizations need to embrace the technological advances in cloud and internet scale computing to graduate to a modern hybrid IT 1 architecture capable of meeting the operational demands of modern business applications. See Appendix: Hybrid IT for Modern Business Applications for an overview of the components of a modern hybrid IT architecture. Modern hybrid IT infrastructure IT organizations also need to evolve their Application Lifecycle Management (ALM) practices to integrate the business, development, QA, and operations functions in a virtuous cycle for greater agility in delivering continuous value. 1 hybrid IT enables companies to use private clouds for application delivery and also take advantage of public cloud infrastructure to create resource flexibility and address temporary infrastructure demand needs 4
Modern ALM Accelerating value delivery with Enterprise DevOps Modern ALM practices need to emphasize the integration of business, software development, and operations functions to enable a continuous cadence for the delivery of modern software. ALM practices have conventionally emphasized the upstream phases of software development (requirements management, development, and testing), with strong momentum and advances in the adoption of agile practices to increase efficiencies in those phases. To proactively address last mile velocity impediments that surface due to the conventional divide that exists between the development and operations functions in deploying and operating software, modern ALM practices need to also extend beyond development to the delivery and operations of that software. As with functional quality, lapses in non-functional operational readiness will adversely impact value delivery cycle times and are more expensive to fix when detected late in the application lifecycle. Last mile velocity impediments 5
Enterprise DevOps entails full lifecycle ALM practices that span software development and operations to accelerate last mile velocity in delivering value with agility and quality. The practices promote and enable integrating the development and operations functions early in and throughout the application lifecycle to address last mile velocity impediments and shorten value delivery cycle times. From a development perspective, this extends agile practices to operations. In today s world, this integration for deliveriing continuous value is crucial as it allows a natural way to implement a continuous feedback loop between the development and operations functions. Enterprise DevOps Enterprise DevOps Practices Enterprise DevOps practices span the full application lifecycle as it integrates the development and operations functions to continuously deliver operations-ready software with agility. Operations readiness in modern business applications entails proactive measures to deliver operations friendly software, reactive mechanisms to address the realities of managing production incidents in internet scale software, and actionable operations analytics to evolve software investments based on learnings in production. The DevOps workflow 6
The following sections introduce the lifecycle Enterprise DevOps practices that organizations should look to infuse into their ALM practices. As with ALM practices, adopting and realizing the business benefits of Enterprise DevOps is a people, process, and tools change management curve that organizations must surmount. It must be noted that Enterprise DevOps is not an all-or-nothing undertaking. Organizations should look to prioritize and graduate their adoption of practices over time. Operations readiness enablement Operations enablement commences at the very beginning of the development cycle by capturing operations readiness requirements as software acceptance criteria. Software acceptance criteria, spanning functional and non-functional operations requirements, must be captured and prioritized to scope the development and delivery iterations. Prioritizing operations requirements requires team discipline. It is not uncommon to see operations requirements being de-prioritized to accommodate functional needs. The de-prioritization of high priority operational requirements can lead to the accumulation of technical debt and increase value delivery cycle times when the negative side effects of that debt surfaces in production. Capturing and prioritizing operations readiness requirements and acceptance criteria is a team effort. Operations stakeholders need to be proactively engaged in the process of defining and prioritizing software acceptance criteria, and business stakeholders must be made aware of the business impact of not addressing operations criteria. The business, development, and operations stakeholders must collectively prioritize operations readiness tasks to minimize technical debt accumulation and avoid last mile velocity impediments. Capturing and prioritizing operational readiness requirements in TFS 7
ALM requirements management and project planning tools can help integrate the business, development, and operations stakeholders in defining and prioritizing operations readiness requirements and acceptance criteria. Accounting for operations readiness tasks when breaking down user stories in product backlogs, capturing operations acceptance criteria in development and testing tasks, and using practices like Kanban to manage the flow of work across the development and delivery pipeline, are emerging agile ALM practices that organizations look to adopt in enabling operations readiness in the upstream phases of the software lifecycle. Managing the flow of work using a Kanban board in TFS Developing operations ready software With acceptance criteria defined and prioritized, the emphasis of the development and testing efforts that follow must be on fulfilling the criteria. Development practices such as unit testing and continuous integrations can be applied to continuously validate functional and non-functional operations acceptance criteria. Quality of Service testing to validate criteria for application performance and throughput under load should also be executed continuously throughout the development process. 8
Load testing during development One of the significant challenges faced by development and testing teams in validating operations readiness is in the operations dependencies and cycle times to stand up environments that are representative of production. Organizations should look to invest in hybrid IT capable lab management automation tools that enable development and testing teams to shorten cycle times in provisioning and managing test environments needed to support build, deploy, and test workflows. Such tools enable self-service experiences for development and testing teams to define and manage environment templates that can be used to automate the provisioning of representative testing environments across on-premise and/or cloud infrastructure. The environments can be configured to mirror production environments and can be used to run a variety of tests including manual tests, unit tests, build verification tests, automated user interface tests, and load tests. Environments can be snapshotted when defects are detected and used to provision repro environments for debugging without requiring development access to production environments. Lab management automation 9
Continuous integrations and deployments The automation of continuous integration and deployment workflows help shorten cycle times by orchestrating the frequent deployments needed in a continuous delivery cadence, to environments management by operations. Such workflow automation solutions should be fully customizable, capable of working across a hybrid IT infrastructure, and supports multi-phase deployment pipelines that are typical in enterprise environments. Parallel and sequential workflow options enables organizations to delineate the intent of testing environments and maximize parallelism in shortening testing cycle times, while also being able to adhere to sequential flows when needed by compliance requirements, such as a manual approval and triggering of deployments to production. Shortened cycle times Organizations should also validate that a deployment automation solution enables the definition and management of runbooks needed to orchestrate complex deployment and rollback procedures on failure. Deployment runbook 10
Integrated incident management to reduce the mean time to resolution Production incidents are the norm in internet scale and connected software. Production incidents could occur due to a variety of reasons spanning hardware malfunction, network anomalies, and software defects. When production incidents surface, it is critical to reduce the time to resolution and reinstate a working experience for your customers. The Mean Time To Resolution (MTTR) is a more significant IT metric in the world of modern software than the Mean Time Between Failures (MTBF). Organizations investing in modern business applications should prioritize investing in integrated incident management practices that shorten operations and development collaboration cycle times in managing production incidents. One of the common challenges faced by IT organizations in this context is the proliferation of incident and defect management tools across the operations and development teams. The often disparate tools used by the development and operations teams are commonly integrated by a bridge ticketing system and manual workflows to manage the flow of communications and information across the teams. Such practices increase the cycle times to report, diagnose, and resolve incidents. Integrated incident management An integrated incident management solution should eliminate the need for bridge ticketing systems and manual collaboration workflows. An integrated solution should be capable of seamlessly integrating the systems and tools used by the development and operations teams in automating workflows for incident reporting, diagnostics, and resolution. Operations engineers should be able to detect and assign incidents that are potential software defects to the development team directly from within an operations management system. On assignment, they should have instant traceability to the work items generated in the development system. Development and operations should then be able to communicate status updates without requiring manual processes, wherein updates to incident statuses in either system should be seamlessly visible in the other. 11
It must also be possible for operations engineers to collect and provide production diagnostic logs that are immediately actionable by the development team. Developers should be able to use the diagnostic logs to debug potential software defects from within a development environment and without requiring access to production environments. Incident diagnostics Testing in Production is a rapidly emerging integrated incident management practice that organizations investing in internet scale software and services should look to explore. The practice entails re-using tests created during development and testing in production environments to run points-of-presence synthetic tests against mission critical application and service access points. Testing in production 12
Such synthetic tests can be used to proactively detect anomalies in mission critical application and service access points, and take action to instate customer friendly experiences while steps are taken to diagnose and resolve the incident. Key takeaways and the Microsoft Enterprise DevOps Solution Implementing Enterprise DevOps practices is a people, process, and tools change management undertaking. This paper discussed the business case for Enterprise DevOps in enabling continuous value delivery with modern business applications, and introduced the related full lifecycle practices. The right tooling solution can help organizations committed to traversing the people and process change curves, and accelerate their adoption of Enterprise DevOps practices at a custom and sustainable pace. Microsoft Visual Studio ALM and Microsoft System Center deliver an integrated and hybrid IT capable Modern ALM solution with Enterprise DevOps capabilities. Using the Microsoft solution, organizations can implement and realize the value benefits of the Enterprise DevOps practices discussed in this paper. Visit the Microsoft Visual Studio ALM web site (http://aka.ms/alm) to learn more about Microsoft s ALM solution, and the DevOps section of the web site to view videos of related capabilities and explore success stories. The Microsoft DevOps solution 13
Appendix: Hybrid IT for Modern Business Applications IT organizations need to embrace the technological advances in cloud and internet scale computing to graduate to a modern hybrid IT 1 architecture capable of meeting the operational demands of modern business applications. The following sections provide an overview of the components of a modern hybrid IT architecture: Modern hybrid IT infrastructure Hybrid infrastructure A hybrid infrastructure is an amalgamation of the following components, integrated and managed as a unified logical infrastructure to power an organization s application portfolio. Bare metal Conventional on-premise IT infrastructure with or without hardware virtualization. Private cloud Internally hosted cloud infrastructure. A private cloud infrastructure is different than conventional bare metal infrastructure in the following ways: A data center built using commodity machines and hardware, reduces the cycle times needed to beef up physical infrastructure when the demand surfaces and 1 hybrid IT enables companies to use private clouds for application delivery and also take advantage of public cloud infrastructure to create resource flexibility and address temporary infrastructure demand needs 14
eliminate cumbersome infrastructure needs forecasting that can be especially difficult to do for dynamic investments such as those enabled by modern business applications. Uses software virtualization to pool and manage the collective resources of the data center. Enables self service provisioning of infrastructure resources for workloads authorized by IT (e.g., development and testing). Incorporates fine grained metering of virtualized compute resources (memory, hard disk, and CPU usage) to enable a cloud model bill of IT where in business groups only pay for the compute resources used (versus purchasing full machines based on upfront forecasting). Standing up and operating a private cloud from ground up without a solution to enable is no simple undertaking. The skillsets required to do so are very different than those required to operate a conventional IT data center. Infrastructure as a Service (IaaS) Offloading servers from self-managed data centers to IaaS providers to reduce IT management costs is increasingly popular. It is a common first step in adopting the cloud for organizations, as it does not require refactoring existing solution, technology, and application architectures. Platform as a Service (PaaS) Platform abstractions exposed as development frameworks and APIs for cloud scale application development, with applications deployed and run on PaaS provider infrastructure fully abstracted by the surround development frameworks. An ideal choice for select green field application development investments and a more involved undertaking for migrating existing applications due to requirements to refactor architectures and code. Big Data/DaaS (Data as a Service) Using cloud services for augmented and high scale data storage, processing, and/or accessibility. Big data processing workloads and requirements to expose data for web scale accessibility are common scenarios for adopting DaaS options. Unified Management A hybrid IT infrastructure with cloud options folded in expands the service management purview of IT operations, even if optimizing the physical infrastructure management workloads. Service management in a cloud context entails application monitoring, performance monitoring, and incident management. A unified management service is needed to enable a single pane of glass view for IT operations to manage applications in a hybrid IT infrastructure. 15
Business Applications and Web APIs Edge facing modern business applications and internal facing line-of-business applications are run on the hybrid IT infrastructure and managed using a unified management service. The edge facing modern business application services need to be exposed as Web APIs that enable their accessibility across the spectrum of current and growing web channels and connected devices. REST APIs to expose application services and OData APIs to expose application data are the widely popular standards for API-fication in the start-up world that are rapidly gaining traction in enterprises in the context of enabling modern business applications. The practice of exposing application services as Web APIs isn t limited to just modern apps, it can also be applied to conventional line-of-business apps to enable multi-channel access while limiting access to internal users. Identity Federation In expanding reach to a broad spectrum of external users, modern business applications need to be capable of federating identity with external identity providers gaining adoption and prominence in the ecosystems of relevance to an organization. For instance, it is common these days for edge facing applications to federate with Facebook, Windows Live, Google, and Yahoo as identity providers in authenticating and authoring user access. Such federation minimizes the risk of user deflection and user erosion resulting from the overhead of managing yet another user id and/or the user forgetting his/her application identity. Connected Device Experiences Modern business applications by virtue of being edge facing need to be accessible across the spectrum of devices that are widely used phones, tablets, desktop computers, laptops, televisions, etc. Delivering connected experiences across relevant devices can further help increase service affinity for example, a consumer can today start watching a movie streamed to his/her television by Netflix, can stop watching, jump on a bus, and resume watching the movie from the paused segment on a tablet or phone on the bus. Such connected device experiences can also extend to business scenarios for example, an employee could commence submitting an expense report on his/her PC at work, jump on a bus ride to get home, and complete the expense report submission using a tablet while on the bus. In enabling reach across the broad spectrum of devices, organizations need to make informed decisions on investing in native versus mobile web versus hybrid app (mobile web apps packaged as a native app) development to balance reach goals with development and maintenance efficiencies. 16
External Services Modern business applications will have requirements to integrate with external services. Such integrations could be at the layers of an application s user interface, the application s services, or the data collected and processed for the application. Organizations need get comfortable with recognizing and integrating with external services that enable their reach goals. External service integrations can help extend the reach of user engagement, commerce channels, and B2B integrations to mention a few scenarios. Application development and lifecycle management Organizations need to evolve their application development and lifecycle management practices to deliver new and continuous value with modern business applications. Modern business applications take advantage of new technological advances and evolve at a continuous cadence in enabling new opportunities to further organizational causes. Organizations need to evolve their development competencies to embrace modern application technologies and platforms. Organizations also need to embrace Modern ALM practices for integrating business, development, QA, and operations functions to deliver continuous value with agility and quality. 17