Solving the database deployment problem

Similar documents
WHITEPAPER. Solving database deployments with Database Lifecycle Management

Continuous integration for databases using Redgate tools

Continuous integration for databases using

Continuous integration for databases using Red Gate tools

WHITEPAPER. Improving database development

Improving database development. Recommendations for solving development problems using Red Gate tools

Key Benefits of Microsoft Visual Studio Team System

Software Development In the Cloud Cloud management and ALM

ALM/Quality Center. Software

SQL DBA Bundle. Data Sheet. Data Sheet. Introduction. What does it cost. What s included in the SQL DBA Bundle. Feedback for the SQL DBA Bundle

Enabling Continuous Delivery by Leveraging the Deployment Pipeline

What is Application Lifecycle Management? At lower costs Get a 30% return on investment guaranteed and save 15% on development costs

Tech Notes. Corporate Headquarters EMEA Headquarters Asia-Pacific Headquarters 100 California Street, 12th Floor San Francisco, California 94111

Nick Ashley TOOLS. The following table lists some additional and possibly more unusual tools used in this paper.

IT Operations Management: A Service Delivery Primer

Quality Assurance - Karthik

5 barriers to database source control and how you can get around them

Why Standardize on Oracle Database 11g Next Generation Database Management. Thomas Kyte

IT Governance In The Cloud: Building A Solution Using Salesforce.com

Orchestrated. Release Management. Gain insight and control, eliminate ineffective handoffs, and automate application deployments

Delivery. Continuous. Jez Humble and David Farley. AAddison-Wesley. Upper Saddle River, NJ Boston Indianapolis San Francisco

Best Practices in Release and Deployment Management

An introduction to the benefits of Application Lifecycle Management

Zero-Touch Drupal Deployment

Life Cycle Management for Oracle Data Integrator 11 & 12. At lower cost Get a 30% return on investment guaranteed and save 15% on development costs

Serena Dimensions CM. Develop your enterprise applications collaboratively securely and efficiently SOLUTION BRIEF

Visual Studio Team Edition for Database Professionals. Woody Pewitt Developer Evangelist

Continuous Delivery. Anatomy of the Deployment Pipeline (Free Chapter) by Jez Humble and David Farley

Practicing Continuous Delivery using Hudson. Winston Prakash Oracle Corporation

AB Suite in the Application Lifecycle

Continuous Integration

The Benefits of Deployment Automation

Integrity 10. Curriculum Guide

How to set up SQL Source Control. The short guide for evaluators

Development Testing for Agile Environments

SOFTWARE TESTING TRAINING COURSES CONTENTS

CASE STUDY FINANCE. Republic Bank - Searching SQL Server databases, a typical quest begins...

Course Outline: Course 6317: Upgrading Your SQL Server 2000 Database Administration (DBA) Skills to SQL Server 2008 DBA Skills

Agile Delivery Framework Automation & Deployment With Puppet

Urbancode Deploy Overview

PIVOTAL CONNECTOR FOR MARKETO. Copyright 2015 Tokara Solutions. All Rights Reserved.

Frequently Asked Questions Plus What s New for CA Application Performance Management 9.7

CASE STUDY LUMIDATA. SQL Toolbelt. Essential tools for SQL Server. 91% of Fortune 100 companies use Red Gate

Introduction to Source Control ---

Upgrading Your SQL Server 2000 Database Administration (DBA) Skills to SQL Server 2008 DBA Skills Course 6317A: Three days; Instructor-Led

Best Overall Use of Technology. Jaspersoft

Getting Started with WebSphere Application Server v8.5 Version to Version Migrations

Application Release Automation (ARA) Vs. Continuous Delivery

How to Migrate From Existing BusinessObjects or Cognos Environments to MicroStrategy. Ani Jain January 29, 2014

Centralized Secure Vault with Serena Dimensions CM

Software Construction

White Paper. Software Development Best Practices: Enterprise Code Portal

The Role of Feedback in Continuous Integration, Continuous Delivery and Agile ALM

WHITE PAPER. Five Steps to Better Application Monitoring and Troubleshooting

Developing Solutions for Microsoft Dynamics AX in a Shared AOS Development Environment

Vladimir Bakhov AT-Consulting +7 (905)

How To Improve Performance In A Database

Continuous Delivery: Automating the Deployment Pipeline. Solution Brief

Building Value with Continuous Integration

Microsoft s Team Foundation Server (TFS) Canute Magalhaes Richland County (IT) SYSTEMS ANALYST / PROJECT LEAD 1

Bridging Development and Operations: The Secret of Streamlining Release Management

SQL Server. SQL Server 100 Most Asked Questions: Best Practices guide to managing, mining, building and developing SQL Server databases

Software Configuration Management Best Practices

Releasing High Quality Applications More Quickly with vrealize Code Stream

Agile extreme Development & Project Management Strategy Mentored/Component-based Workshop Series

Managing Agile Projects in TestTrack GUIDE

Agile Business Suite (AB Suite)

CONTENT STORE SURVIVAL GUIDE

CONTINUOUS INTEGRATION. Introduction

BI xpress Product Overview

Global Software Change Management for PVCS Version Manager

Improving Cognos Upgrades Methodology to Lower Costs & Improve Upgrade Management

Smarter Balanced Assessment Consortium. Recommendation

Stellar Phoenix Exchange Server Backup

Database Build and Release will get started soon

Nexus Professional Whitepaper. Repository Management: Stages of Adoption

Microsoft Dynamics AX

Agile Development with Jazz and Rational Team Concert

Effective Release Management for HPOM Monitoring


How Silk Central brings flexibility to agile development

Continuous Integration (CI) for Mobile Applications

The Complete Performance Solution for Microsoft SQL Server

Using Git with Rational Team Concert and Rational ClearCase in enterprise environments

Continuous Integration. CSC 440: Software Engineering Slide #1

Developing Software in a Private workspace PM PMS

This presentation introduces you to the Decision Governance Framework that is new in IBM Operational Decision Manager version 8.5 Decision Center.

VMware vsphere Data Protection 6.1

EnterpriseLink Benefits

MOVING THE CLINICAL ANALYTICAL ENVIRONMENT INTO THE CLOUD

Backing Up CNG SAFE Version 6.0

Copyrighted , Address :- EH1-Infotech, SCF 69, Top Floor, Phase 3B-2, Sector 60, Mohali (Chandigarh),

Selecting the Right Change Management Solution Key Factors to Consider When Evaluating Change Management Tools for Your Databases and Teams

Source Control Guide: Git

Agile Software Factory: Bringing the reliability of a manufacturing line to software development

MS-40074: Microsoft SQL Server 2014 for Oracle DBAs

PRODUCT OVERVIEW SUITE DEALS. Combine our award-winning products for complete performance monitoring and optimization, and cost effective solutions.

Transcription:

Solving the database deployment problem with Visual Studio ALM and Database Lifecycle Management (DLM) 1

Contents Overview 4 Background: Application Lifecycle Management (ALM) and Microsoft Visual Studio ALM 5 Application Lifecycle Management (ALM) 5 ALM and continuous improvement 6 Microsoft Visual Studio ALM 6 The database challenge 7 Defining Database Lifecycle Management (DLM) 7 DLM with ALM, and continuous delivery 8 Implementing a robust data management strategy 10 Development and operations for databases with Visual Studio ALM and Redgate 12 What Visual Studio ALM and Database Lifecycle Management (DLM) mean: 13 For organizations 13 For development teams 14 For operations and database administration teams 15 How Redgate and Visual Studio ALM support DLM and continuous delivery for databases 16 Development 16 Integration and testing 18 Operations 20 Rollback / recovery and monitoring 22 Conclusion 22 Further reading and resources 23 2

"The journey of managing a team-developed application and its database involves many stages, and it is vital a process is put in place to manage all these people, their code, and their environments. "If all this isn't enough, the very purpose of having a database, keeping data, storing it, retrieving it, presenting it to the user, means that you can't just throw away yesterday's database. It has data. That data is kept for a reason. It may be vital financial information or it may be very simple lookup data that helps you maintain consistency. Regardless, it needs to be kept around and that makes the job of building databases even harder." Grant Fritchey, SQL Server MVP and product evangelist at Redgate 3

Overview Building great software is never just about the code. As Grant describes, it s also about managing multiple teams, timelines and, frequently, the steady evolution of customer requirements. If an application has a back-end database with business-critical data to be safely and correctly preserved, this brings additional, database-specific challenges for teams to solve. It s the complexity of managing databases that sits at the heart of Database Lifecycle Management (DLM), an approach that takes a wide-angled view across the software lifecycle, encouraging teams to use a range of processes and technology infrastructure to manage database changes. These processes coexist with, and should be managed within, the processes and policies used in Application Lifecycle Management (ALM). Microsoft s Visual Studio ALM solution is designed to help your team apply proven practices in the application lifecycle for project management, version control, continuous integration, testing, and managing the releases of application and database changes. This whitepaper explores how DLM can work with Visual Studio ALM to solve the database deployment problem, the important questions to consider for a strong data management strategy, and how Redgate tools provide support for the continuous delivery of SQL Server databases within DLM. 4

Background: Application Lifecycle Management (ALM) and Microsoft Visual Studio ALM Application Lifecycle Management (ALM) Application Lifecycle Management (ALM) has long been used by organizations to carefully manage the lifecycle of software applications from the point of inception through to delivery, monitoring, and maintenance. This means that the reach of ALM extends much further than just the software development lifecycle, covering three major stages: governance, development, and operations. From the very beginning of the lifecycle, ALM covers the process of generating, refining, and project managing requirements for the software application also known as governance. The governance stage broadly continues to the end of the lifecycle to ensure that the requirements are met and the application is then correctly maintained, reviewed, and updated as needed. The development phase of the lifecycle the creation of the software application comes in soon after the stage of refining and approving requirements. When the software application or changes to it are close to being ready to deploy, the operations stage begins. Its main aim is to safely manage and monitor the deployment to the customer. Implemented well, ALM supports teams by overcoming the challenges of multiple teams, and disparate processes, metrics, and infrastructures. Its three iterative stages facilitate an agile, repeatable lifecycle that is highly compatible with a continuous improvement approach towards evolving quality, efficiency, and communication. HOW DO ALM AND DLM FIT TOGETHER? Database Lifecycle Management is part of Application Lifecycle Management, specifically, the part that s concerned with the database. Application Lifecycle Management itself consists of three parts: governance, development, and operations. 5

ALM and continuous improvement ALM s three stages are designed to manage the flow of people, processes, and activities in an incremental approach to delivering software. The demand in the application lifecycle for feedback and measurement to drive quality and efficiency means processes and technology that can automate repetitive activities and provide increased traceability are important for a practice of continuous improvement. Microsoft Visual Studio ALM Microsoft s Visual Studio ALM solution is a set of integrated software development tools to help you manage the lifecycle of your application. In addition to the Visual Studio IDE for developing application code, Microsoft Visual Studio ALM also includes tools for project management, version control, continuous integration, testing, deploying, and monitoring your application changes. If you re using Visual Studio ALM to develop and manage your application changes, it s possible that you re using Visual Studio ALM to develop and manage the changes required for the database back-end of your application. The IDE that the database development teams are using to develop the database changes varies between teams some prefer to use Visual Studio and others prefer SQL Server Management Studio (SSMS). Whatever your team s preference for development environment, the ALM tools Microsoft provides for version controlling, continuously integrating, testing, and releasing the database changes provide a common integration point. This flexibility means that your team can build a strong release pipeline to support a range of working practices and business requirements both for the application and its database back-end. This consistency of process and diligence in both application and database changes is particularly important given the unique challenges databases present for the development and operations stages of ALM. "Agile Application Lifecycle Management (ALM) changes the way an organization develops software and its culture. It s great to see teams really benefitting from ALM as they bring transparency and collaboration to their processes. It s through this that continuous delivery evolves from a catchphrase into something that s truly achievable." Karsten Kempe, ALM consultant and Visual Studio ALM MVP 6

The database challenge Databases present a particular challenge for integration with an application lifecycle management strategy and it all comes down to protection of data. In contrast to applications, databases contain state, and this needs to be preserved as part of rolling out new or updating existing software. You can t simply discard or overwrite yesterday s database when you deploy database changes. To do this risks data loss, and maintaining data throughout the lifecycle is critical to the functioning, and, usually, legal compliance of any organization. It is out of the complexities that databases and their data bring to the application lifecycle that Database Lifecycle Management (DLM) has evolved. DLM brings additional processes, checkpoints, and controls to ALM in order to provide a more rigorous approach to managing the schema, data, and metadata for the database supporting an application. Defining Database Lifecycle Management (DLM) Due to the requirement to safely manage the schema, data, and metadata, DLM introduces additional processes to the development and operations stages of the application lifecycle: Data management and migration Data monitoring Data recovery These additional processes give greater visibility, predictability, and efficiency to database change management. 7

DLM with ALM and continuous delivery A database application development project depends on the successful coordination of the application and database development efforts. It is essential to be able to keep the scripts together so that any working build can be reproduced, and to do this, we need to be able to tie database changes to the code changes to which they relate. The way to do this is to check them into source control as a part of the same changeset transaction. This makes it far easier to make consistent builds and deployments. Grant Fritchey, SQL Server MVP and product evangelist at Redgate As Grant explains, to better protect the database and its data it is important that teams coordinate application and database development processes at specific stages of the lifecycle wherever possible. Teams working on the application and the database should also be aware of and support each other s practices. For further reading on team processes around agile development and continuous delivery, see the Organization and team processes category in our Patterns and Practices library UNIFYING TEAMS WITH ALM AND DLM By providing a common series of processes for development teams to manage the lifecycle of their applications and database back-ends, ALM and DLM facilitate: Greater collaboration on application features and their database requirements One system with shared metrics Coordinated points to receive and deliver feedback earlier and often These help teams to incrementally improve quality over the course of software lifecycles and in turn help organizations reduce risk at the point of release. As Grant describes, getting the database under version control lays the foundation for greater consistency and collaboration during a team s development of database changes. Otherwise, database development can too easily sit out of step with application development processes. If this happens, it can lead to a fragmented workflow, which in turn can hamper an organization s capacity for efficient delivery of software. 8

ALM, DLM, AND CONTINUOUS DELIVERY If, as part of ALM, your organization is practicing continuous delivery to automate application deployments across a pipeline, you can bring the same concept to your databases. Additional data management, migration, and monitoring processes are sometimes required but the broader benefits remain the same: Repeatability of processes, giving:»» Greater predictability over releases»» Efficiency by eliminating the repetition of a range of manual activities Faster speed of response to change through:»» The automated deployment of smaller units of change Greater reliability of the release process by:»» Providing a series of automated test stages prior to deployment Microsoft s Visual Studio ALM solution is designed to help your team apply proven practices at the development stage of the application lifecycle for version controlling, continuously integrating, testing, and managing the releases of application and database changes. Redgate tools extend the database change management capabilities of Visual Studio ALM to provide even more fine-grained support for the deployment of SQL Server database changes in a release pipeline. DLM Tools SSDT Development Version control Build automation & CI Requirements management Release management Project management TFS Application insights 9

This means that as part of the development and operations areas of a robust data management strategy, Redgate tools, together with Visual Studio ALM, provide your teams with a toolchain to manage your database changes across a release pipeline. Implementing a robust data management strategy A badly implemented data management strategy threatens not only the safeguarding of data, but also the performance of your application. Teams developing an application with a database backend, but without a robust data management strategy, can suffer from: The loss of some or all data within a database Inability to roll back to the previous version of a database Little or no traceability of database changes Risky database schema changes, with limited understanding of breaking changes Little or no testing of database changes prior to deployment Faulty (or missing) backup procedures Poor application performance due to ineffective database design Overly complex or outdated data structures Little visibility of the impact of changes deployed on the database, data, and the application Legal compliance issues with unknown changes of unknown origin A combination of changes to both team processes and technology can make a difference to your data management strategy and how successfully you manage your databases within the context of ALM and DLM. As you consider what you need to evolve as part of your data management strategy, there are three main questions for teams to consider for handling database changes across a release pipeline: What has changed in your database schema which objects were changed, removed, added? How can you migrate schema and data changes safely»» across multiple environments to your live production environment?»» when working with greenfield and brownfield projects? 10

Understanding the answers to these questions can help your team identify which tools and processes are required to build out a stronger data management strategy that takes into account the following areas: ALM area of planning Development Data management processes Database development Database change management Data migration Database testing Database tuning and performance Operations Database deployment Database release management Database backup and recovery Database monitoring 11

Development and operations for databases with Visual Studio ALM and Redgate For each of the data management processes in the development and operations stages of the lifecycle for database changes, you can use Redgate tools with Visual Studio ALM to evolve a robust release pipeline. Starting from a strong base of project management and requirements, your team can develop database changes as part of a rigorous testing and feedback cycle before the changes enter a review and approval gate prior to deploying the changes to pre-production or production environments. Development Operations Dev DB Dev Testing Dev DB Dev VERSION CONTROL Trigger CONTINUOUS INTEGRATION RELEASE MANAGEMENT QA Staging App Dev Dev DBA Review / approval Deploy MONITORING & MEASUREMENT App Dev Dev DBA Production Redgate tools Tools Redgate plugs into Team Foundation Server Team Foundation Build Release Management Application Insights Management Studio Data Tools Continuous delivery with DLM 12

What Visual Studio ALM and Database Lifecycle Management (DLM) mean: For organizations Greater consistency If the application and database development teams are working in parallel and coordinating changes through the same processes for the release pipeline, this increases the likelihood of the teams releasing more stable, consistent builds. Over time, this increase in predictability and process orchestration will greatly support governance in the application lifecycle, reducing waste, risk, and the likelihood of delays to software delivery. Agility and flexibility to change Greater coordination of application and database development processes also supports a more agile delivery of software by cross-functional teams. Following this way of working encourages a more flexible and rapid response to change. It also better supports automation and standardization of deployment processes through continuous integration and continuous delivery. Frequent, iterative releases Frequent, faster releases mean organizations have an opportunity to feed back to development teams sooner and work together much more incrementally throughout a project, rather than having to wait longer to see results and value from the investment in the project. Support for a first class data management strategy Organizations can also benefit from a first class data management strategy, allowing for application and database changes to go hand in hand and be deployed more efficiently as a result. Client-specific changes also become easier to manage as database and application versions for each client are better coordinated. 13

For development teams Databases keep pace with application development Orchestrating database and application development changes in the same project lifecycle processes brings important benefits to development teams. The database is no longer disconnected from the management of the project, ensuring that the database and its data are developed from the start for the lifecycle requirements. It is then much easier for databases to keep better pace with changes in the application. Earlier feedback for greater efficiency Automating database deployment processes brings further benefit too. Continuous integration and continuous delivery for databases encourage an iterative approach to development. Smaller, more frequent database changes mean that database development teams integrate their work sooner, see test results and feedback earlier, easily identify the cause of any failures, and are therefore better able to resolve any issues. This level of automation, combined with an agile approach to development, means teams also have a greater ability to release database changes frequently and faster. Changes get into the hands of users sooner, providing value to the business earlier in the project lifecycle. "The key word to associate with a CI server is 'automation'. You need to set up automatic deployment and automatic testing so that you can validate the build and deployment without involving a single human being. This provides a solid early warning system." Grant Fritchey, SQL Server MVP and Product Evangelist at Redgate 14

For operations and database administration teams Rigorous testing before pre-production Greater orchestration of processes earlier in the development part of the lifecycle carries through to database administrators in operations teams too. Rigorous, automated testing earlier in the pipeline, thanks to continuous integration and a thorough set of testing processes before a release reaches even pre-production, means that any problems are detected much sooner in a release pipeline than they might otherwise be. Teams can also use Redgate tools to incorporate production-like data into their testing cycles, making it possible to find data compatibility issues earlier in the release pipeline. Greater visibility and control before releases enter production Release pipelines can be engineered to include an approval gate before a release enters production. This enables database administrators to carry out further validation checks on deployment scripts. For example, take a build artifact (containing the database package that has been validated in CI and test environments). Database administrators can deploy this artifact against a staging or pre-production environment as a copy of production. This produces an upgrade script, which, when combined with the build artifact, produces a complete deployment artifact. Database administrators can then review this artifact, check for database drift in production, and if all is clear there - if the appropriate backup and recovery strategy measures have been taken and other approval tests have passed - finally deploy to production with greater confidence. Increased traceability and monitoring The placement of an approval gate, with restricted access, prior to releasing to a live production environment can be useful if your organization requires your deployment processes to segregate DBA and IT administrator duties as part of a separation of duties policy. Likewise, this stage gives further traceability of changes across a pipeline. If your team is already version-controlling and testing database changes at the development stage, your team can use tooling to provide further monitoring and reporting on each change as it passes from development into testing, QA, pre-production, and production environments. 15

How Redgate and Visual Studio ALM support DLM and continuous delivery for databases In a release pipeline, changes go through different stages, from coding to testing and deployment, with reporting and monitoring in a range of forms throughout. Each of these stages breaks down into sub-processes. Redgate tools work alongside Visual Studio ALM to provide support during the development and operations stages of DLM. Development: Database development Using Redgate tools, database development teams working with Visual Studio (VS) or SQL Server Management Studio (SSMS) can work in parallel or independently when coding database changes. Developers working in SSMS can make the most of Redgate s productivity tools when coding database changes: SQL Prompt - IntelliSense-style code completion and enhanced support for refactoring SQL Server databases SQL Search - quickly find SQL in SSMS, including fragments of SQL text within stored procedures, views, functions, and more SQL Doc - automatically generate database documentation SQL Data Generator - generate realistic test data for testing on local development machines SQL Dependency Tracker - explore object dependencies and visualize complex databases ANTS Performance Profiler - profile database requests made by a.net application Developers working in VS can also use SQL Prompt to support faster coding and share SQL snippets between team members. Similarly, they can also use SQL Search, SQL Doc, SQL Data Generator, and SQL Dependency Tracker against their build of the database. 16

Source control Version controlling database changes is a vital step for database development teams to ensure that they communicate their changes with others in the team, always have a version to roll back to if required, and maintain a solid audit trail. If the SSMS database developers are also working alongside VS developers, the version control system is a fundamental integration point for both teams. The build server, for example Team Foundation Server Build, will detect a change in version control and kick off a build process for the changed SQL file. "Source control provides several important solutions to almost all development processes, including: Ability to share code, allowing multiple people/teams to access pieces of code, or a database, at the same time A way to manage and protect the code generated A way to version each piece of code, so that a history of changes can be kept A way to version, or label, whole sets of code, so that you can deploy from a known state, or revert to a previously known state Facilitation of change management, so that new software or new functionality is carefully tracked and approved A way to support multiple development streams by allowing developers to create branches of the codebase for testing or for releases" Grant Fritchey, SQL Server MVP and product evangelist at Redgate Database developers working in SSMS can check in their database changes directly from SSMS using Redgate SQL Source Control, a plug-in for the IDE that integrates with your version control system, for example Team Foundation Server (TFS) or Subversion. Both SSMS and VS database developers are able to check in database schemas to version control. SSMS database developers using SQL Source Control can additionally check in static data and migration scripts. This additional flexibility means that VS and SSMS development teams can work in parallel and develop complex database changes much more easily and efficiently. The ability for SSMS database developers to check in static data using SQL Source Control means that teams can track any changes to, and migrate, any static (lookup/reference) data that s required for an application to function. This static data is typically non-transactional and updated infrequently (a table of US states would be one example). 17

SSMS developers working with SQL Source Control can also create and commit migration scripts to reduce risk when making specific database changes. WHAT IS A MIGRATION SCRIPT? A migration script is a user-supplied (manually generated) change script that is used during a database deployment. It provides flexibility for how schema and data changes are made. Users typically supply a migration script to supplement a change script that is automatically generated by a schema comparison tool, for example Redgate SQL Compare, when, for specific database changes, the schema comparison tool would be unable to interpret precise developer intent and would risk loss of data as a result. Migration scripts ensure that the developers intent is carried out and that data is preserved correctly. The specific database changes where it is important to add and check in a migrations script include: Table renaming Splitting or merging columns Changing the data type or size of a column Dealing with data motion and data transformation Adding a NOT NULL column without a default There are some important best practices on how to write and handle migrations scripts. For more information, see the Database Delivery Learning Program s Patterns and Practices library for free articles and guidance. Integration and testing Continuous integration of database changes As soon as database changes have been checked in to version control, a continuous integration (CI) process can be triggered that automatically builds and tests those changes in a CI environment. Teams using Team Foundation Build as their CI build server, for example, can configure the server to automatically pick up committed changes and run any series of tests that has been previously committed to version control via a series of build steps. 18

Database developers have several additional options available at the CI stage, using the Redgate DLM Automation Suite and other Redgate tools with Team Foundation Build as part of an integrated process. By configuring the build steps on the build server, SSMS database developers can specify command line switches with SQL CI (part of Redgate's DLM Automation Suite) to automate further tasks, such as generating test data and running tsqlt unit tests as part of the continuous integration process. These tasks are covered in four distinct steps in Redgate SQL CI: Build, Test, Sync, and Publish. FOUR CI STEPS SUPPORTED BY Redgate SQL CI Build - builds the NuGet database package, then generates and validates the database creation script from it. Test - generates test data using SQL Data Generator and runs tsqlt tests against the database package. Results are output in JUnit XML format. Sync - updates an existing database with the latest version in source control. Publish - publishes the database package to a NuGet feed artifact repository ready for deployment. Teams using TFS with Visual Studio can use the Redgate Team Foundation Build Plugin to automatically configure the four CI steps in Redgate SQL CI in TFS on behalf of the user. Additionally, the DLM Automation Suite can be used to run the command line version of Redgate SQL Doc to automatically generate database documentation. For more information and worked examples, see the whitepaper on continuous integration with Redgate tools. Deployment to testing and QA environments As part of continuous integration for database changes, the CI process should execute and validate the database build script. It can also run automated tests against those changes on a temporary test database. If those steps proceed successfully, it can also test the deployment of those changes against a target database in your CI environment. Any migrations scripts that have been checked in are also executed against the target database during this step. If this step is also successful, the CI build server publishes the package to an artifact repository, to be picked up by a release management tool. The build artifact contains a snapshot of a state of the database schema, any source-controlled static data, and any migrations scripts. 19

THE IMPORTANCE OF THE ARTIFACT The database deployment package artifact is important in the release process because it represents a version that has been validated through testing as part of the CI process. Once the artifact is stored in an artifact repository, it becomes a starting point for a release management tool to deploy to specified environments. Operations: Deployment to production Database development teams may run their database changes through different testing stages across multiple environments after the continuous integration stage. This could include acceptance, performance, and smoke testing among other stages, for example. Once the changes have passed those stages, there s an important review and approval gate as part of the continuous delivery process to help teams determine when the changes are ready to deploy to pre-production or production. This is usually where DBAs as part of the operations team step in. Although a database package has been validated in a CI environment (and potentially testing and QA environments too), it s important to recognize that the package hasn t yet been validated against an actual production server. This is an important part of the approval and review step before a release reaches production, and the DBA needs to take the build artifact from the repository and deploy this against a staging database, as a copy of production, in order to generate an upgrade script for deployment. It is the combination of the build artifact and the generated upgrade script that completes the whole artifact for deployment to production. The DBA can review the whole artifact to check whether it is production-ready. Database package build artifact + upgrade script = complete database artifact to deploy to production However, before deploying the complete database artifact to production, DBAs need to pause to ascertain whether there have been any changes to the database in production since they generated the validated script to accompany the build artifact. 20

Deploying changes to a production database in an unknown state could risk the deployment ending in errors, failure, downtime, or data loss. Unless an organization has absolute certainty in change management and permissions for making changes to a production environment, there s always the possibility that someone might have made a hotfix without telling the DBA, or that a change was made outside the scope of change management, for example in an emergency at 3am. To guard against the DBA missing these sorts of unexpected changes, commonly known as drift, DBAs can use Redgate DLM Dashboard to monitor and detect if any changes have occurred. When the DBA is ready to deploy, there are three different modes of production deployment, using Redgate tools going from the fully manual approach through to managed deployments via a preproduction environment: Fully manual DBAs can use SQL Compare to compare the structure of the pre-production database against production and generate a script for the differences. They can then review this script before running the updates against the target environment. Automated, one-click releases using a release management tool Alternatively, DBAs can use a release management tool to deploy database changes to various environments. This process uses Redgate s SQL CI functionality to firstly build a release (as a NuGet package), then secondly to use the release management tool, for example Microsoft s Visual Studio Release Management, to deploy that package to the target environment (using Redgate s SQL CI functionality integrated into the release management tool). This allows DBAs to deploy changes to database environments with just one click, automating the manual process outlined above. Managed deployments using a pre-production environment For many DBAs, the one-click deployment process doesn t provide enough control and insight for database deployments to production. For these users, Redgate s SQL Release (part of the DLM Automation Suite) functionality integrates with release management tools, such as Microsoft s Visual Studio Release Management, to provide the update scripts, change reports, and review steps needed to make database changes to production efficiently and safely. A DBA can review the changes that are about to be made, check that the pre-production and production environments match before carrying out the deployment (to make sure the upgrade to preproduction is an appropriate test for the production deployment), and then use the exact same script to deploy to production. 21

Rollback / recovery and monitoring An additional step to help safeguard against any unforeseen problems is to take a backup, for example using Redgate SQL Backup Pro, prior to deployment. There are further practices and patterns for a rollback and recovery strategy as part of the deployment step. For more information, see the Patterns and Practices library for articles and guidance. After database changes have been deployed to production, it s important to continue monitoring closely for any unexpected results or impact on the database or application. Application Insights in Visual Studio ALM enables you to monitor your live, deployed application. For example, you can see if CPU resources are overstretched, get stack traces from exceptions, and search through log traces. Redgate SQL Monitor can work alongside to collect performance metrics on your database and alert the DBA to any changes in the performance profile. For example, you can review the top most expensive queries being made to your database, spot unusual job durations, or identify performance bottlenecks using wait stats, among other potential performance troublespots. Redgate DLM Dashboard is also a useful monitoring tool to help teams identify and resolve database drift, by keeping track of database schema changes, and giving an overview of the state all of your environments are in. When databases change, DLM Dashboard sends you an alert both for notifications of unexpected changes and confirmation that intended changes happened successfully. On the web dashboard, you can also see the history of SQL changes line by line, who made the changes, and when. At that point, you can decide whether you want to roll back the change using the best practice for that particular change, or if you wish to deploy the change to more databases. Conclusion This whitepaper has outlined important considerations for solving the database deployment problem as part of the application lifecycle. It has also shown the possibilities for database development teams, working in VS or SSMS, to be able to orchestrate database changes in parallel and coordinate a more robust development and operations set of processes in a data management strategy. Whether database development teams choose to develop changes in VS or SSMS, the very first step on the path to continuous delivery of database changes starts at version controlling your database code. This vital step ensures that there is one source of truth for your CI build server to work from, enabling every committed change to be built and tested. 22

Once teams have the database under version control, they can use Redgate tools together with Team Foundation Build or their chosen CI build server to continuously integrate and test each change committed. A database package can be created as part of the CI process and, with minimal effort, deployed through multiple environments using a release management tool. That s not the end of the story This whitepaper has highlighted the advantages of Database Lifecycle Management (DLM). Further reading, resources, and rewards await: Reading You might be interested in three of the most popular articles on Simple-Talk that discuss further aspects of DLM: Continuous Delivery and the Database by Phil Factor Common database deployment blockers and Continuous Delivery headaches by Matthew Skelton Integrating Database Lifecycle Management into Microsoft's Application Delivery Process by Jason Crease Resources To find further resources on database delivery, and to download free trials of the Redgate tools mentioned in this whitepaper, visit www.red-gate.com/dlm-automation-suite/ Rewards DLM is an ongoing path that leads further rewards such as faster deployments, increased efficiencies, and reduced errors. If you d like to learn more about DLM, visit the DLM Patterns and Practices Library, a bank of free online content. With articles and tutorials, it covers key learnings that help you implement source control, continuous integration, continuous delivery, and monitoring for databases. Manual No source control, or manual, ad hoc source control Source control Automated source control solution Continuous integration Source control and CI for the database Continuous delivery Source control, CI, and automated deployment Monitoring Drift checking and performance monitoring If you have any questions, or if you d like a demo of how you can set up DLM for your team, please contact dlm@red-gate.com. 23