WHITEPAPER DATABASE CHANGE MANAGEMENT 5 barriers to database source control and how you can get around them 91% of Fortune 100 companies use Red Gate
Content Introduction We have backups of our databases, so in a way, we already have source control We ve never had a problem before, so what we ve been doing must be good enough and we don t need to change We re too small/large to recoup the cost of development time and a 3rd-party tool Database source control requires a change to a way the team works, and it isn t worth it to try and get everyone on board Database source control is too hard to ask the team to learn Learning more Conclusion 3 4 5 7 9 10 11 11 2
Introduction Many businesses have a process in place to keep track of database changes, whether the team has a process to handle manual scripting, a couple of different source control systems stitched together to do the job, or even a single source control system that isn t fully-implemented. None of these solutions allow the business to enjoy the reliability, security, efficiency, and cost-savings of database source control. Developers need to be able to build and deploy their application from scratch should the need arise, and database scripts and schemas are an integral part of that process. Despite this, research shows that 24% of developers are still spending far too much time trying to manage database changes, bug fixes, and deployments because they aren t using source control. Red Gate has been speaking to our customers and industry experts to shed some light on what is stopping businesses from reaping the benefits of source control. In this paper we ll look at five potential roadblocks and how to get safely onto the path of database source control. Developers have more time to develop, since they no longer need to manage deployment scripts and they no longer need to deploy them. We were able to identify differences between instances of our databases in different environments. In some cases the changes were minor, in other cases the changes were quite large (e.g. missing columns and different stored procedure parameters). We now have fewer issues when we deploy changes to production. We have the ability to roll back changes. Ernest Hwang, Practice Fusion 3
We have backups of our databases, so in a way, we already have source control Backups are good. Backups are important. Backups are not source control. Here are a couple of the salient differences: Revision history Essentially, a backup is there to preserve the latest version of the database. This will be overwritten when space is needed, if not immediately. Conversely, source control keeps every version. Restoring a specific previous version of the database is a basic task of source control (essential during disaster recovery or bug hunting), but is complex or even impossible using only backups. Each change made to a database in source control is done through a commit. Commits are recorded so there is a history of each change that is made, who made it, when, and why. This is especially useful for auditing and checking regulatory compliance. Multiple Concurrent Universes Most source control systems allow the ability to branch and merge, a much simpler task than performing endless COPY and DIFF commands on a backup. The idea behind this is to make working across teams easier and more efficient, even when team members aren t in the same place and without an endless stream of emails with backup scripts. This also means that developers can use branching to work on separate tasks at the same time without getting in each other s way. Safeguarding all changes Backups are triggered by the passage time (e.g., every x hours, nightly, weekly) regardless of how much work is being done on them. When actively working on the database, the backups will always be out-of-date until the next one is taken, and when it is, you d have to perform a diff to see the differences between versions. If you need to roll back the database for debugging or recovery, you don t want it to take a lot of time. If you re debugging, you ll have to know which version of the backup you need. You may have a full backup for each day of the week, but doesn t mean finding the right backup file won t be time-consuming. 4
Having a backup and restore strategy is no substitute for database version control. It addresses business risks around securing data, but it s a poor fit for real database change management. For a real history of schema changes, a canonical latest version, and for easy deployment, you need to have the database under version control. We ve never had a problem before, so what we ve been doing must be good enough and we don t need to change This is a common misconception, often justified with the maxim: if it ain t broke, don t fix it. This is not the way you want to ensure the reliability and security of your business data. Just because you haven t suffered the pain of missing or corrupt data with no way of retrieving it before, that doesn t mean that that your current system is robust enough to exist without source control, it just indicates you re currently operating with a high degree of luck. How do you currently ensure the reliability of your data? How easy would it be for a developer to accidentally make a disastrous mistake? How would you go about recovering from that mistake? If you needed to restore a previous version of the database from last year, how long would it take to retrieve that data from the offsite storage where it s hopefully held (if it hasn t already been deleted)? If you don t know the answers to these questions, you may have a problem. Consider how a disaster would affect your business, even if it s only a full day of lost development time. Database source control is vital to making sure your business applications can survive a physical disaster, or a disastrous accident. Troy Hunt once ran a competition for his readers to contribute their most painful moments during life without source control. If your team uses what s referred to as Cut and Paste Versioning these stories are a must-read to see what can go wrong. 5
Can you really blow away all your development environments and retrieve everything you need to recreate it? I mean completely recreate it so that the app actually, you know, works?! Troy Hunt Source control increases the transparency of the development process, which has a couple of benefits. With a centralized source, the team lead always has an accurate view of how the project is progressing. With versioning, if a problem is discovered, your team can quickly revert changes and restore a previous version. In addition to this you can see each change that s been committed per version, which means you don t have to open files and run a diff to look for the problem line by line, which has the potential to waste hours of development time depending on the size of the database. Every release involved a lot of finger-crossing because we could never be 100% sure what we were testing accurately represented the production environment when we deployed. Ernest Hwang, Practice Fusion Source control is touted as good development practice, but what does this actually mean? It means the team can work together more efficiently by ensuring everyone is working on the most current version of the database without endless back and forth between team members to make sure everyone has all the latest changes, the commit history will also document who made what change, when. It means that there s a greatly reduced chance of accidentally overwriting another developers work, which can cause a halt to development and waste time. Finally, if you ever have a compliance audit, source control makes conforming to the regulations much easier by having a complete history of changes/ audit trail. 6
We re too small/large to recoup the cost of development time and a 3rd-party tool Even if your development team is a team of one, you need database source control. At its most basic, source control works as a safe, reliable undo button. Even a single developer can make a mistake that needs to be rolled back, or a way to properly revert a database for debugging. Small teams often grow into larger ones, and what was once a workable process becomes more complicated as new members join. The larger the team is, the more necessary it is to have a con-trolled system for maintaining database code. Return on Investment There is no hard and fast equation that covers every business to determine the return on investment from database source control, though some have tried. IBM produced a document called ROI: The Value of Deployment Automation which includes a section on database source control. It attempts to calculate the value of benefits like error-prevention, automating manual work, and reduced overtime from release engineers. Results will vary business-to-business, but the document makes a number of good arguments. We have found that the most obvious benefits to your team and business will depend on the state of your current process for handling database changes. We have no automated source control system For those in this situation, there should be an immediate noticeable reduction in complexity and time lost during common tasks. Being able to tell which database version is being used in the various environments, simplifying rollbacks, version control. Every benefit of switching to source control will apply. 7
We have our own home-grown solution You have your own system in place that has been working for you, the team has been making changes to it over the years to fill the gaps that have been discovered. But these systems can be brittle, and it s hard to know how it will hold up under more pressure than usual, or if something goes wrong that you haven t encountered before? Your team will see themselves saving the time they would normally have to spend maintaining a solution that wasn t fully doing the job anyway. It might mean only saving 15 minutes a day as was the case with Practice Fusion, but those 15 minutes per developer will add up to significant savings over time. Our ROI has been high, not because we are writing more code, but because we are doing less maintenance, and encounter fewer conflicts in work. This enables us to have a larger team with more separate lines of development than might otherwise be possible Annette Allen & Dave Green, First Databank We have a number of systems, each one for something different This is a situation that many businesses find themselves in. When they were a smaller business they had one system, but once they started growing they found they needed new functionality which they added by stitching in another system. These solutions tend to scale especially poorly, and can easily crack when tested in a non-standard situation. A single database source control system will be more reliable and much simpler for the whole team to use. The time saved hunting down a bug in a previous version of the application, or being able to create a current version of the database without downtime in the event of a disaster, will be even more apparent when scaled to larger organizations. 8
Database source control requires a change to a way the team works, and it isn t worth it to try and get everyone on board Change can be difficult but when it comes to source control, it s worth it There are many examples of businesses that have benefitted from source control, between better collaboration across the team, moving away from manual and inconsistent authoring of SQL scripts, fewer problems during deployments, and more. Practice Fusion documented their frustration with working without a single source control system, and what introducing a single system with Red Gate s SQL Source Control did for their business. Using a tool like Red Gate s SQL Source Control means that the team will not need to make significant changes to the way they work. You don t need to go out and buy a separate source control system for your databases, you can just connect SQL Server Management Studio to the system you use for your development code. SQL Source Control connects SQL Server Management Studio to the source control system you re using, so you won t need to switch between environments, or change where you re working (as you would, for example, if adopting Microsoft s DBProjects). Phillip Securities found that when they decided to move to database source control, using SQL Source Control made the switch much easier. Adding SQL Source Control to your workflow is also the first step to thinking about continuous integration and automated deployments for database changes. First Data Bank talk about their move to a new source control system while introducing continuous integration at the same time (a major change to their development and release process). It s a great read if you re looking into introducing continuous integration. 9
Database source control is too hard to ask the team to learn Maybe you ve tried an implementation in the past which has poisoned the waters. Maybe you re afraid that your senior developer/team lead doesn t have the time to teach the rest of the team. No matter how this misconception took root, you can t let it fester. If you trust your team to be responsible of your databases without source control, they can definitely handle learning a new tool. Yes there are some differences between application and database source control which make database source control seem complex by comparison, but that s just it, by comparison. Adding source control will simplify your database deployment process, not add complexity. If you have specific concerns about the system providers, then the amount of support and help documentation each one offers should be taken into account when you look into your options.there is also plenty of information out there to help you get started. People learn in different ways, there are articles, videos, and webinars to build your confidence and get you and the team on track. When we tried SQL Source Control, it stood out against the competitors because it connected to SQL Server Management Studio (SSMS) with no separate interface. It was easy to use and we could link it to Subversion, our version control system, and close a major gap in our workflow. Phillip Securities 10
Learning more We ve collected a set of resources to help you learn more about database source control and change management, as well as running a series of learning webinars for teams. Articles: Database Source Control Basics series by Michael Sorens Database Source Control- The Crib Sheet by William Brewer The 10 Commandments of Good Source Control Management by Troy Hunt Videos: Team Based Database Development with Version Control by Steve Jones Best Practices for Database Deployment - Grant Fritchey & David Simner Conclusion Replacing your current process with a reliable source control system is not as hard as many people imagine, and introducing SQL Source Control at the same time means that your team will not even need to significantly change the way they work. Database source control will save your team time spent on debugging, disaster recovery, and change management, as well as eliminate many of the unique issues you find maintaining your current setup to keep it running. If you re ready to find out more about how to implement database source control in your environment, feel free to get in touch and arrange a consultation or attend one of our webinars. 11