How do you get your database schema changes live on your production system? As your team of developers and DBAs work on changes to the database to support your business-critical applications, how do updates wend their way through dev environments, possibly to QA, hopefully through pre-production and eventually to production in a controlled, reliable and repeatable way?

In this article, I describe a model we use to try and understand the different stages that customers go through as their database change management processes mature, from the very basic and manual, through to advanced continuous delivery practices. I also provide a simple chart that will help you determine how mature your process is.

This method of managing changes to the database – which all of us who have worked in application/database development have had to deal with in one form or another – is sometimes known as Database Change Management. And it’s a difficult process, often painfully so.

Some developers take the approach of: “I’ve no idea how my changes get live – I just write the stored procedures and add columns to the tables. It’s someone else’s problem to get this stuff live. I think we’ve got a DBA somewhere who deals with it – I don’t know, I’ve never met him/her”.

I know I used to work that way because I assumed that making the updates to production was a trivial task – how hard can it be? Pause the application for half an hour in the middle of the night, copy over the changes to the app and the database, and switch it back on again. Voila!

But somehow it never seemed that easy. And it certainly never was that easy for database changes. Why? Because you can’t just overwrite the old database with the new version. Databases have a state – more specifically, 4Tb of critical data built up over the last 12 years of running your business. If your quick hotfix happens to accidentally delete that 4Tb of data, then you’ll be looking for a new role pretty quickly after the failed release.

There are a lot of other reasons why a properly managed database change management process is important for organizations, besides job security, not least:

1. Frequency of releases. Many business managers are feeling the pressure to get functionality out to their users sooner, quicker and more reliably. A really good book which I highly recommend, Accelerate by Nicole Forsgren, Jez Humble and Gene Kim, shows how companies can measure – and improve – software delivery performance using four key metrics. (Our software development teams at Redgate use the same metrics and it works extremely well.)
2. Auditing and compliance. The old SOX and HIPAA regulations have now been joined by other compliance frameworks like the GDPR in the EU and the CCPA in the US. One common thread is that they all demand companies implement proper processes for managing changes to their databases. That applies whether they’re managing schema changes, making sure the data itself is being looked after correctly, or introducing other mechanisms that provide an audit trail of changes.

We’ve found, at Redgate that we have a very wide range of customers using every possible form of database change management imaginable. Everything from: “Nothing – I just fix the schema on production from my laptop when things go wrong, and write it down in my notebook”, to: “A full Continuous Delivery process – any change made by a dev gets checked in and recorded, fully tested (including performance tests) before a (tested) release is made available to our Release Management system, ready for live deployment!”. And everything in between of course.

Because of the vast number of customers using so many different approaches, we found ourselves struggling to keep on top of what everyone was doing – grappling to identify patterns in customer behavior. This is useful for us, because we want to try and fit the products we have to different needs – different products are relevant to different customers and we waste everyone’s time (most notably, that of our customers) if we suggest products that aren’t appropriate for them.

If someone visited a sports store, looking to embark on a new fitness program, and the assistant suggested the latest $10,000 multi-gym, complete with multiple weights mechanisms, dumb-bells, pull-up bars and so on, then the store is likely to lose that customer. All he needed was a pair of running shoes!

Introducing the Customer Maturity Framework

To solve this issue and attempt to simplify how we understand our customers and our offerings, we built a model. This is an attempt at classifying our customers into a ‘Customer Maturity Framework’ as we rather grandly term it, which somehow simplifies our understanding of what our customers are doing.

The great statistician, George Box (amongst other things, the ‘Box’ in the Box-Jenkins time series model) gave us the famous quote: “Essentially all models are wrong, but some are useful.”

We’ve taken this quote to heart with our own model – we know it’s a gross over-simplification of the real world of how users work with complex legacy and new database developments. Almost nobody precisely fits in to one of our categories, but we hope it’s useful and interesting.

There are actually a number of similar models that have existed for a long time for more general application delivery like these from ThoughtWorks/Forrester and InfoQ. Initially we tried just taking these models and replacing the word ‘application’ for ‘database’, but we hit a problem. From talking to our customers we know that users are far less further down the road of mature database change management than they are for application development.

As a simple example, no application developer who wants to keep his/her job would develop an application for an organization without source controlling that code. Sure, they might not be using an advanced Gitflow branching methodology but they’ll certainly be making sure their code gets managed in a repo somewhere with all the benefits of history, auditing and so on. This often isn’t the case for the database. A very large segment of the people we speak to have no source control set up for their databases whatsoever, even at the most basic level like keeping change scripts in a source control system somewhere. By the way, if this is you, one of the most viewed posts on the Redgate blog is An introduction to SQL Source Control, which explains what it is, what it does, and the advantages it brings to day-to-day development.

This difference in maturity in database change management is the same as you move into areas such as continuous integration (common amongst app developers, relatively rare for database developers), and automated release management (becoming increasingly common amongst app developers, growing for the database).

So, when we created the model we started from scratch and biased the levels of maturity towards what we actually see amongst our customers. But what are these stages? And what level are you? The table below describes our definitions for four levels of maturity – Baseline, Beginner, Intermediate and Advanced. As I say, this is a model – you won’t fit any of these categories perfectly, but hopefully one will ring true more than others.

Where do you go next on your database change management journey?

Does this model reflect your team at all? Where are you on this journey? We’d be very interested in knowing how you get on. We’re doing a lot of work at the moment, trying to help people progress through to continuous delivery for the database, often as part of their DevOps transformation.

For example, if you’re currently not source controlling your database, then it might be worth exploring the business advantages of doing so. If you’re already source controlling your database, what about the next steps – continuous integration and automated release management?

To help understand these issues, there’s a library here on Simple-Talk to help you get up and running with the tools you need to progress with database change management. All feedback is welcome and it would be great to hear where you find yourself on this journey.

This post was originally published in 2014 and has remained relevant – and popular – ever since. It was reviewed in August 2020 to ensure the content is up-to-date and still of value to Simple Talk readers.