Matt Hilbert

15 January 2018

Bringing DevOps to the database. Part 1: Version control

For some years now, DevOps practices have been exciting application developers with their promise of short iterations, fast releases, and features that get into the hands of users sooner. Those same practices are now entering the database space, but how can database development adapt, and where should it start?

DevOps has been claiming converts everywhere. No surprise there. Developers like it because it streamlines processes, improves software quality, automates repetitive tasks, and supports continuous delivery, freeing them to concentrate on what they’re good at: coding.

Companies like it because, as shown in the State of DevOps Report from DORA and Puppet Labs, DevOps practices lead to higher IT performance, which in turn increases productivity, profitability, and market share.

But what about database developers and administrators? They don’t deal with anonymous code that, if it breaks, can be rolled back and fixed at leisure. They handle live data and if that breaks there’s a problem. A big problem.

Hence the belief that applying DevOps practices like continuous delivery and the automation of laborious tasks to database development is a monstrous exercise. It means introducing new techniques, unfamiliar processes, and untried tools to existing practices, risking faster database deployments more often when current deployments are already hairy enough.

Or does it? By applying DevOps thinking to databases, the development process becomes leaner and faster, there is less waste, and quality is built into the code from the start, avoiding many deployment problems later.

The rules of the game have changed

Truth is, while DevOps has been sweeping through application development, there’s been a lot of movement in the database development arena too. It’s a natural extension because business is moving faster, features need to be released sooner, and the database can’t be a bottleneck.

In database development, testing, and deployment, there are now tools and processes that can be adopted alongside those used in application development. They relieve the burden on database administrators (DBAs), make testing easier and faster, and turn deployments from occasional big bang releases full of worry to frequent releases that are simple and error-free.

That’s quite a change and, perhaps surprisingly, it’s relatively easy to start doing database DevOps.

Get everyone on the same page

The first step is to think collaboration not competition. In the well-known novel about DevOps, ‘The Phoenix Project’, there’s a wonderful comment by the major character, Bill Palmer, about the division between Dev and Ops:

You can’t just throw the pig over the wall to us, and then high-five each other in the parking lot, congratulating yourselves on how you made the deadline. Wes is telling us that the pig will probably break its leg, and it’ll be my guys who work all-nighters and weekends to keep that pig alive.

This divide doesn’t need to be there – and with DevOps thinking for the database, can’t be there. The goal is to move to smaller, more frequent releases, with development and operations teams communicating and always cooperating on the bigger issue: getting the right software out there.

Choose the right tools

The second step is to remember that introducing database DevOps means enhancing the way you work. That doesn’t mean employing tools that enforce change and impose rigid, unfamiliar rules, tools, and processes on everyone. Instead, it calls for using the tools you already have in a better way.

You could, for example, plug into the same source control, continuous integration, and release management tools used by your application developers to version control, test, and deploy your database changes.

That way, rather than introducing an untried tool that does the job but imposes unwanted controls over people, you enrich the way you work by doing more with database tools that work alongside existing application development tools.

By default, this also brings the added advantage that application and development teams work together toward the same goal, and there is true integration across the development process.

Start small

Continuous delivery, continuous integration, automated release management – the phrases alone are enough to make the talk about DevOps for the database sound more complicated than it actually is.

The key is to start in a small way, by source controlling, or version controlling, your database first. Source control is the enabler for DevOps practices such as continuous integration. Without it, you can’t do true DevOps. By adopting it, you prepare the way for DevOps for the database.

The bigger advantage is that by source controlling the database as well as the application, truly atomic commits of full application and database changes can be made. And with changes to the database code tied to changes to the application code, any working build can be reproduced at any time.

Tribal memory is not source control

The manual, script-driven process that many companies and organizations still rely upon is not source control. This approach is typically built around the close understanding of the database by one person or a small group, and a memory of what worked before and what will probably work again. Developers and DBAs work directly with scripts rather than the database, a history of changes is hard to maintain, and additional processes are needed to ensure objects are scripted in dependency order, and referential integrity is maintained.

This is a fear of source control based on the fact that, unlike application code, database code is all about persistence, so the data and structures that already exist must be protected at all times.

This is said despite the fact that source control delivers significant advantages to almost all database development processes. Multiple people or teams can all have access to pieces of code, or a database, at the same time, for example. Each piece of code can be versioned so that a history of changes can be maintained on an ongoing, auditable basis. And whole sets of code can also be versioned, giving you the ability to deploy to a known state, or revert to a previous state, with ease.

As importantly, perhaps, with a source control system in place, the very practice of source control becomes a normal part of everyone’s daily working life, rather than the domain of a limited number of people. So if issues do arise, they can resolved quickly instead of becoming a potential crisis.

Keep it simple

As has been seen, source controlling database code offers big advantages. Those same advantages would be negated if, in pursuit of them, the job of developers and DBAs became more complex instead of being made simpler.

Introducing a source control tool that forces developers to work differently, for example, with policies that are foreign to existing practices will simply irritate them. Similarly, choosing a tool that is separate from your application source control system will continue the divide rather than bridging it – and deny you the advantage of having the application and database code together at the same point in time.

The key is to integrate database source control into your current development process so that it aligns to the tools and practices developers are already familiar with. Inevitably in the SQL Server world that means working alongside – preferably inside – SQL Server Management Studio.

With a plug-in source control tool, developers can continue to work in the environment they’re comfortable with. It automates the source control of object creation scripts, alerts users to differences between the database and source control versions, and makes it simpler to commit changes to source control.

With source control as part of the normal database development workflow there are productivity gains too. It takes less time to do, makes it less likely that committing code to source control will be forgotten, and reduces task switching, improving efficiency.

Source controlling the database alongside the application is not only possible, it delivers many advantages. It synchronizes your database structure across the development, test, and production environments, reducing the work – and the errors – involved. Additionally, it ensures database development teams communicate changes with others, provides a version to roll back to if required, and helps maintain a solid audit trail.

And of course, it opens the door to DevOps practices like continuous integration.

Read part 2 of ‘Bringing DevOps to the database’ to see how database development, once source control is in place, can be included in continuous delivery processes.

If you’d like to know more about how database DevOps can help you deliver value faster while keeping your data safe, visit our online resource pages.