Matt Hilbert

29 August 2017

DevOps and the database: what’s going on?

If you’re doing DevOps differently for your database and the rest of your system, there’s something wrong. There shouldn’t be a separate conversation about it, says Microsoft’s Donovan Brown.

When DevOps was first talked about in Flickr’s seminal ’10 deploys per day’ Velocity presentation in 2009, it was regarded by some as strange and alien to corporate culture. It was the antithesis to the accepted way of doing things, a threat to established and proven processes: it was downright dangerous.

Times have changed. Unicorns like Uber, Airbnb, Spotify and GitHub couldn’t live without it. Microsoft has embraced it and actively promotes it. It’s become the key to gaining competitive advantage in a technology marketplace that is moving faster and changing quicker than it ever has before.

But what about the database? Why are practices like version control, continuous integration and automated deployment being introduced to application development but left on the shelf when it comes to the database? The 2017 State of Database DevOps survey, for example, found that continuous integration was in place for 39% of applications, but fell to 20% for databases. What’s going on?

In search of some answers, I spoke to Donovan Brown, Principal DevOps Manager at Microsoft and DevOps advocate. Also known as The Man in the Black Shirt, his unofficial tagline is #RubDevOpsOnIt. He lives DevOps. In fact, you can’t stop him talking about it and enthusing about its advantages.

He describes DevOps as: “The union of people, process, and products to enable continuous delivery of value to our end users.” The most important of those three is people. It’s also, conversely, the most difficult to fix.

People are creatures of habit.

“If you’ve been doing something the same way for 30 years,” he says, “why would you fix it? We all resist change, even if change is for the better, and DevOps requires a lot of change.”

A large part of that needs to come from the Ops side of the equation because, while processes like version control and continuous integration have become second nature to developers, Ops teams don’t traditionally do them on a daily basis. In fact, Ops are often seen as the opponents of change.

Donovan does, however, hold his hand up as being partly responsible for the division. “Historically,” he says, “they’ve had to become the gatekeeper because developers like me 20 years ago were wreaking havoc on production servers, running on there with admin credentials. Their job in my opinion has never been to deploy software for us. Their job is to protect and manage the infrastructure, to make sure it runs as efficiently as it can. That history, though, stays with us because there is so much regulation and red tape to get code from the fingers of developers to the hands of users.”

He sees DevOps as the solution to the problem. “What I’m hoping is that DevOps will help automate what we’re forcing them to do, which was never their job, and rebuild the trust between Dev and Ops. We really need to help Ops people realize that we’re not trying to replace them, we’re trying to empower and enable them to allow us to continuously deliver value.”

It’s not down to Ops teams on their own, however. C-level executives also have a part to play because they often favor a traditional approach to software development, with targets and long term strategies standing in the way of DevOps.

“What I’ve noticed in a lot of companies that are struggling to adopt DevOps is that they’re not getting the backing from the executive level,” Donovan says. “You have this waterfall mentality from the top that still wants GANT charts and milestones, and things that just do not line up with agile thinking at all. I’ve had the best success implementing DevOps in organizations where the C-level executive got it. When you have a grassroots team who get it, but no one else does, they usually get snuffed out by tradition.”

The database is not separate from Donovan’s goal of people, process and products coming together. Prior to joining Microsoft, he was a Process Consultant and Scrum Master, helping a range of companies introduce DevOps to their development practices. He made sure the database was part of that.

Then, as now, including the database was a challenge, but Donovan worked around it, using the technologies available to allow customers to treat the database just like every other element in their DevOps solution.

He does, however, admit it was hard work. “It’s actually the most difficult part of the pipeline to automate because of the risk of failure,” he says. “You can easily roll back a front-end – you can’t do the same for your database because any transactions that have occurred during the deployment could potentially be lost. So, it’s a crucial, integral part of your DevOps pipeline and I challenge any company that claims to be doing DevOps that only does it for the front-end but their back-end is still done manually. To me, they’re faking it.”

Infrastructure as code should not be a special snowflake.
It’s just part of your solution.

The key is to remember that the code written to make changes to a database is just that: code. Traditionally, it has been managed outside of and separate to application code, with database changes often managed on-the-fly, at the last minute. Consequently, deployments have become worrying at best, with teams on hand for the expected firefight against downtime.

Include it in version control, however – and continuous integration and automated deployments – and the database can be developed in parallel with applications. Problems are then picked up earlier, deployments become predictable and reliable, downtime and rollbacks are a thing of the past.

What has made this even more appealing is the tooling now available – software that in Donovan’s days as a consultant was only just emerging. As he admits: “The tooling had to mature to allow us to get over a lot of the hurdles that are unique to database deployments so that we can have everyone incorporated into the DevOps pipeline, not just the unicorns. Everyone should have the power available to them to be able to take their databases and incorporate them as part of their DevOps pipeline. Until you do, you still have that being the bottleneck.”

There is a price to pay, however, if by introducing tools to bring DevOps to the database, companies get bogged down by what Donovan calls ‘the vendor explosion’.

Every vendor you add to your pipeline
also adds integration tax.

Integration tax is the cost to companies, in terms of training and introducing unfamiliar ways of working, in order to use a new piece of software that resolves a piece of the DevOps puzzle. If that cost is too high by, for example, moving people out of the development environment they’re comfortable with, it betrays the ideals of DevOps. One onerous way of working is simply replaced by another.

Donovan is keen to remove that tax by encouraging companies to work with as few vendors as possible – and preferably those vendors who have already paid the integration tax in advance. By, for example, introducing tools that work inside or alongside existing software, thereby reducing the time it takes to become familiar with them, and making the most of the current infrastructure rather than adding to it.

Otherwise, Donovan concludes, “You’re paying tax for that integration. When we at Microsoft have partners who create extensions for our products, it’s the vendor and Microsoft who pay the tax in advance.”

Finally, I asked Donovan what the biggest challenge was for companies who adopt DevOps for the database. His answer was the most telling:

“If you’re doing DevOps differently for your database and the rest of your system, to me there’s something wrong there. When I think of DevOps, the database is part of that. I don’t need a separate conversation about it.”

This interview was originally published on DevOps.com in August 2017.