Database Lifecycle Management: what it is, what it does, and why it’s so important

A couple of years ago, Redgate adopted Database Lifecycle Management (DLM) as a unifying context for our existing and future database tools. But what exactly is DLM? Why is Redgate so keen on it? Is it a change in direction, an acknowledgement of customer demand, or just a different way of describing the DevOps thinking behind the tools? And is Redgate suggesting organizations who use databases should adopt the approach? Simon Galbraith, CEO and Co-Founder of Redgate, explains DLM by answering the most common questions that come up.

What is DLM?

Database Lifecycle Management, or DLM, describes the big picture of developing, deploying, and maintaining databases more easily. It’s a natural partner to Application Lifecycle Management (ALM) because at Redgate we believe all the things you can do in ALM like source control, managing changes, and making the development process more efficient and productive, should be made available for the database.

In fact, we think if anything it’s more important you do it with the database because, when you make a mistake with your application, you can delete it and start again. When you make a mistake with your database, you’ve potentially permanently corrupted your data.

Despite that, the database has been a neglected aspect of agile development and DevOps. DLM is changing that.

Why is Redgate so keen on DLM?

SG1It all started when we first released SQL Compare and SQL Data Compare over a decade ago. We wrote the tools initially to solve our own database deployment problems, and they were enthusiastically welcomed by others having similar problems with database deployments. SQL Compare performs a sort of dark magic that replaces lots of hand-coding and having to really investigate things, and is a wildly successful and popular bit of technology.

What intrigued us was that the SQL Compare tools weren’t just solving one problem. Developers and DBAs were putting them to many different uses, ranging all the way from build, integration, release, and testing through to deployment.

The one thing all these pain-points had in common was that, ultimately, they were about the delivery of database-driven applications. Due to the similarity with ALM, we decided to start referring to these pain points in the context of Database Lifecycle Management, and we’re quite excited to see it’s now a fairly established term used in the community and by other vendors when they talk about DevOps for databases.

Looking at the problems in more detail, what struck us was the fact that they were quite different from those in application development, and the techniques database developers were using were unfamiliar too. Yet we believed some of the solutions derived from application source control and versioning were relevant.

Basically, the technology was being used throughout the lifecycle of the database. So in that sense, we’ve always been doing aspects of DLM, though we didn’t call it that at the time.

Why is DLM gaining traction now?

Over the last few years, there’s been a huge rise in Agile – the idea you should ship changes to applications much, much more frequently. It’s true at Redgate too. We used to try and release products once a year, or once every six months and, just like everyone else, we found what we thought was ready to release wasn’t. Actually shipping to production was a significant amount of the work – as a rule of thumb, somewhere between a third and a fifth of the total amount of work on a project.

Now it’s all changed. By working in an Agile way and taking a DevOps approach, we try to ship once every two weeks and many of our teams will try and ship something every day. We want to emulate that for the database side of things because there’s no point shipping changes to your applications more rapidly and efficiently if your database development can’t keep pace.

Why should an organization adopt a DLM approach?

Probably the greatest benefit is predictability. Once you’ve put in place version control and processes such as continuous integration and automated deployments, it’s relatively painless. Chances are, you’ve taken a lot of the guesswork out of the process and can predict when a release will get into production.

By automating processes such as integration testing, code quality checks, and scalability testing, you gain more confidence in what you’re releasing. And by having the discipline of versioning databases, configuration management, and source control, you can allow branches, special features, and easier development. You avoid losing ad-hoc patches such as indexes, and have more protection against unauthorized changes on the production system. It also makes it easier for Operations staff to make checks on configuration issues without having to access the server.

As importantly, by making the process more visible to the various teams and functions involved with a release, you make cooperation and adaptive teamwork easier, and introduce a DevOps culture that benefits everyone.

How do organizations actually start doing DLM?

The foundation stone of DLM is source control. Once you have your database under source control, you can introduce other elements of the DLM process – and it’s easier than it sounds.

Take application developers. They source control application code as part of their normal working lives. SQL Source Control allows database developers to treat database code in the same way, providing a single source of truth for the database schema.

In fact, you can even keep static data in source control so the functional database can be built, just like you build your application. That’s important because if you’re making lots of changes, or several people are working on changes, it’s a much more robust process than just changing a database, and then having the database you’ve changed being the thing you push to test and deploy into production.

What’s the next step in DLM?

DLM processWe were talking about source control but that’s just the beginning because DLM brings all of the loose ends of the development process together in a coherent way. Alongside SQL Source Control, Redgate’s SQL Toolbelt contains all the tools you need for Database Lifecycle Management, including automation tools to simplify database continuous integration and automated deployments.

The idea is you can make a change as a developer and, as you commit the change, it will be tested and pushed through to a staging or even a production environment. If it fails the testing, you know straightaway rather than being surprised when it doesn’t build later, and you can resolve the issue before moving on.

Crucially for our customers, Redgate tools plug into the infrastructure they already use for their applications. So even though introducing DLM appears to require a change in processes, those changes are minimal.

Our approach to DLM is to say: “If you’re happy with the way you’re working, we have the tools to make it work better. If you’re not, we can help you develop a system you’ll want to use.”

We don’t demand you change the way you do database development. We aim to provide tools that allow you to accommodate to the prevailing methodology and special requirements you have. Ultimately, we want to provide tools that give you the broad view of the process, the friction points, the issues, the places where technical debt is mounting, and so on.

Perhaps most tellingly, we learned from our first experiments that Developers and Ops people like the idea of making the process more visible, and they want to encourage a team approach by providing better, more intuitive tools for developing, checking, and configuration. So we’re not saying they should adopt DevOps for their databases – we’re actually following their lead and giving them the tools to enable it.

Finally, in addition to the tools that help you bring Agile practices to database development, the SQL Toolbelt also contains tools that let you monitor the performance of your SQL Servers, keep your data backed up, and alert you if drift from the expected database state occurs in one of your environments.

Do the database and application have to be close-coupled?

SG2Not at all. We like to think we can bring something to help with any type of database deployment. The reason we talk about integrating database development with application development is that it’s one of the toughest use-cases.

The close-coupled application is also a handy mental model when we’re explaining integration to people. We like to emphasize the fact your application isn’t just application code. There’s database code, data, and possibly other configuration code as well. All those different things need to be managed and changed properly, particularly in terms of configuration.

But that’s not the only scenario. Probably the most difficult deployment is when you’ve got a single database which has ten or more applications calling on it, because now you’ve got to make changes to your database code, knowing you might break a different application altogether as you make those changes.

DLM has to provide a way of making this type of deployment easier as well. Many companies including Redgate have lots of applications that use multiple databases, and those databases are used by multiple applications. If you think about it, that poses a challenge in itself.

What about point tools?

DLM puts a common thread through all of our tools and gives our customers a choice. They can select a single tool to address a particular pain point, or they can choose the SQL Toolbelt and have access to the full suite of tools for Database Lifecycle Management.

We’ve sought to improve our products and invest in all of them to make sure they’re current and relevant, whatever out customers need them for. While the automation story has seen a lot of interest, SQL Prompt, for example, has probably seen a greater degree of innovation over the last two years than it’s ever done before. The same is true for SQL Monitor where we’ve added support for SQL Server 2016 and monitoring for availability groups.

DLM is a very broad category and our products have a role to play in every part of the database development process, from introducing source control to including the database in DevOps practices.