Kendra Little

22 June 2020

Future Proofing: Database DevOps Now and In Two Years

While individual buzzwords will come in and out of fashion, the ideas at the heart of DevOps aren’t going anywhere. In this post, I give my take on three initiatives you should invest in today to transform your database development and operations practices. These initiatives will pay off over time, no matter what changes in terminology and technology come our way.

Prefer to watch or listen to a talk on this topic? Watch a YouTube or Vimeo video with chapter links for each topic.

What is DevOps?

There are several good definitions of DevOps out there. My favorite definition comes from @IanColdwater, who defined DevOps in terms a teenager would understand:

Devops is a set of ideas about how process, tools, and people can engineer software better

I love this definition because it doesn’t focus on existing assumptions about what “development” and “operations” are or should be — the point of DevOps isn’t to stick with those old definitions, but instead to evolve our own ideas about how we can create software in ever-better ways.

This evolution isn’t simply implementing automation: instead these ideas involve changes in people’s roles, process definitions, and the tools which are used.

Will we still be talking about DevOps in two years?

I can easily remember a time before I heard the words “digital transformation,” and I feel that term is already starting to lose popularity.

There may be similar fatigue around the term “DevOps.” In two years, we may find ourselves using that word less. But even if this is the case, I’m confident that in two years we will still be building on the set of ideas central to DevOps about how to engineer software better.

This set of ideas includes focal points of focusing on the customer, maintaining a steady flow of work, and reducing toil. Work in these areas improves code quality, reduces time to market, and focuses individual contributors on value-added-work– which many find more enjoyable than repeatable tasks. These results are attractive both to leadership and to employees across the organization.

Even if the specific word “DevOps” falls out of fashion, the set of ideas it embraces are well positioned to stay at the heart of technical initiatives and efforts for well beyond the next two years.

Three investments to bet on

One common piece of advice for those looking to improve their software engineering is to look for the team’s most significant constraint. Once this is identified, you should work to reduce that pain or eliminate that constraint, and them improve iteratively.

While this is good advice, when you first begin you may see constraints and pain points everywhere. It can be difficult to see the forest for the trees — and in looking at the forest, you want to identify how to invest time and money in the right actions which will help you be productive now, and which you won’t need to re-do or reinvent after a year or two passes.

For database development, I recommend investing in three areas:

1. Infrastructure as code for databases
2. Designing resilient pipelines which align with other development practices
3. Communities of Practice for Database Engineering

Investment 1: Infrastructure as Code for databases

Database development and deployment practices have largely remained the same in most organizations over the last 20 years. A common model in use is that developers share the use of a development database. In the 2020 State of Database DevOps Survey, we found that more than 70% of respondents reported that all developers work on a single shared database for the most business critical database in their environment.

A traditional database development model where developers use a single share development database and push changes to a set of DBAs concerned primarily with production stability

Developer and security expert Troy Hunt outlined many problems with the shared development database model in his 2011 post, “The unnecessary evil of the shared development database.” The issues he outlines remain fully relevant.

To sum up why shared development databases need to change:

• Shared development databases lead to worse quality code, as a lack of alignment with distributed source control, the “last writer wins” problem, and the “unpredictable data state” problems lead to mistakes.  The use of a shared database also discourages experimentation, as dramatically changing database structures or data will likely have negative impacts throughout the whole team.
• Shared development databases lead to higher time to market. If object locking is used to prevent mistakes, this directly increases time to market and blocks the work of the team. (Note: this also tends to lead to worse code quality, as team members tend to hurry when others are waiting, which leads to mistakes.) Shared development databases also require more time in examining code and testing it before shipping: in this mode it can be difficult to tell why a change was made in the database, who made it, and if it needs to be added to version control.

Infrastructure as code for databases makes it easy for team members to quickly create development environments.

This empowers developers and DBAs to quickly and programmatically spin up databases for independent use when needed, then dispose of the resources when they are done with it. By combining infrastructure as code for database environments with modern distributed source control practices, team members no longer need to block one another, or worry about accidentally changing data or structures that can interfere with anyone else’s work.

Investment in this area has both short term and long term benefits: teams immediately can move more quickly and have less risk of making mistakes. Longer term, the ability to programmatically create and tear down database environments gives your team richer options for automation and testing, leading to even greater reduction of manual work and higher assurance for the quality of your database changes.

Investment 2: Future proofing your pipeline designs

Pipelines are more than simply automation: pipelines bring together your people, processes, and tooling in modern software development.

Thoughtfully designing pipelines for your database development takes some investment. After all, Redgate has found that even in 2020, only 56% of database users report that their database code is in a version control system (VCS). Only 27% report that that automated builds are in place for database code, and only 15% report using automated unit tests against their database code.

For the 73% who have not yet automated builds, it is worth the effort to invest time in researching and implementing pipelines to help your team members use automation to validate their database code, get code reviews early in the software development process, and generally reduce manual work and the risk of errors.

The good news is that there is no need to reinvent the wheel. While many database teams are not familiar with automated workflows, many of these teams work inside organizations where successful pipeline patterns have already been implemented for application code. The best investment of your time is to find the proven workflows within your organization for application code, and to adapt this pattern for your database code.

For most organizations, this means:

• Using Git to store your database code along with the branching strategy of your organization’s choice
• Implementing a Pull Request based workflow with automated build of database code, and ideally automated deployment of database code to a freshly spun-up environment for review and early feedback
• Automating continuous integration, leveraging build and unit tests against your code

Pipelines that manage this workflow also bring together teams: not only developers, but also database administrators can be empowered to create Pull Requests and participate in the early review of code changes for the database.

Investment 3: Communities of Practice for database engineering

The third area to invest in is a Community of Practice for database engineering.

Communities of Practice are a bit like user groups or coding clubs at work. These work best when they are truly communities: when starting a Community of Practice, the goal is that no single member or team dominates the discussion, or even the management of the group in the long term. Like any community, however, it takes a significant investment of time as well as persistence to establish a healthy, self-sufficient group.

Communities of Practice are worth this effort, however, because they benefit your organization over time. These groups:

• Create a network of users who may rely on each other for technical expertise and tips on how to solve challenges
• Provide a way to share experiments, successes, and learnings across groups
• Create a sense of intrinsic motivation for people engaged in engineering changes for the database
• Break down longstanding silos between development and operations

The Accelerate State of DevOps Report in 2019 found that:

High performers favor strategies that create community structures at both low and high levels in the organization, including Communities of Practice and supported Proofs of Concept, likely making them more sustainable and resilient to reorgs and product changes.

Top Takeaways

The tech world changes rapidly. Amid this change, it’s a safe bet to invest in the following areas to improve both speed and quality in engineering changes for your databases:

1. Infrastructure as code for databases
2. Designing resilient pipelines which align with other development practices
3. Communities of Practice for Database Engineering

Video – Future Proofing: Database DevOps Now and in Two Years

• 00:00 What is DevOps?
• 01:28 Will we still be talking about DevOps in two years?
• 02:29 Three investments to bet on holding value
• 03:50 Why we need infrastructure as code for databases
• 06:20 The model to invest in for database development environments
• 08:05 Future proofing and pipeline design
• 10:18 Demo of a Pull Request workflow for the database SDLC
• 12:05 Why to invest in Communities of Practice
• 16:44 Top takeaways

If YouTube is blocked in your organization, you can also watch this video on Vimeo.