Stephanie Herr

1 October 2018

1 Comment

CALMS and how it relates to Database DevOps

CALMS is the acronym for a framework which allows businesses to assess how prepared they are on their journey to DevOps, and where they can improve. CAMS (without the L) was first introduced by Damon Edwards and John Willis after the very first US-based DevOps Days in 2010. Jez Humble later added the L, standing for Lean, and the full meaning of the acronym is now:

Interestingly, advances in database development software now mean the same framework can be used to evaluate if a business is also ready for database DevOps.

Culture

When talking about DevOps, culture is at the center of the transformation it can inspire. A DevOps culture changes the way businesses work, enabling teams to produce great results that make users happy.

A big part of the culture is breaking down the silos that exist within an organization, so that Dev and Ops can work better together towards the same goal – those happy users. There’s no longer an It worked on my machine or It’s not my problem atmosphere. Now, it’s Let’s work together to make this the best experience for our users. It’s about empowering teams to do what’s best for users, and make sure they can release as needed.

This is really important when it comes to the database. Traditionally, DBAs have been very siloed in their own departments, and have often been seen as a bottleneck to releasing changes faster. They took complete responsibility for the performance of the production database and the security of its data. Now the whole team – developers and DBAs – should be thinking about how to make the release as smooth as possible, how it will run in production, and where to build security in.

Automation

Automation is key when it comes to DevOps. If you want to release more frequently, the release pipeline is a perfect candidate for automation. It’s completely repetitive and automation will help eliminate any manual errors.

Another benefit of automation is the traceability it provides. You’ll be able to see exactly what changes were applied to which environments and when. You can even see who made the changes and why; maybe they relate to a user story or perhaps they were needed for a bug fix. If you have any manual approval steps in your pipeline, you’ll also be able to see who reviewed the changes and signed off the release.

Continuous Integration with a thorough set of automated tests is important for automating releases. The output of CI is a package with all the files needed for the release, which will be used to deploy to other environments, so you only build once and deployments are consistent. Automated tests (unit, smoke, integration, performance, GUI) are also important to provide confidence in releases and catch any issues before they reach production and affect users.

Automation does NOT mean continuous deployment. In continuous deployment, every time a change is committed, it flows continuously through the release pipeline all the way to being deployed. In order to do this, you need to have a lot of trust in your automated tests.

Instead, I’m talking about continuous delivery, where automation is used to progress changes to the point where they can be released as needed, after being reviewed. This is increasingly common in application development and the process looks something like this:

A typical deployment pipeline with database development as a siloed process

A lot of DBAs get scared when they hear about automating the releases of database changes to production. But again, I’m talking about automation and continuous delivery, not continuous deployment. It’s also incredibly easy to include a review step as part of the automated pipeline so that DBAs can see exactly what’s going to happen before anything changes in production.

Include the database in continuous delivery and, rather than complicating the process, it actually looks a lot simpler. It can also be far easier to introduce if the tools used for database development integrate with and plug into the tools already used for application development:

A deployment pipeline with database development as part of the continuous delivery process

Another great thing about automation and databases is the ability to do a test deployment on a staging/pre-production environment that is as close to production as possible, automatically. This provides the best chance for success by testing the deployment script one last time before running it on production.

Lean

Lean was added after the initial talk and is an important value of DevOps. Lean is focused on incremental improvements and splitting the work into small batches. The small batches allow you to release frequently as you develop. This is important so that you can get real user feedback on your work and learn from it so that you can pivot and adjust according to what you learn as needed.

It’s hard to apply Lean to just the database. You should really care about the whole system. The database is part of that system. This further emphasizes the importance of including the database as part of the culture (breaking down silos) and the automation so you can have these frequent incremental releases to improve on.

Measurement

Measuring is important to get quick feedback and continuously improve, because you can only know if you’re making a difference and improving if you have a baseline to measure against. The first area to start measuring is your internal processes, like how long it takes for committed code to be running on production, the frequency of releases, the failure rate, and the mean time to recover from a failure.

You can also include telemetry – how your users are using your system, whether they discovered a new feature, and how it can be improved.

The second area to measure is the health of your system, and to understand that, monitoring is required. That way, you can react quickly if there’s a problem, or even better, proactively fix something that might become a problem, so you can keep your users happy.

For databases, measuring your internal processes and telemetry is done as part of the system since your database should be included in your release process. Monitoring databases allow you to ensure they are performing correctly, and it can be configured to provide alerts about performance issues or low disk space before they become problems.

Sharing

Sharing brings us back to the first value, Culture. If you want to make a huge impact on the culture, it’s important to share the lessons you’ve learned and your best practices with the rest of your company. This helps to break down those silos so that people work together to continuously improve and do what’s best for your users.

It’s also important for DBAs to share the findings with other DBAs and teams across the organization so they too can take advantage of all the benefits of including the database in their DevOps processes. That way, barriers will come down and DBAs will see the value of working on complex changes with development teams early in the development process, instead of finding problems when the changes reach production.

Conclusion

CALMS are important DevOps values, which enable you to break down silos, work better together, and provide frequent value to users that can be continuously improved. These values also relate to databases and it’s important to break down this silo as well so you can really fly.