DevOps, Continuous Delivery & Database Lifecycle Management
Culture and Organization
DevOps has become quite the rage in the information technology world. With the increasing desire for companies to build and deploy code faster, the DevOps philosophy gets more focus as a path to accomplishing these goals. However, applying DevOps is a significant shift in the way technology operations has been approached over the years.
DevOps, as a concept, is easier for the Development side of any shop to grasp. Concepts such as “releasing software frequently” and “treating assets as disposable” do not mesh with the way Operations is used to managing their environments. This is especially a challenge for database administrators, who are used to rigorously managing the hardware behind their instances, stewarding previous data, and cautiously examining code to prevent that late night unplanned outage. Operations staff take the view that speed invites chaos, resulting in instability, downtime, and a lack of sleep.
The reality is that chaos, instability, and downtime are not the result of speed, but the result of variance. Often, when things are done quickly, code is delivered unevenly and servers are built inconsistently. DevOps drives us to improve and fine tune our process. The goal is to enforce consistency, and to manage our resources so that software is deployed in the same way that a car factory builds an automobile. Components should be standardized, builds made consistent, and tasks automated. The result is speed, yes – but speed that is the result of control and standardization.
This article is intended to show you what DevOps is from an operations point of view. Beyond that, you should also see what it means to database administrators, both those who choose it and those who have it thrust upon them. If your company hasn’t already spoken to you about DevOps, they will soon and this article should help prepare you for that conversation.
Words have meaning
The first challenge with DevOps, like many other corporate catchphrases, is understanding what it means. We’ve all seen what happens when pointy-haired boss talks about Web 2.0 and databases in mauve. DevOps actually exacerbates the issue, because it is a philosophy more than a defined set of steps you can apply to your teams. Companies need to embrace DevOps, not deploy or implement it. In fact, if you have a DevOps manager or team, you are likely going about it incorrectly.
To define DevOps, we need to understand a few things. Initially, it must be understood that DevOps is not a thing that you will have or your company will possess. It is, instead, a mindset with the goal of stabilizing technology delivery via effective collaboration. Companies should not have a group or person responsible for this, but instead bring everyone into the process. We all need to participate in the DevOps culture and share the load and responsibility with our teammates.
The first step is to remember the obvious: that DevOps is an amalgamation of Development and Operations. By extending that thought, we can then describe three distinct facets of that amalgamation:
- Development for Operations
- Development to Operations
- Development and Operations
These three facets allows us to bring clarity to what DevOps means and how we can implement it.
Development FOR Operations
When we talk about development for operations, it means operations should act like developers when dealing with their assets, builds, and tools. While operations staff have always had the idea of run books and build processes, it’s hard to maintain consistency when these items are not codified. To benefit from DevOps, we have to start treating our work as code. This means builds should be scripted as much as possible, and these scripts should be managed in a source control system. Operations staff should focus more on building and using tools for their work. Tools are designed with a specific purpose in mind and get used in a repeatable manner. This means that when you create your own tools, it forces consistency in your practices and processes – a key principle of DevOps.
Development TO Operations
The second facet is that of development to operations, which closely aligns with the first facet. If we consider the car manufacturing plant analogy, development is responsible for designing parts and specifying how they want the car to be built. Operations is responsible for how the manufacturing floor works, as well as what stations and machines operate within the manufacturing process. When it comes to software, we know code is always going to inexorably flow to us and that no system remains static. Operations must take control of the tools and process used to deploy this code, building a streamlined and reliable system that developers can use to push their changes in a controlled and rapid fashion. Again, the stability of the system is maintained through the consistency of the system, a result of tool use and established process.
Development AND Operations
The final facet is the one most DevOps folks talk about: development and operations. While tool building and process management are great, it means nothing if both ends of the process are not working together. This is a key nuance: there will still be operations and development. The term “full stack developer” gets bandied about quite a bit, but it is a myth. Technology stacks are far too deep and complex for any individual to understand everything, and so T-shaped people are the best you can realistically expect to find. Personally, I know a little bit about coding, some about hardware, and enough about network infrastructure to be dangerous. However, I specialize in databases and bring to the table a deep understanding of that aspect of the stack. Just as I rely on my team mates to be experts on the things I do not know, I contribute my database expertise to fill out the team’s total knowledge base.
Another key point is to remind ourselves that development and operations are a team. Specifically, these two groups are ultimately the same team. The largest hurdle to making DevOps work is to dissolve the antagonism that has grown between development and operations. Operations must reach out and get involved in the development process. If not, they abdicate total control to the development teams to define and build, leading to the kind of team division we see today. This division is what causes outages and issues, not bad code or buggy deployments. If operations and development are working together towards common goals, stable code and stable environments, it is easier to achieve.
How Do DBAs Fit In?
These three facets of DevOps can be applied to databases and database administration, though it does require database professionals to think differently about their tasks and their roles.
Database object deployment and development is the easiest area to apply these principles. When talking about developing for operations and managing responsibilities between teams, database professionals should be concerned with how database code is deployed. Usually this will mean the administrators will be involved in selecting tools for source controlling databases and building processes to deploy this code to the production environment. They should also work with the development teams to make sure code is written well and along agreed upon standards. However, this should not be approached in the traditional code review process, as this is much like the developer kissing the DBA’s ring for permission to deploy. Instead, DevOps DBAs should act as mentors and guides to build stable and efficient database code during the development phase.
Another element that DevOps DBAs should begin to consider is how they construct their environments. More and more commonly, servers and their binaries are considered disposable and stateless, and so the only stateful element of a database system is the data itself. Since most relational database systems offer data replication for high availability (such as Availability Groups for SQL Server and DataGuard for Oracle), DevOps DBAs have options to distribute their data in such away that they can replace server assets without affecting the data. This offers several strong advantages for maintaining asset consistency and resolving server issues without affecting the uptime or response of the data systems. Naturally, there is a lot of detail we could go into around this, but I’m afraid I’ll have to save that for a future article.
This brings us to the biggest challenge of databases in a DevOps world: the data itself. Most DevOps processes consider assets, such as code and configs, as stateless and easily replaceable. Deploying code rapidly, as well as doing rollbacks, can be easily performed by simply overwriting these stateless assets. However, by its very nature, data should be considered stateful, and must be maintained. This means data professionals must marry DevOps principals with data stewardship and act, not only as data guardians, but shepherds that guide the process.
Molding Our Environment
How do we start to then implement some of these principles? As technology professionals, we need some steps to focus on in order to achieve our goals.
Mend Fences
The very first addresses the third facet – that of development and operations acting as a team. There’s a lot of acrimony between these two groups that has developed over time, and if you want to really succeed with DevOps, you need to start breaking down that division and treating your counterparts better (and yes, this absolutely cuts both ways, but someone has to make the first overture.).
A phrase I often hear is “developers are idiots”, but we all know that is simply not true. Developers are usually very good at writing code and building software, but are typically ignorant of operational needs. Vice versa, operations are often unaware of the hurdles and challenges that developers have. Education is the foundation of DevOps. Talk to your counterparts and help them understand how you can help them achieve their goals. Gone are the days where we can work in our silos, and good collaboration is increasingly the key to successful software.
Get the Right Tools
The second phase is to focus on tooling and automation. Look at your tasks and ask yourself how you can remove those tasks from your workload, and automate them into oblivion. One of the biggest reasons for instability and loss of time in any operational environment is that processes are manual, and therefore subject to human error. A key mantra about automation is that you do not automate to speed up your work, but instead automate to make your work repeatable and consistent. The benefit, of course, is that you remove work from your plate and allow it to be accomplished in a faster time frame. The end result is that you elevate yourself as a resource, becoming more productive and increasing the reliability of your work.
It’s Code All the Way Down
Third, but not necessarily least, is to treat your work as code. Any work you do should be source controlled and released as if it were a code component. This makes obvious sense when dealing with the actual application or database code we write, or the tools we build, but it should also extend to our servers and other resources. Not only does this bring stability, but allows you to rapidly repair or rebuild work if something is wrong, and have the confidence that you can do so.
The challenge here for database administrators is the data. As we discussed before, data’s stateful nature means that we can not simply source control it and replace it when it goes bad, or a deployment fails in a particularly nasty way. However, we can manage everything around the data, and by thinking creatively about this we can increase the reliability of our platforms and environments. By maintaining the separation of data and infrastructure, the integrity of the platform can be controlled while ensuring our database consistency.
The Road Ahead
DevOps is not really something new, it is just newly codified. We have always heard the constant drum beat of doing more with less, and organisations with foresight have seen the value in cross-functional teams. What DevOps gives us is some guidance on how to make that drum beat marchable, instead of just a frenetic pace of 60+ hour work weeks! I encourage you to think about this future and not just about the extra hassle it will mean for you in the short term, but rather how it can make your life as a technology professional better.
If you have not yet done so, read The Phoenix Project. While the story is interesting, and it glosses over some fairly big challenges, the principles it teaches are invaluable. In the coming months, I will also be providing some more technical articles on how I am implementing these principles and helping to foster a DevOps culture in my own organization.
DevOps, Continuous Delivery & Database Lifecycle Management
Go to the Simple Talk library to find more articles, or visit www.red-gate.com/solutions for more information on the benefits of extending DevOps practices to SQL Server databases.
Load comments