I\u2019ve been frequently asked by people interested in DevOps: \u201cHow do I convince management that this whole automation thing is worth our time?\u201d<\/p>\n
It\u2019s a great question. What I always say is, start by documenting your pain. However, just saying that statement may not be clear enough for everyone, so let me explain further.<\/p>\n
One of the simplest things to start documenting is whether or not you\u2019re experiencing downtime because of your existing deployment processes. This is clearly one of the most worrisome pain points that the business could have to deal with. I\u2019ve worked with organizations in the past where any kind of downtime resulted in the loss of serious money. I\u2019ve got friends who worked within healthcare where downtime could literally result in deaths. Downtime is a very serious problem.<\/p>\n
Downtime from deployments comes in several flavors. One possible situation is that the deployment itself causes the downtime. When you\u2019re running scripts for the very first time on your production system, the chances of creating downtime are very high. Even if you\u2019re practicing the scripts before you deploy in your non-production environment, you may still see errors in production. This is especially true if you haven\u2019t used some kind of \u201cshift-left\u201d mechanism to ensure that your non-production environment is as near to production as you can make.<\/p>\n
The second kind of downtime is caused due to the manual nature of your deployments. We used to keep a checklist on a whiteboard. We\u2019d carefully walk through the checklist, ensuring that everything was done. However, every so often, we missed a step. One time, the boss came by to check in with us, leaned against the board, and wiped out the checklist, causing more delays and rework on the deployment.<\/p>\n
Another kind of downtime that you have to take into account is the amount of time it takes to deploy. Before I started automating deployments, we used to do it all manually. Step one was to bleed off all connections, run a backup, then deploy. Every step took a long time. As noted above, errors on any of the steps would certainly impact downtime; however, the process itself caused our servers to be offline.<\/p>\n
All this together negatively impacts the business, so this is the first pain point I would document.<\/p>\n
It\u2019s entirely possible that you can deploy to your databases and never have an error, never experience data loss of any kind. If so, you\u2019re very lucky. However, in a manual environment with minimal, or no, automated testing, it\u2019s very easy for an error to slip in. Without that shift-left process, you may be running a script for the very first time on your production server. Also, if you\u2019re not deploying in a Continuous Deployment methodology using lots of fast, small deployments, that script is likely to be 15,000 lines or more. I\u2019m sure you looked through it, but you didn\u2019t really read every single line of code.<\/p>\n
Without automated, frequent, early, testing, the chances of causing data loss are radically heightened. This quickly becomes a case of testing your ability to restore your database to a point in time. Regardless of how well you do this, you\u2019re introducing more downtime for the business. Even worse, you may have to go and explain to the company just how much data was lost, irretrievably. Do you have automated testing for your backups? If not, they could be corrupted or missing, and you won\u2019t even get the chance to retrieve the data you lost.<\/p>\n
I\u2019ve heard the argument made that, well, you\u2019re paid anyway, so whatever time you spend building deployments manually is all part of the job and effectively free. However, that\u2019s just not true. There are so many hours in a day, and nothing any of us does can change that. How you spend your day matters. You need to document how you spend your day, especially as it relates to deployments.<\/p>\n
Like uptime, there are different pain points where the time you spend becomes an issue. The first, and easiest to define, is the time it takes you to build out the deployment script. If you\u2019re not working out of source control and using branches or tags to identify which pieces of functionality are ready to move to production, then a considerable part of your day could be spent in figuring which parts of the database are OK to deploy and which parts are still under development. Not only can these manual processes chew up your day, but they\u2019re also another vector for introducing error into your deployments. Moving to a mechanism whereby deployment scripts are created through automated processes can help to eliminate this pain.<\/p>\n
Another big chunk of your time is going to be spent on environment refreshes as well as manual deployments. The QA team would like a new copy of production. If you have yet to automate the refresh process, you must complete many manual steps to for this request: get a backup from production, restore it to QA, and deploy the code after building it manually. You must also run some scripts to clean the data before handing it over to QA. (Please tell me you do this, even if you\u2019re not subject to a compliance regime like the GDPR or the CCPA; you shouldn\u2019t have production email addresses in non-production environments and that\u2019s just one example.) Multiply this effort across multiple development streams, multiple teams and multiple environments, and suddenly you\u2019re spending a lot of your time dealing with deployments.<\/p>\n
The most important thing to remember is all the time you document that you spend on deployments is time you could be working on automating those same deployments, tuning queries, or, most importantly, helping build out new functionality for the organization.<\/p>\n
Silly terms like \u201cat the speed of business\u201d can be bandied about, but the fact of the matter is, in a rapidly changing world, your organization needs to change. This means code has to change, and your database has to change. One more piece of pain you have to document is just how long it takes to get a change out the door successfully. Please note, the keyword in that sentence is \u201csuccessfully\u201d. We can rapidly deploy the wrong thing or something that doesn\u2019t work. It takes a little more time to ensure that things go successfully.<\/p>\n
You know when new functionality requests come in the door. Start the clock to understand just how long it takes to deliver them to production. Document how much of that time is spent deploying the database, resetting the database, building the deployment process, or dealing with the problems caused by deployments. This collective time is a cost added to everything your organization does, making the entire thing less efficient. If nothing else I\u2019ve suggested above will sway people, this measure alone may.<\/p>\n
If it looks like I\u2019m equating time and pain, I am. There\u2019s a classic saying:<\/p>\n
You can have something quick, cheap or good. Pick two<\/p>\n
To a degree, this is true. However, if you\u2019re willing to spend some time, because time is always the hard part, to automate your processes, adopt a DevOps mentality and methodology, you can arrive at the place where you\u2019re delivering good functionality quickly. You just have to make the investment to arrive at that spot. To begin to deliver this message, show how you\u2019re only satisfying one of the three, not two. You\u2019re not delivering things either quickly nor cheaply because of all the time and effort. Further, you may be missing out on all three if you\u2019re also experiencing a lot of downtime and recovery.<\/p>\n
While there are many benefits to the adoption of DevOps, the ability to reclaim time so that you can spend it where it should be spent, building systems and delivering value, is arguably the most important.<\/p>\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
In this article, Grant Fritchey explains how leaving the database out of DevOps hurts you and your organization and why you should document the pain it causes.…<\/p>\n","protected":false},"author":221792,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[143517],"tags":[95506],"coauthors":[6785],"class_list":["post-86870","post","type-post","status-publish","format-standard","hentry","category-culture","tag-automate"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86870","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/221792"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=86870"}],"version-history":[{"count":2,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86870\/revisions"}],"predecessor-version":[{"id":86872,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/86870\/revisions\/86872"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=86870"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=86870"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=86870"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=86870"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}