What is ‘DevOps’ really?  Although the term is now being treated as if it were a marketing buzzword like “cloud” and “big data”, there is a real grassroots initiative behind the hype that grew out of some good ideas generated by a few smart people working on application delivery. They went on to develop these ideas until they did real, concrete, awesome things. Others saw how successful these new teamwork methods were, and tried to figure out shortcut formulas to get there, without fully understanding the original ideas. This has caused problems and confusion because it is impossible to implement DevOps practices without understanding and putting in practice the underlying ideas. Unfortunately, there is no way to shortcut the DevOps ideals, or to implement them only by using an expensive software package. Although there are tools, procedures and techniques that you can help once you have adopted the DevOps ideas, there is an essential prerequisite; a positive working relationship between teams. This is essential in order to identify the problems and find appropriate solutions.  The focus must be on the teams, the individuals within them, and their interactions rather than on processes and tools. 

At its core, DevOps is not a tool. It’s not a process. It’s not even a methodology. First and foremost, in its most valuable sense, it’s an idea to strive for.

History

To me, DevOps is just the next stage in a succession of grassroots ideas by successful software teams trying to find better ways to be more effective, and extend this to the whole development process. All of it is traceable back to the ideas of the Agile Manifesto. 

Although there have always been new ideas to make team-based software development more effective, the past several decades have seen a recurring pattern in how these initiatives have evolved in their scope, starting with programming, and ending by encompassing the whole delivery cycle:

• For a while, the focus was on getting the complexity of enterprise software under control. This seemed to have, as its objective, building and writing that code as quickly and efficiently as possible.  You saw the introduction of RAD and 4GL tools, development patterns like object orientation, and design approaches such as UML and RUP.  This focused on the rapid communication of ideas between analysts and developers, and rapid development directly from that communication.
• Once developers had figured out ways to make individual and independently-developed components easier to build, they had to ensure that their code fit together nicely with everyone else’s. It was the effort to solve this problem that gave birth to the idea of continuous integration. This helped to build the stability of interaction and coordination between development teams.
• As the development teams got faster and became more flexible, they had to work closer with the project managers that defined and managed what they should be building and when they could expect it to be done. Agile project management processes such as Scrum filled this need.  This helped to improve the communication and effectiveness of the whole project team, from developers to QA to project manager, and even product owners and business users, who were very under-represented in previous approaches. 
• Another challenge of building software so quickly, with so many changes along the way, was that of validating the software.  When you are moving fast enough, and handling changes to the project scope and desired functionality, it’s hard to capture what the actual correct functionality should be, and whether the software you are building meets those requirements.  This brought along several approaches to automating the test process; unit testing, integration testing, and UI testing.  Developers starting using patterns such as Test Driven Development, and started working with, and communicating with, the QA team to ensure that there was a shared vision of the expected quality of the system.  This increased communication between development and QA resulted in less focus on futile distractions like bug counts and arguing about whether something is actually a defect by strict definition, and more focus on everyone working together build the highest quality system they could.
• Having partially conquered many of these problems, many product teams took the Agile ideas a few steps farther to get the business users more involved.  While it was always important to make sure that the product being built was what the users actually needed, it was very easy to lose sight of this when focusing on spreadsheets and feature matrices.  More importantly, they wanted to ensure that they were genuinely solving the user’s problem: This required working with the users, asking questions, getting to the root of the problems, and offering potential solutions, rather than just demanding a requirements document.  To help along this communication and discovery process, ideas such as Behavior Driven Development and Specification By Example were developed to ensure that the business users, the only people who really know what needs to be done, are more involved in the project (serving as Pigs rather than just Chickens, if you will).
• Now, having confronted many of the hard parts of the process of efficiently building software that actually solves the user’s problems, it became more obvious that there was work to be done in improving the process of shipping and supporting that software.  This has involved working with operations teams to streamline the deployment process and monitor the systems throughout various test and integration environments. 
• This “DevOps” approach has already solved a long list of long-standing problems in this business, and so might be expected to thrive. However there is a risk, which becomes obvious if we study what happened to past attempts at improving the software development process. The initiative and development of a movement that is rooted in grassroots experience is somehow morphed into a commercial opportunity. Could it be that a movement that has been driven by Ops people and Developers will become hijacked, and subsequently distorted, by our marketing colleagues for commercial reasons, by hyping it into the next Big Thing, the next Game Changer, the next Silver Bullet? If so, then that is a Bad Thing.

 Learning From History In Improving The Software Delivery Process.

Repeatedly, over the past decades, there have been grass-roots attempts by IT people to improve the software delivery process. Each one has been an every-growing vortex, pulling in more people from diverse teams.  The history of trying to improve the development process ‘from within’ started with just the developers, and then expanded to other development teams, QA, project management, business users, and now operations.  Each step of the process involved building bridges of communication and cooperation across teams. Each of these phases then went sour. They all went through a similar buzzword-ification, when a complex idea became over-simplified by a slick phrase.  Generally speaking, they all went through the same steps:

Some pioneering teams start to develop new processes to help them succeed in working together and with other teams.  Being analytical folks who can react and respond to new information, the most successful processes are developed over time with feedback from the other teams and a critical eye towards continuously improving the process.

• Other folks notice this success and want to capture it, so the processes start to become more formalized and prescribed.  While the original processes were very customized, the industry starts to get a better of idea of what parts work well in more generic settings.
• As the patterns become better understood, the repetitive friction-points are identified and companies begin to build tools to automate away that friction and give people freedom to focus on the core benefits of the process.
• More people, looking for the quickest way to get the most value from the concept and begin to think that the quickest way to accomplish the change is to adopt the tools and formalized processes.
• Eventually, the IT industry attempts to capture the buzzword-magic by bottling it as a set of processes and tools, missing entirely the real teamwork magic.
• While lip-service is paid to the idea of communication, and cross-team meetings are set up, the valuable ways of enhancing communications take a backseat to the process and tools.  This is, of course, the complete reverse of the Agile Manifesto that drove the concepts to the great idea in the first place. This is a great temptation for the IT Industry as a whole because cooperation and communication takes a lot of work over time to build, and that is not something you can easily package up and sell in a box. 
• The magic is dead. Complex ideas are reduced to a marketable recipe, and the Software Development industry tries to follow the recipe and so often fail to be effective with it, the ground-breaking folks who originated the ideas become disillusioned and are off to solve the next problem.

 Next Stop: DevOps

So what is this DevOps thing?  In its simplest sense, from the developers’ perspective, it aims to expand the development team beyond just developers and QA and PMs to include the operations and deployment teams.  The result is what you might call a “product delivery team”.

The first step, like all of the other previous steps, is to “stop being a jerk”.  Get yourself OK with that first, and come back when you’re done.  “But I’m not a jerk, some people are just idiots”.  Yeah, that means you’re still a jerk.  As soon as you find yourself saying or even thinking that someone on your team (or worse one of your users) is an idiot or a moron or useless and their job responsibilities are pointless, you have more deep-seated issues to work out first.  Before you can succeed, you MUST treat everyone on your team and other teams with dignity and respect.  And the hardest part about this is that you can’t just go through the motions, but deep down inside you need to actually believe it.  If you think that this is too touchy-feely, or you don’t think you can do that, or you don’t want to or you don’t believe that it’s necessary, that’s fine: If you’re determined not to learn to be a team-player, then don’t choose a job that demands team-working skills.  The idea of the “smart but difficult-to-work-with-developer” is a crock.  If you can’t work effectively with other developers and people on other teams, I don’t care how many books you’ve read, you will never be a good developer, in my humble opinion.

The next step is to actually talk the other people.  Recognize the fact that, as hard and important as your job may be, theirs is probably just as hard and just as important: The only way that your team is going to be more effective is if everyone works to make everyone’s job easier, so set up a meeting with them, and ask, with a straight face and genuine interest, “what makes your job more difficult, and what can we do to make it easier”: Then watch their face light up.  I guarantee that they have a list of annoyances, manual steps, and remediation procedures that waste their time every day; they will be happy to have an opportunity to gripe about them in a welcoming setting without having to worry about being labelled a “complainer”.  Examples would be “we don’t know when something is deployed or needs to be deployed”, “we don’t know what changes are including in each release”, or “every time I have to deploy something I need to copy files all over the place and edit some scripts and the dev team always forgets to include a certain file”.

Now, you will be tempted to smile and nod and shrug your shoulders and explain that it’s not a perfect system but it has worked so far.  Suppress this urge, write down the concerns, and start talking about them.  Throw around blue sky ideas of possible solutions.  Get an idea of not just want hinders them, but what would actually make them succeed. 

OK, now what is the official title of this part of the DevOps process?  There are probably several names, but I prefer “stop being a jerk, talk to people, and find out you can solve their problems”.  What tool should you use for this?  Usually Notepad, Evernote/OneNote, or a piece of paper, and an open mind.

Now, before you have finished talking to them, pick a few of the most irritating and/or easiest problems and promise to fix them.  It is a good idea to schedule a follow-up meeting a few days or a week or two ahead, before you even leave the room. In this meeting, you’ll need to show the progress you’ve made towards solutions and get feedback about whether it actually is going to solve the problem or what you might be missing. You could also take this opportunity, now you’ve given them something to visualize, to see if they’ve thought of a brand new process improvement that would make their work ten-times better.  It could even be that they now realize that what they first suggested was wrong, so maybe we need to try something else; this is not a bad thing, so try not to get frustrated if this happens.  Instead, repeatedly stress to them that your main goal here is to make their life easier, not just because they want to hear that, but because it’s 100% true.

Sound simple?  It really is.  But that is the core of DevOps.  If you do this first, everything else will start to fall into your lap.  There are a bunch of tools and procedures and techniques that you can leverage to solve the problems, but you need to be sure that you are actually solving the right problems, and to that you need to build a positive working relationship to help root out and identify the solutions to those problems.  The tools are awesome, but you HAVE to focus on the “individuals and interactions over processes and tools”.  But once you have this in place, you can do DevOps.

Again, it’s not about a tool or a process or a methodology. It’s not about cross functional teams or configuration management. It’s about development and operations and everyone else who is responsible for delivering and supporting a software system to come together, recognizing that they have a single shared goal of building and supporting the best software they can as efficiently as possible, and working together as a team to accomplish that.