Book Review: The Phoenix Project

After hearing from Grant Fritchey that "Anyone who wants to know what DevOps really means should read this" Claire Brooking picked up a copy of The Phoenix Project. This parable of an IT project on the brink of destruction is told with humor and insight. Claire reviews the book, finding that conflict, incidents and mistakes are inevitable - what counts is how the team members grow to manage and resolve them.

DevOps, Continuous Delivery & Database Lifecycle Management
Culture and Organization

As often found in novels, there are a number of characters to keep track of. To help follow the review, we have bolded the character’s name when first mentioned.

A Brief Synopsis

2067-phoenixproj.jpg The synopsis is simple. Bill, the protagonist, is the Director of Midrange Operations at Parts Unlimited, a US-based $4 billion per year manufacturing and retail company. Bill is swiftly pulled into the spotlight by the CEO and persuaded hoodwinked into taking up the post as VP of IT Operations. It soon becomes clear that among standard responsibilities, Bill and his team are responsible for making the launch of the risky doomed Project Phoenix a success. Project Phoenix not only seems to be hugely overscoped for its ambitious – and imminent – timelines, but it also faces enormous pressure elsewhere.

This is mainly because the future of the company appears to hang almost entirely on Project Phoenix. The political storm brewing behind the project, led by Sarah the marketing manager, adds further tension. Faced with the conflict of Project Phoenix requirements against firefighting and competing projects, Bill and his team enter a spiral of problems, mistakes, gloom and despair. Thanks to a growing insight into Lean, Agile and DevOps concepts, Bill and his team gradually evolve their way of working. By the time Project Unicorn arrives later in the story, they’re able to release and support the project reliably, efficiently, and emerge with strengthening morale out the other side.

A Confession

I’m not an IT practitioner. I’m from the “dark side”, or at least the same one as the irrepressible plotter and marketing manager Sarah. So why did I want to read a novel about IT and DevOps? For one thing, I’ve worked with various technology companies and I was curious to find out more about concepts I’ve seen developing. I’m also part of a research group in the database lifecycle management team at Red Gate and I wanted to better understand how Agile and DevOps work out in a deeper context. If you’ve read about Sarah’s Machiavellian tendencies, you’ll know that the book wouldn’t make comfortable reading for anyone in marketing.

I hope my admission of sharing the same profession won’t put you off reading further, if only because I feel that the book also encourages us to look beyond preconceptions of teams and roles in an effort to persuade us to work much more openly across an organization. This level of communicating, cooperating and, sometimes, making compromises together to solve IT and business challenges completely bypasses Sarah, but other characters in the novel get it.

IT Becomes an Organization-Wide Capability

In the book’s message of business and IT teams working better together on IT projects, the authors also draw out practical examples of ways of working that, in my opinion, are useful parallels for professions beyond IT. For one thing, I’ve been thinking differently about WIP (work in progress) on my Kanban board of activities ever since.

At both the detailed level of insight into leaner, more efficient ways of working and at the wider level of an organization evolving its approach towards IT, I believe everyone with an IT requirement has something to gain from reading The Phoenix Project. It’s true that there’s a fair amount of technical detail on the challenges that the IT team face, which may be daunting for some business team readers. It’s also true that the novel describes a big adjustment in behaviour and ways of working between the development and operations teams specifically, and it’s this that lays many of the foundations for better collaboration.

But it’s when both the IT and business arms of Parts Unlimited reconsider their biases, share responsibility and better understand each other’s challenges (yes, that also means the business teams working harder to understand the technical risks facing the organization) that the cogs become better oiled. That’s not in any way to suggest that sales, marketing, finance, and other business teams are somehow the secret sauce in the success of IT. They’re not. The point is that in the book we see IT growing from an isolated function to an integral capability across the organization, and everyone has a shared part to play in this.

Overturning Preconceptions

I’ve spoken with others about The Phoenix Project and the way it puts the DevOps philosophy into sharp definition. Yes, it’s a novel about this and other IT themes – and for some this seems to raise an eyebrow at first – but those I’ve spoken with agree that its status as a novel doesn’t translate into “DevOps lite”. W. Edwards Deming’s manufacturing philosophies, in addition to Lean and Agile development methodologies, feature heavily; concepts that sit at the roots of the DevOps philosophy.

In essence, the authors of the novel – Kevin Behr, Gene Kim, and George Spafford – have written a DevOps parable, and strive to make it recognisable at every turn of the page. Indeed, having all worked in organizations of varying sizes, maturity and sectors, they’ve also taken pains to place characters in an easily identifiable setting, and embrace the human elements of the story. So, quite apart from the technical challenges (such as the hauntingly familiar server outages and lagging database performance), you’ll find certain home truths: donuts sweeten meetings and pizza sustains chaotic all-nighters. The evolution Bill and his team undergo is pretty amazing – but then this is also DevOps in fast motion. Like many novels, it takes the reader on an unrelenting journey with the underdog, Bill, as he is thrown into battle, meets a (sort of) wizard also known as Erik, and finally triumphs with the organization at his side.

The setting, the IT challenges and the characters at times feel caricatured, but to dwell on the realism of this journey and its existence as a novel for too long feels like it risks missing two of the other key points of the book – that this is a parable with an enterprising spirit at its heart. Via the medium of fiction based on reality, it unconventionally presents DevOps concepts, and it feels like it also wants to dare the reader to be open-minded. Being a fictionalised depiction, it’s easy for the book to ask you to consider turning accepted practices and attitudes on their head in the search for progress.

Evolution Rather than Transformation

In the movement of IT from an isolated function to a central capability – both metaphorically and physically, almost all teams are challenged to look beyond their department walls.

This isn’t just about the dev and ops teams turning around an unhealthy relationship that’s damaged by (a) what Jez Humble would perhaps describe as ‘risk management theatre‘ and, (b) throwing, as one character describes it, “the pig over the wall” for the next team to tackle. This is also about an evolution of attitudes and approaches across the business.

In the book, Bill tries a range of processes to bring his dev and ops teams closer together, including the use of Kanban boards to prioritize and manage work. As these efforts bring results, he and his teams’ focus starts to shift to the rest of the organization. In fact, one of the book’s surprises is the transformation of the CISO, John, outcast and ‘political commissar’, into a team member who is instrumental in bringing the business teams and organization objectives closer to IT. As Deming would describe it, John is essential in helping the various teams gain ‘an appreciation for the system’ that IT works within.

DevOps Across the Organization

For me, John’s series of actions at his point of transformation also raises one of the most important practical examples in the book of DevOps reaching throughout the organization.

Bill and John meet with managers in the business teams, from sales to finance, to understand performance measures across the organization, how they rely on IT, and the business risks IT brings for each measure. At first this approach feels worryingly like we’re headed down a one-way street. Several business team members, for example, are antagonistic towards the open approach Bill and his IT co-workers encourage.

Yet by the end of the book Maggie, part of the business team in charge of promotions and pricing roadmaps, is attending stand-ups for the IT project linked to her objectives. She’s even demoing results to the team at the sprint retrospective in the kind of cross-functional approach to projects that really sings when everyone has a shared goal. There’s a sense of energy in the team again, similar to the kind of morale Steve Thair talks about when comparing one of the tenets of DevOps with the satisfaction of craftsmanship.

Early Feedback Loop

This approach of shared responsibility and ensuring projects are clearly linked with the organization’s objectives also highlights another interesting DevOps message in the book – that an early feedback loop in the lifecycle, involving all stakeholders and project members, feeds efficiency and quality.

As Bill and his team adopt a more Agile approach to prioritizing their backlog of projects and gathering requirements, they avoid the shifting goalposts of requirements and timelines that they suffer earlier in the story when releasing Project Phoenix.

Automate (What You Can) and Review

By the time Project Unicorn arrives, Bill and his team are ready to run shorter, iterative development lifecycles that mean they can identify issues earlier, get results out to the business, and move WIP off the line sooner. They also set in place a series of automated practices to standardize deployment processes and build up a release pipeline that eliminates the majority of variance that caused the teams grief previously. Cycle times shorten and any fixes can be developed, tested, reviewed and released much quicker than they were with Project Phoenix.

Quite beyond being a series of examples for how to run an IT project with Agile and DevOps practices, this is also a lesson generally in PDCA (Plan, Do, Check Act). Essentially, larger projects run better when:

  • they’re broken down into manageable chunks
  • we make the effort to check back with others across different teams,
  • we check results,
  • we make adjustments to the project as needed,
  • we learn how to standardize (even automate processes for repeatability and greater accuracy if possible), improve for next time, and communicate that experience.

Avoiding Isolation

This willingness to open up a project much earlier for feedback touches on another example of DevOps in the book that also carries parallels for the wider organization. In the closing chapters of the book, John’s security team works closely with the development team, integrating security tests into the build procedures, testing and reviewing them earlier in the cycle. As a result, we see a radical shift in approach from earlier chapters, when John and his team were viewed as a disruption to be avoided.

As John and his team make efforts to integrate with other teams, they even start to use the same testing process that the QA team uses. For me, this example of the teams’ turnaround in approach and results comes very close to the point David Poole makes in his article on developer and DBA relations when he discusses the idea that ‘functional silos create more problems than they solve.’

As the security and QA teams open themselves up jointly to the potential of exposing errors, so they have a greater ability to resolve any issues found head on. John no longer needs to parachute in at the last minute with his dreaded black ringbinder of problems to register project shortcomings. Fundamentally, John’s team are no longer ‘special’ or exempt from agreed practices, nor are they isolated and overspecialized in their own departmental universe.

Concluding Thoughts

At its most raw, this book can easily feel like a lesson in humility. It’s not the case that at the end of the book, DevOps and Agile approaches to working have magically evaporated all the challenges facing Parts Unlimited. Conflict, incidents and mistakes are inevitable – what counts is how Bill and other team members grow to manage and resolve them. By the end of the book, Bill, his team, and Parts Unlimited have structure, process, and a more open attitude to change and adaptation to stand them in good stead.

At its most practical, The Phoenix Project is an illustrative series of examples and suggestions for ways to evolve IT from a function that’s viewed as a bottleneck to one that’s widely agreed to be an indispensable capability. And at both levels, The Phoenix Project makes it clear – DevOps includes the wider organization, and the wider organization can learn a lot from DevOps.

DevOps, Continuous Delivery & Database Lifecycle Management
Go to the Simple Talk library to find more articles, or visit www.red-gate.com/solutions for more information on the benefits of extending DevOps practices to SQL Server databases.