In pursuit of frequent releases & short development cycles

First invented in the 1990s, XP has had a checkered history. Originally lauded, then derided, later expanded, often criticized, XP has adherents and critics alike.

XProgramming.com describes it as: “…a discipline of software development based on values of simplicity, communication, feedback, courage, and respect. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation.”

Those who like it admire the way it results in frequent releases, short development cycles, and a fast way to respond to customer feedback. Detractors point to its lack of documentation, the potential of scope creep, and the absence of an overall design specification.

XP is not Agile, but a discrete method within the Agile classification. It predates it but has given us the important Agile practices of short development iterations, broken down into ‘stories’ chosen in collaboration with project stakeholders. Teams are multi-disciplinary, and work together with the principles of co-ownership and shared responsibilities. They insist on clean code and simple design, and aim for frequent releases using continuous integration.

When Aaron Law and David Priddle took on SQL Prompt just over a year ago, they weren’t interested in the history of XP, Agile, or their pros and cons. Instead, they were looking for a way to fix bugs quickly, and swiftly address the problems that customers were raising. Which just happens to be the kind of approach that cries out for XP.

Especially when they also realized that the very people who were mentioning problems with SQL Prompt were the people they needed to be talking to. They were the power users, the ones who cared about the product and the way that it helped their productivity in developing SQL Server T-SQL code.

 Aaron and David like great code. They hate bad code, bloated code, unnecessary code, code that you look at and wonder how it ever escaped from the shameless idea white board.

They set to work.

Out with the old

Their first suggestion was that they become a self-organized team. As Aaron comments: “Rather than constant day-to-day management, whiteboards, and checklists, we wanted to organize our work schedule ourselves and keep everything simple: The kind of approach recommended in the Agile Manifesto. That would let us do what we wanted to do, which was get on with the job.”

Their next idea was to be active – and be seen to be active – on the support forum: Very active. When a bug was reported, they were determined to recreate the bug, fix it, and send a private build of the bug fix to the person who reported it to confirm it worked. Not next week or the week after: Now.

Their third idea was to involve the users, let them see the team were on their side, by engaging with UserVoice. An online feature suggestion list, UserVoice allows users to enter new features they would like developed – and vote on features other users have suggested. Aaron and David wanted to work fast, work smart, and release new features regularly. They realized that UserVoice would make sure those features really were valuable ones that met the real needs of those who were using the product on a daily basis.

And then there was the code. They peeked under the hood of SQL Prompt and saw a lot of legacy code that didn’t need to be there. This is inevitable in any fast-developing product but you know their views on code. They wanted to make it simple and clean.

In with the new

Ideas are great. Sitting down and making them work is a different story. When they first started work on SQL Prompt, Aaron and David were dealing with something new. Bug fixes that might take an hour today took a week. Unit testing covered only a small portion of the code. And then there was that code, that damn redundant code …

But it slowly got better. In XP, the software development process follows four paths: coding, testing, listening, and designing. Aaron and David were determined to make each of those paths as efficient as possible.

Programming side by side, sometimes pair programming on particularly difficult issues, they followed the XP mantra: work in short cycles, aim for frequent releases and introduce new features that users want.

To increase the efficiency of their coding, and speed of releasing, they wanted unit testing to cover as much of the code as possible. But the testing was taking time, often several hours, so they worked and worked and worked to increase the unit test coverage.

And they talked to users, encouraged feedback from them, got them involved in the process. “If a bug was reported on the forum,” David says, “we fixed it. When new features were requested, we asked users to vote on which feature they wanted next. In a way, they were in control of what we worked on.”

And let’s not forget that redundant code. Slowly, they delved their way through its complexities, cutting out the bad parts, editing other parts, trimming it down to the XP ideal of simplicity and clarity.

The results speak for themselves

When SQL Prompt 6.4 was released in September 2014, it followed 33 beta releases in four months – releases that resolved issues, fixed problems, completed no less than 41 UserVoice requests, made SQL Prompt more enjoyable to use. For users, the official release of 6.4 was merely the icing on the cake.

Behind the scenes, the results are even more impressive and have shown the effectiveness of co-location and pair programming. Despite adding many new features, Aaron and David have edited down the codebase by 30%. This investment of time and effort has reduced the technical debt and brought significant advantages. By refactoring and improving the design structure, they have reduced unnecessary dependencies in the system, making future changes far easier.

Their test driven development (TDD) approach has also improved the tests considerably. While TDD requires a steep learning curve and writing more code up front, the payback when it works is significant. Alongside the integration tests that covered most of the code, they have increased unit test coverage from 3% to 30%, and drastically decreased the time from committing changes to having a fully tested installer.

Perhaps most tellingly, they have demonstrated to users that they will engage with them, listen to them, and help them.

All of which goes to show that while XP can sometimes attract extreme points of view, it can also – followed correctly – produce extremely attractive advantages for programmers and users alike.