Throwing the BDUF out with the bathwater

Recently, it has been interesting to observe a more critical and considered reaction to some of the more extreme ideas in software development, from berating the nonsense of the 10x developer, to a burning desire to consign SCRUM to the flames. Underlying some of these critiques, there is a nostalgia for a return to a slower, more sustainable approach to software development.

What does all this angst come from?

All of the ‘Agile’ methodologies aim to increase quality by applying an engineering-like rigor to the development process. However, despite all the frenzy, and some excellent ideas, there seems to be little hard evidence that many teams can, reproducibly, execute projects that produce higher quality software, on time, and to budget.

In struggling to understand why, Greg Jorgensen observes that, in his experience, projects succeed not due to some magical process or methodology but “when the key people on the team share a common vision”, and everything just clicks into place.

“…What I don’t understand is why I had that feeling a lot more in the bad old days of BDUF and business analysts than I do now.”

Could this be the key? However good the methodology, and however talented the development team, projects succeed only when the developers fully understand, up front exactly what they have to develop and why. In the bad old days, a big part of the “up front design” would entail employing an analyst whose job was to understand and describe the business domain, and the problems that needed to be solved, and to impart that shared understanding to the development team. These people were business experts, not technologists, with many years of experience in that sphere of commerce, and a good background in business, and so could define precisely the customer’s requirements.

While a central strand of a methodology such as SCRUM is that a business expert from the client is “part of the development team”, does this really happen in practice? If someone in the business understands the broad activities of the company, they are generally too valuable to be seconded to a development team as a ‘stakeholder’.

This is the dual problem. Firstly, too many projects seem happy to assume that technologists are the right people to describe both the business domain and the technical solution. They aren’t. Secondly, in a relentlessly iterative process, where the only measure of progress is working software, how does one incorporate a “proper” up-front design phase? What does it even mean? Alternatively, how does one define “just enough design”?