A breathing-space for Software corpulence

A guest editorial for Simple Talk

The greatest struggle for anyone designing, building or maintaining applications is the fight against complexity. It chokes the lifeblood from any application. Chuck Moore, one of the founding fathers of IT recently said, in an interview with Simple-Talk:

“Complexity is the problem. Moving it from hardware to software, or vice versa, doesn’t help. Simplicity is the only answer. … I despair. Technology, and our very civilization, will get more and more complex until it collapses. There is no opposing pressure to limit this growth.”

This is not the howl of a lone wolf. Brian Kernighan once wrote, “Controlling complexity is the essence of computer programming.” And Ray Ozzie too, “Complexity kills. It sucks the life out of developers, it makes products difficult to plan, build and test, it introduces security challenges and it causes end-user and administrator frustration.” Probably the most memorable take on the problems of software complexity comes from Tony Hoare. “There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.”

A long time back, commercial-scale applications became too complex for mortals to track without using complex application-management applications. Project and configuration management systems were joined by bug-tracking systems, build management, automated testing, continuous integration and release management. With tools like these, we could all cope with more complexity, and run larger software teams. Integration was ever the issue, especially with the temptation to include the entire life-cycle of an application, including the architectural, deployment, and production-management phases; this is particularly true when sourcing your tools from different vendors. Within the past five years, Application Lifecycle Management (ALM) tools, such as Microsoft’s TFS, HP’s ALM Suite and IBM Rational Team Concert have promised the obvious solution: accepting a degree of single-vendor lock-in in return for a single integrated set of tools in which application providers can manage all aspects of their work.

This increasing automation is well-suited to promote ‘Agile’ methodologies, since it allows much shorter build cycles and provides a much closer cooperation between developers and testers. It certainly promotes the flow of information and makes it easier to spot potential problems.

My own nervousness comes partly from thinking of those ground-breaking tools such as distributed source control, software testing frameworks or development wikis that would never have penetrated through wall-to-wall management suites based on less open-ended ideas. I also wince at the subtle problems that would ensue from integrating cross-application databases into an ALM. Most of all, I wonder if there is now even less motivation to tackle the root-cause of the increasing inflexibility of application management, the uncontrolled complexity of application. Now that we can throw more silicon at corpulent software so cheaply, and use applications to manage the complexity of applications, where is the cost-benefit of keeping things as simple as possible?

Your views would be very welcome, as always.