Avoiding Continuous Disintegration

A build is a regular health-check for the database. If all is well, then automated builds will run regularly and smoothly. Assuming all necessary database and server-level objects have been accounted for in the build, the team will be confident that little can go awry as it is shepherded through the release process to deployment.

Of course, if the database builds break frequently, then all bets are off. I recently read of a fairly typical report of a serious problem with a database deployment. It knocked the confidence of the affected organization to the point that they requested that the team perform less frequent releases, until they’d sorted out their build processes. I smile imagining talk of moving to “dis-continuous integration”, until the build is sorted out.

When builds fail, however, blaming the quality of the build process is akin to blaming the thermometer for the high temperature of the patient. Most often, a failed database build is merely the symptom of deeper architectural issues.

Many diverse problems can break a build, just as many illnesses can cause a high temperature. A common blight of the database build is the existence of cross-database dependencies. At some point, the build chokes on a reference from one object to a dependent object in another database, which it can’t resolve for one reason or another. If the build is blessed with detailed error reporting, then it will at least highlight with surgical precision the exact cause of the problem, and it can be fixed quickly. More often, of course, the developer will see a vague “can’t find this column” error, and will be left to sift through the remaining few thousand objects that didn’t load to find the one that caused the problem.

No wonder, in such circumstances, database builds and releases are infrequent and often delayed, and therefore potential problems spotted much later in the cycle. So many databases ‘evolve’ from small-scale systems and come to rely on a rat’s nest of database inter-dependencies. Little thought has been given to structuring and modularizing the system via, for example, the use of schemas and interfaces. As the databases increase in size and functionality, the complexity and fragility balloons exponentially.

If you’re starting to dread the build process and wince when integration tests are run, isn’t it time to get as serious about the design of your database as you are towards the design of your application?