It is hardly surprising that, in general, only around a quarter of software projects succeed, given the number of things that can go wrong. Creating a successful Software tool has its own special difficulties, but because the process is so creative, it can give great job-satisfaction if you get it right.

The members of a development team will inevitably bring their individual biases and opinions and you’re going to have to reconcile all these if you are to succeed:

• The developers will want to cram in as many features as possible. A typical quote is “If we are late so be it. Hey it was difficult, what can you expect, things took longer than I thought.”
• The testers vary in their pragmatism, but generally want to make sure that the product is perfect and hence aren’t happy when there are bugs in the software.
• The customers want new features or they won’t buy it
• The management want to release products on a given time scale and budget. They don’t like surprises.

It is almost inevitable that a project starts with a phase of euphoria. Developers will always want to over-stretch the software and be too ambitious. At this stage, before reality kicks in, almost anything seems possible. Expectations are high.

Once the planning phase is complete, development begins apace, and the testers start getting heavily involved. Pain points start to appear. The developers have overreached in terms of features, and the testers (quite rightly) are piling on the bug reports. In extreme cases, when rampant feature-creep couples with a culture of ‘zero-defect’ testing, the whole project can lock up, and features are pulled as a result.

Many projects suffer from these problems to greater or lesser degrees. At Red-Gate, I’ve also encountered the added problem that the programmers feel a parental loyalty to tools they worked on previously. This can mean that 50% of the team is occasionally pulled off to do other work. This obviously de-focuses the team and leads to slippages. The answer is easy, just say “No”; but it’s not always easy when the product was perhaps something that you worked on 2 months ago, and still feel attached to.

The SQL Data Generator Project: a different approach

A short while ago, we decided to develop a new product called SQL Data Generator, to provide a quick way to generate meaningful test data for Microsoft’s SQL Server. In many ways it was a typical software development project, and probably destined to follow a course similar to that described above. However, we made the decision to try a slightly different approach to the development cycle.

We were determined to circumvent some of the typical problems described above, and at the same time to provide the features that real DBAs, and Database Developers really wanted. We knew that if DBAs and developers think they can truly influence the feature set of a tool, they will provide valuable advice.

Therefore, rather than enthusiastically try to cram in all the useful features that we could think of, right from the start, we decided instead to create a simple framework for the project that contained only the features we considered vital.

We would get a working beta into the hands of the people who were going to use the tool – people with a very different type of workload and skill-set to us – as early as possible. We planned to then rework the existing feature set in the light of their feedback, and add new features that came from their suggestions. We hoped that this would allow us to ship an extremely useful 1.0 product.

In short, our development strategy was to:

• Define and design only the core software functionality and UI
• Develop and test the software to Beta release
• Redefine the functionality based on the Beta feedback
• Develop and test for the final product release.

The core software functionality and UI

At this stage, we wanted to produce a basic framework of ideas and concepts that would define the basic tool. The team could then use this framework for their development efforts.

We had a target delivery date in mind but, at the beginning, we didn’t want to get overly distracted by the timescales. Instead, we just concentrated on getting the ideas and concepts down. We achieved this by:

• Talking to developers and DBAs, directly and through surveys and emails
• Investigating some existing products on the market
• Using some in-house product knowledge

From this information we built up some typical scenarios and personas. Here is a simplified example of a ‘persona’

Typical Persona:

Mr Jeff Banks, DBA for medium-sized US-based Insurance Company. Leads a team of 5 people.

Scenario:

Jeff’s team is setting up the test of a merchant shop application. Jeff needs to populate the customer table in the database with a large amount of data to test the response time of this application.

Pain points:

Jeff  requires some elaborate data with several rules. This may require significant time to generate. This will start to matter if he requires millions of rows and we need to make sure that the generation routines do not get out of hand, and slow the process.

From a set of around five scenarios we constructed our initial feature list. As ever, of course, this feature list was too long.

Prioritizing the feature list, setting timescales

We assigned each feature on our list into one of three categories, essential, nice to have and next version. This gave us a list of preconditions i.e. features that must be included as well as a list of limitations i.e. things the software wasn’t going to do.

From here, we generated some rough project plans and timescales. As am I sure you could guess, the timescale we came up with didn’t fit with the target delivery date and, as a result, we had to go back and re-evaluate the feature list and our initial preconditions.

The feature list got trimmed and some of our initial ideas were dropped. So, for example, when developing the initial framework, we had the precondition that the tool should generate a SQL Script, which the developer could execute either through the tool or using Management Studio. All of Red Gate’s previous SQL tools worked in this manner and this, initially, was regarded as essential. However, at this stage, we decided that the tool would automatically generate the data through Bulk Copy Inserts, and no script would be produced.

This reduced our timescale but at the expense of not fulfilling one of the initial preconditions. In this case, the performance gain from a Bulk Copy was too great to ignore.. This was typical of the sort of balancing act that went on. Some of these debates were easy and some were more heated. It’s always difficult to remove functionality but we had a delivery date and we wanted to stay focussed on reaching it.

Visio Prototypes

In order to help us prioritize and rationalize the initial list of features, we developed some simple ‘paper prototypes’ based in Visio, which allowed us to test the workflow for our scenarios. Based on these simple prototypes, we asked some users if they could work through some of our scenarios. Their feedback was immediately fed back into the development cycle.

We also got the help and advice of technical authors at a very early stage in the design of the product, because, let’s face it, developers come up with sentences and phrases without any verbs and, given half a chance, words with no vowels.  We wanted to get the vocabulary, and the names of the components right at the ‘paper software’ stage. We also wanted parts of the documentation to develop before the software was written.

The paper software was very ephemeral, and changed almost by the hour. You could almost feel the software evolving. It seemed that each stage was only understood retrospectively, when we reviewed what we’d done..

Develop and test the software to Beta

Having, at last, finalized the initial feature list, we could crack on and start writing some code.

The testers immediately created:

• Some basic databases
• Some very simple, overnight regression tests

These simple testing frameworks were extended and enhanced over the project lifetime. In the end these tests became fully featured with most edge conditions etc catered for. We decided to split this project up a little differently from the way we usually do it. We started work simultaneously on the three key areas of the software:

• User Interface (i.e. the GUI and the command line)
• Engine (defines all the data structures etc and glues all the bits together)
• Generators (the code that actually generates the data)

After some debate, we laid out the base of our Interfaces and classes. These interfaces were deliberately quite loose to allow us the ability to change them relatively easily.

The developers could now go off and develop their parts of the software. Although the developers had responsibility for their specific areas of code, we all had to ensure that the code worked together. This, at times, involved updating other developers and testers code.

The Usability engineer continued to refine the UI design and change it when more of the requirements of the different types of  end-user was understood.

We did not feel that we had enough time to design everything up front and we would pay for inefficiency of reworking existing code by the fact that we could start sooner. This dynamic approach does have some drawbacks; the coordination, planning, and the monitoring of progress was harder to achieve accurately, However, the team were aware of the potential problems this would cause, and worked together to avoid any risk of delivering the produce late as a consequence.

We identified the code features that, we felt, defined the product, and concentrated on these. We wanted the Beta release to be as open-ended as possible so as to maximise feedback. As the Beta release date loomed, we completed and tested these features..

Bug Hunting

Then as ever we had a big crack down on bugs making sure that the product was of Beta standard. This involved as ever a Bug Hunt. At Red Gate we setup an afternoon where anyone from the company can come and break the software. As an incentive we give prizes to the person(s) for the best bug and most bugs. At these events there is always a large degree of competition to break someone else’s code.

We fixed the bugs and went through a final round of testing and released the Beta.

Re-defining the functionality based on the Beta feedback

We were amazed by the helpful and useful responses we got. However hard one tries, it is never possible to be aware of all the requirements of the developers at the sharp edge, requiring a SQL data generator. We agonised for a short while with the alternatives of just picking the best suggestions and hitting our scheduled dates, or incorporating all those wonderful suggestions, so painstakingly provided for us. We chose the latter.

We changed our minds about some functionality. Some functionality that was put on the nice to have list was moved up onto the essential list. Some of the functionality that wasn’t liked or needed was dropped.

It will be interesting to see how this software pans out. Everyone on the team has their own views and ideas, as a result even as we incorporated all the feedback from the Beta, we had internal struggles between the issues of functionality, quality and end dates.

As ever as developers wanted more and more functionality, the testers wanted the current functionality to be totally correct and bullet proof, and the project manager wanted to ship on time and to budget and the users are always right.

The ship is launched

The final stages of a project aren’t ever the most fun, as it is all about attention to detail. However, we were very buoyed up by the good response we had from our Beta testers, and the time went very quickly.

So now it is all ready, and even the testers have smiles on their faces. After working through the hundreds of jobs that have to be done before a product release, we can, at last, pause to celebrate. Will the developers and DBAs out there like the tool? We think so, from what they’ve already told us, but we’ll be on tenterhooks until our sales team give us the thumbs-up. Easy? Heck, no, but none of us would want to do anything else.

If you’d like to check out the software to see the result of all this work, then you can get an evaluation copy from the SQL Data Generator product page. What do you think? Please let us know!