More and more frequently, the chore of installing a new application is interrupted by the application asking you to allow anonymized data about your use of it to be sent to the application’s publisher. In the ‘startup’ culture, there seems to be an increasing acceptance that an application just needs to capture a particular business idea, implement something that just about works and looks kind of cute, and then the application can evolve stepwise towards perfection by dint of user feedback and feature-usage reporting. A/B testing will take care of any design issues, user feedback will determine new features, and usage stats will reveal the ground-swell of opinion regarding what users like and dislike about the product. Continuous delivery then provides the necessary means of rapid change.

Feature-usage is powerful magic, of course, but there are real snags. Getting meaningful results from A/B testing, for example, requires rigorous exclusion of all other factors. Correlations between individual metrics can’t be detected easily if there is any latency between an improvement and its subsequent effect on usage. The closer you get to the data, the more illusory the significance becomes. Your key performance-indicators may look as though they can measure the effectiveness of your application but are too likely to be misleading. Unless you’re very lucky, the actual number of active users will be too low to justify drawing valid conclusions from the data you are getting. Worst of all, if these metrics actually start to work for you, they can beguile you into believing that the creative idea, intuition, and ‘thinking outside the box’ are of diminished or negligible value.

Had this type of ‘design by evolution’ become general practice within technology in the past, we would be travelling to work pulled by highly-refined and efficient steam engines. Evolution provides blind alleys as well as the road to success. Certainly, a piece of design such as a hammer can be refined and refined until it is beautifully fit for purpose, but no conceptual leap is possible. The real process of evolution requires that the vast majority of life-forms become breakfast for another species.

Good design can’t easily be crowd-sourced. After all, art galleries don’t provide felt-tip pens for visitors to make step-wise improvements to works of art. There is no doubt, however, that rapid feedback from users, and data about usage, is becoming an increasingly vital part of the development process. Perhaps the only danger is in letting the feedback about usage dominate the design process to the exclusion of the creative thought.