Case study: Over 50 SmartAssembly reports came in overnight
If your software testing relies on user feedback, then why not gather much, much more of it? And what about getting it faster, too? Those were the questions developers on SQL Source Control found themselves pondering when they began using SmartAssembly, Red Gate's new error-reporting and obfuscation tool.
The team used SmartAssembly to add automated error reporting to the SQL Source Control early access release, and customized it to provide them with all the details they needed for quick fixes.
The results were dramatic. In the time it took 100 users to get in touch by email, 600 had reported through SmartAssembly.
David Simner, a developer on SQL Source Control, says,
SmartAssembly makes early access releases really useful. Users can send error reports about bugs and usability issues at the push of a button. Developers get full details about what's happened and enough information to reproduce it. You can fix the bug straight away and get the next version ready much more quickly.
Previously, the team had relied on emails and forums to get information on bugs. Few users sent in reports and even fewer were able to provide full details. It was difficult to pinpoint errors, so testing was a drawn-out process and tools took a long time to get to market.
The tool's error reports include a full stack trace and the values of all the variables – the information needed to isolate the problem. As SmartAssembly's reports are created automatically, the burden of reporting is taken off the user, and developers don't need to chase up the specifics. For the SQL Source Control team this has been invaluable in getting a stable release ready quickly.
With the first release candidate, there was a bug in the code used to migrate the user's settings. Over 50 SmartAssembly reports came in overnight. The bug was really easy to fix, and the second release candidate went out two hours later,
More than 60 distinct bugs have now been reported with SmartAssembly.
Some of the errors have been really obscure. My favorite was one where the program crashed when you moved the mouse while typing. It happened because we were trying to show tooltips a certain distance from the mouse cursor. Windows hides the cursor by default when you're typing, which meant our code didn't work if the user was holding a key down when they moved over the UI. With SmartAssembly, we were able to get to the heart of the problem directly.
The pace and precision of SmartAssembly's reporting have been decisive in getting a prompt launch for SQL Source Control. After a successful trial with SQL Source Control, Red Gate is rolling it out across all its early access programs. As Ellie Ryder, brand manager with the SQL Source Control team, says:
With SmartAssembly we've been able to release SQL Source Control in a much shorter time. It should be more stable than if we'd relied on email reporting alone. And, without the hassle of email reporting, developers could focus on actually fixing things. It's been fantastic to put out a version we're happy with so quickly.
Try SmartAssembly for yourself:
SmartAssembly can protect your Winforms, Silverlight, Windows Phone 7, ASP.NET applications and more. It can also be integrated into build processes using MS Build or the command-line.