How did that Customer get there??

We were launching a fresh range of products with a brand new website, and The Boss wanted it done right. He didn’t mince his words and put the problem succinctly:

‘I spend a great deal of money on advertising, and I have no idea at all which are the effective adverts. Some advertisers, I suspect, provide me with no useful leads at all. Others represent a solid investment. Which are which?’

It might seem so easy to anyone unfamiliar with Internet retailing. At the point of purchase, all you have to do is to ask the customer where he heard about the site, requiring him to select an item from a list. The most popular choices are your best referrers, right? Well, not necessarily. If you were to put ‘Red Bull, My Spirit Guide’, or ‘President Mugabe told me’ as the first choice in the list, you might be surprised by their supposed effectiveness as advertisers. In truth, at the point of purchase, any mortal will be exasperated by all the questions, buttons and options and will just stab at any item in the form that gets between them and the goal of purchasing.

Still, no problem, you might think: we can just capture the ‘HTTP referrer’ in code, or analyse the logs. The HTTP referrer is the string that represents the URL of the page that contained the anchor <A> to the page being accessed. This can have valuable information in it, but it is not the whole story. You can access the IP Address of the user, the ‘User Agent’, and the name and version of the browser. With this little lot, or with Google Analytics, one can wow the marketing department with “Facts”.

However, unless you’re very careful, this won’t keep them happy for long. Getting statistics out of website usage isn’t a real problem. There are many ways to do it and plenty commercial packages that provide a ‘turnkey’ solution. The real problem is getting your hands on those truly useful and meaningful statistics that the Boss wants.

Don’t count your hits until they’re hatched

Too much of the wrong sort of statistics can be a massive distraction and can result in all sorts of misconceptions. Many people rely on crude measures, such as hits, or page views, to assess the interest in aspects of a website, or the effectiveness of a particular advertising campaign. However, in my view, the number of ‘Hits’ or ‘Page views’, or even ‘visits’ should be treated as only a very rough and ready indicator of quality.

A lot of the software that provide these stats don’t even remove the effects of all the search-engine crawlers, or the monitoring systems that check that the website is responding properly to HTTP requests. One can also get inflated numbers of ‘hits’ when someone on another site, ‘borrows’ a graphic from your site. I’ve also experienced hilarious effects caused by an innocent combination of words used in an article on a website that just happened to correspond to a search term related to some strange and exotic sexual practice, beloved of web wanderers. Fortunately, I didn’t ascribe the number of visits to my powerful and arresting prose style.

Ascribing overriding importance to the number of visits to a commercial site is short-sighted. After all, if you are running a shop, you measure your success by the ringing of the till, not the numbers drifting in and out. The only truly useful marketing statistic to come from a commercial website, in my opinion, is the number of purchases, how the purchaser found the site, and why the customer decided to purchase. For The Boss’s website, we had wildly differing numbers of visits, depending on the media mentions we were getting, but the number of visits in any given interval of time didn’t correlate with the number of purchases.

Likewise, you can track the number of “hits” generated from your various advertising campaigns, by adding a tail, or query string, into the URL to indicate the advert it was that provided the referral, and that will provide a rough idea of what works and what doesn’t – but even that is not the whole truth. An increasing number of people, on seeing such an advert, merely go to Google and tap in the name of the product, thereby giving the search-engine an inflated credit for the sale. Furthermore, a lot of advertising is in the print media, or is advertorial comment, so won’t generate direct referrals, but may nevertheless be highly effective.

Despite the wealth of stats that we now have at our fingertips, there is still nothing that can replace good old-fashioned human feedback. One of our tactics while working on The Boss’s website was to phone up every purchaser, on the excuse of making sure they were satisfied with the product and their experience with the website. Unfortunately, though there feedback provided useful information, the number of phone numbers answered was low, around thirty percent, but the quality of that feedback was very high. An added bonus was that the purchasers really appreciated the fact that we cared and were real people rather than automatons.

Understand the purchaser

Perhaps the biggest problem with the sort of statistics that you can mine out of your web logs, is that the information you get is at the level of the HTTP request – it doesn’t tie this information to the higher-level session information, nor does it tell you which of your users then go on to become customers, by purchasing something, or which visitors are existing customers revisiting the site.

Another tactic we used on The Boss’s site was to record into a SQL Server database the IP address, User Agent and HTTP_Referrer, along with a cookie code, session key, and URL, for every page request. This duplicates a lot of the information in the Web Log but ties it in to what the application understands to be the visitor or customer. The cookie code, which was a random code for security reasons, allowed us to relate the session to whatever information we had from the user. By integrating all this, we could then, produce instant reports on who accessed the site, and the steps that were taken to purchase. We could spot people who were having difficulties, who were undecided, and so on. In a few cases, we were able to send helpful emails to prompt users on how to navigate around difficulties.

Although a lot can be done in the way of session tracking by using a ‘Post’ or ‘Response’ code whilst the user is navigating around the site, there is no substitute for a unique code or ‘Key’ left in a cookie. If one can link separate sessions via such cookie keys, one could then track those people who looked at the site, possibly got distracted, popped back a couple of times over a fortnight and then finally purchased. There, at the start of their first visit, would be the vital initial HTTP_Referrer. It is just a matter of spitting on one’s hands and turning out the query that relates the initial referrer to the eventual identification of the customer when he purchases or registers. You then have a better idea how he found the site.

We knew that a database-driven site which held all session, web log and browser information was the only way to be reasonably ‘future-proof’ against the demands of the marketing men, and so it proved. In fact, it had other unexpected benefits such as allowing us to scale up to a ‘web-farm’ without any re-engineering effort.

Build in reporting from the ground up

After all the work and effort, I’m not sure if we provided The Boss with all the answers. We certainly discovered that there were no golden rules, and that a marketing strategy that worked for one product was a dismal failure with another. To sell things over the Internet, one has to gear the marketing campaign to the products you are selling. Sometimes, Google AdWords are perfect, but for other types of product they don’t work. Print advertising is sometimes wildly effective, particularly where the advert catches the mood of the publication. However, what really helped was being able to provide instant feedback, and for that, a good solid reporting system, allowing drill-down into the detail, was a godsend. Google Analytics is a fine tool, which will keep the marketing department happy for a long while before they finds the limit to what it will do. If you are serious about measuring how customers get to purchase on your site, or if you are running a commission system, there is no substitute for a merchant site that is designed from the ground-up with reporting and analysis in mind.