In terms of web analytics and statistics more data is always better. Ask 1000 people who they’ll vote for and you get a 3.1% margin of error. Ask a 100 people and the margin of error goes up to 9.8%. Since you are asking fewer people you are less confident about saying something about the underlying population.
Same thing for AB and MVT testing. You can see for yourself with the handy Google Website Optimizer calculator. Assuming a 2% baseline conversion rate, a 20% expected conversion rate improvement and the only change is the number of visitors who see your experiment page. If 1000 visitors see it a day, the expected experiment duration is 17 days, but if only 100 people see it per day it is 175 days! Ouch…
Add in the fact that web analysis is likely done on an ad-hoc basis without dedicated people (thereby violating Avinash Kaushik’s 10/90 rule), what’s a busy small retailer to do?
I think the answer lies in focusing on big segments, mini goals and the checkout funnel. Let me explain:
Segmentation is the key to finding valuable nuggets in your web analytics. An average metric, such as your site conversion rate, hides the fact that your site traffic is made up of many different segments, some of which are above average and some below average. Once you see differences you can start looking at ways of doing more of the above average stuff and less of the below average stuff. However, if you don’t have much traffic to begin with, there is a risk that you will quickly look at segments that are too small to be significant. An example might be keywords that are converting very well, but only convert once or twice in a period of months. Hard to optimize that. Instead, try to work with big segments, such as:
- Visitors who added to cart. This is a much bigger segment than visitors who placed an order. What can we learn from people who add to cart vs those that don’t? Assuming that your add to cart rate is 6% (which is in line with a 2% site conversion rate), plug in this number into the Website Optimizer calculator. You now get a far less depressing duration of 56 days for 100 visitors if you were to try to optimize the add to cart rate.
- Visitors who use internal site search. I love internal site search anyway because it has lots of tactical stuff you can do, such as fixing zero results searches. But also look at how visitors are using site search, such as by source. Are you sending paid search traffic to a particular landing page, only to find that a large percentage immediately use your site search? Perhaps the landing page is not relevant enough.
- Branded keywords vs non-branded keywords. How are these segments behaving differently? Can you bucket groups of keywords to help you create more and better content that targets both the head and the long tail?
Getting more people to add to cart can also be viewed as a mini goal. Instead of only focusing on sales conversions try to get more visitors to add to cart first. Or you can try to get more visitors to look at product detail pages by making smart categorization choices.
Here is a very interesting metric to look at: Visits to Purchase. Pull it up in your favourite web analytics tool. You’ll probably find that most transactions happen in one visit. Big pat on the back? Not so fast. The way I see it is that you only have one shot to make a sale. After the initial visit, transactions go down rapidly. For this reason consider adding visitors to your mailing list first. This great mini goal helps you keep alive the conversation with your prospects. Obviously you should make sure you have something interesting to say to your prospects. Another example would be to provide your buyer’s guide via PDF in exchange for an email address.
Last, but certainly not least is the checkout funnel. The checkout funnel is of course hugely important for an online store. Your visitors have to go through it before placing an order. If you have not done so, set up a checkout funnel in your web analytics tool, beginning with the cart page and ending with the transaction page. You will have one or more to steps. Make a note of the funnel conversion rate and go back to the Website Optimizer calculator. Remember the example of 175 days? Let’s plug in a typical 25% funnel conversion rate, i.e. a 75% cart abandonment rate, and keep everything else the same. You get an expected experiment duration of only 10 days for 100 visitors. If anything, you can see how important the cart page is and should feature prominently in any optimization plan.