Depending on which network you watch, you may have noticed a distinct skew in post-debate analysis in the recent election roundups. Poll a CNN crowd and Biden is top of the class. Switch to Fox and it’s Trump all the way. This is clearly problematic for those looking to get to an unbiased read.
Customer satisfaction surveys suffer from just the same issue. Unless you sample a large cross-section of your customers, the results are often hard to trust. We tend to see extremes of opinion represented, while a large chunk of your customer base probably just won’t engage.
In polling circles this trend is known as ‘Non-Response Bias’. And it’s our contention, that this can be just as dangerous for retailers, as for those trying to make an accurate call during election season.
In this post, we take a look at some of the key considerations for customer surveys and provide pointers on how to beat the tricky area of misrepresentation.
- What’s a good sample size for a survey?
- How do you promote a survey?
- What percentage is a statistically valid sample?
- How do you boost customer survey response rates?
What’s a good sample size for a survey?
Estimates suggest that in the last 20 years, the median response rate for feedback has dropped from around 20% to just 5%. For many brick and mortar retailers, this has fallen to well under 1%.
While businesses focus on delivering ‘customer-centric’ service, the samples they’re working with are lower and lower. Part of the problem is in practice. Receipt based and email surveys are mostly lost in the mix. While a sample size of 5% may seem okay from a ‘customer experience’ perspective, for your store teams, it’s another story.
Without a constant flow of in-store data, actionable decision-making is near impossible. To deliver here, you need response rates of at least 60-70% daily. So how do you make the jump?
How should you promote a survey?
Typically, retail businesses rely on methods like email, or links on receipts. More recently companies like Survey Monkey or Qualtrics have also enabled SMS surveys. iPad solutions positioned at the point-of-sale also now offer a new way to engage customers.
While these methods have their virtues, there are problems with this approach. Response rates remain lower than ideal. When customers respond after the fact, they may be remembering an experience, in a way that is not reflective of the actual lived one.
Finally, there is the problem of actionability. By the time you receive enough responses to analyse your data, the store environment has changed. Customers increasingly make decisions faster than businesses can keep up with them. A recent analysis of UK supermarket customers showed that 11% had already switched to a new brand due to safety concerns.
The struggle is real.
What is a statistically valid sample size?
The question of statistical validity is not just one for your data team to worry about. The size of your sample directly impacts the margin of error that you’ll likely see in any analysis of customer feedback.
Confidence Intervals are a mathematical technique used to understand how size can impact accuracy. While the advanced stuff can be hard to grasp, at a top level it’s simple. The more data you collect, the better off your chances are of getting a representative sample.
A sample size of less than 100 is likely to have at least a 10% margin of error – dropping to just 2% when you get to around the 2,500 mark. While these figures may seem daunting – the truth is there’s a way to hit these targets. And to do it consistently.
How do you boost customer survey response rates?
The key to delivering high volume response rates, is all about when and where you ask your customers a question. Most surveys consist of a number of questions, asked in a linear fashion. By breaking this down and asking every customer a single question, the results can be transformative.
By collecting feedback as part of the checkout process itself, you can quickly see huge volumes of data roll in across each of your store locations. Depending on where the question is asked within the payment journey, we’ve seen results vary from 65% on average, to as high as the 80’s and even 90’s.
While high response rates are desirable, the true secret of volume, is in the ability to action strategic initiatives at speed. Many retail teams suffer from a critical blind spot into store performance. By leveraging a system that enables continuous improvements on a daily basis, you’ll soon find your brand level KPIs are driven through the ceiling.
__
If you’d like to read more on this subject, why not check out a related post. Here we look at the friction that develops between customer experience and store teams, when data is in short supply.
