Once your customer satisfaction survey has closed, the next step is preparing your survey data for analysis. But this can be challenging for many companies, especially when questions arise around data quality. Did the survey contain errors that potentially introduced uncertainty or bias? Was the contact list up-to-date so that the right customer groups were invited to participate in the survey?
Before you go down the road of making business decisions based on the results, you must ensure you’re collecting clean, unbiased, and accurate customer feedback data—and that means ALL the data.
Eliminating responses for any reason is a slippery slope and can have a significant impact on how your company draws conclusions and takes action based on the feedback. The more you begin to nitpick, the easier it is to find reasons not to include certain data. When that happens, the trustworthiness of your survey data begins to deteriorate.
When Survey Data Cleansing Is Necessary
There are exceptions to the rule, but they are few and far between. At Satrix, we only recommend eliminating answers that fall outside of the clearly documented methodology that should be prepared before the survey launches. These reasons include:
If non-customers received your Net Promoter Score (NPS) or Customer Satisfaction Survey – As we mentioned before, it’s vital to make sure your lists are updated frequently to ensure the right people are receiving the right feedback surveys. If you find responses from non-clients in your customer satisfaction or NPS surveys, it’s best to remove them to prevent skewing the data. This is also a good indicator that it’s time to ensure your customer contact lists are updated.
If the survey was sent to a new customer – A client who’s only been with you for, say, a week, isn’t going to provide the most valuable and insightful feedback in response to your customer satisfaction or NPS survey. In most cases, we recommend a customer be on board for at least a month or 90 days before inviting them to participate in a “relationship” survey. If you ask for their feedback too soon, the opinions shared will likely be based on an incomplete picture, and may contaminate your data.
If you received duplicate responses – Survey technology is sometimes imperfect, as are the people taking them. Maybe someone accidentally took the survey twice. Someone may even be listed under two email addresses. If that happens, it’s generally a good rule of thumb to keep the first response and remove the others.
Wrong customer contacts were invited – Sometimes a database will include customer contacts who were incorrectly added or left on even though their role within the company has changed. Offering incentives increases the likelihood of corrupted data as someone who really doesn’t know the product/service/value responds simply in the hope of winning.
When to Avoid Removing Survey Responses
If you believe customers don’t understand the question – Never underestimate the importance of survey design when it comes to customer feedback programs. Rather than just removing the answers of the people who appeared to misunderstand, this should be a strong indicator that you need to evaluate the question language and scale to ensure there is no ambiguity.
Respondents don’t complete the survey – Unless full completion is essential to the data you’re gathering, there’s no reason to throw out data from partially completed surveys. If you have a high rate of abandonment, it may be another red flag that there are issues with survey design. Perhaps it was too long, or the questions were irrelevant. Regardless, this shouldn’t lessen the impact of the feedback they DID give you because they went out of their way to make sure they answered particular questions.
You’ve met your desired sample size – While it’s true that you only need to reach a certain sample size to guarantee your data is statistically significant, there’s no need to implement an immediate cutoff as soon as that number is reached. The more customers you hear from, the better. Obtaining a larger samples size also helps increase the confidence level of your results and reduce the margin of error.
Additional Reasons You May Want to Clean Survey Data
Here are several other considerations for deciding on whether to eliminate or clean up your survey data. In most cases, we recommend you follow up with customers for more clarification:
- You believe someone accidentally selected “extremely dissatisfied” instead of “extremely satisfied” (for example) by mistake.
- You believe a respondent answered a question in a way that wasn’t expected or maybe took it too literal. This is commonly seen in NPS surveys when a respondent follows up the response with, “I don’t know anyone to recommend you to.” While we don’t generally advocate eliminating that response, it’s important to understand any other considerations that may have led to the respondent scoring the way they did.
- You believe responses are suspicious. Internet service providers often have the technology to automatically determine whether emails are spam. Unfortunately, a couple of them use software that may select a response to your first survey question, registering a selection without human involvement.
In some of the cases above, ‘bad’ responses should be considered for removal under the right circumstances.
Should Your Company Engage in Survey Data Cleansing?
At Satrix Solutions, we sometimes find it necessary to clean survey data on behalf of our clients. This is especially true with smaller data sets where a couple of “bad” responses might impact results in a meaningful way. After all, our clients are making business decisions based on customer feedback so it’s important to ensure accuracy. Which brings me to our next point.
Having people internally who are too close to the survey outcome and making decisions on which responses to remove is risky. That’s why it’s strongly recommended that you work with a third-party expert to oversee your customer satisfaction survey endeavor.
Finally, it’s important to remember that your end goal isn’t about a score; it’s about cultivating a culture around customer-centricity. It’s a valuable opportunity for you to gain a deeper understanding of your customers’ perceptions and experiences in working with your company, and finding ways to fine-tune your systems, processes, products and services to better serve your customers. Plus, it’s a chance to show customers that not only are you listening to what they have to say, you also care about their input and actively put it to use.