The survey industry has changed. Thirty years ago, if you wanted to survey 300 people about your new idea, you’d post questionnaires to people around the country who would walk door-to-door, ask people questions and write down the answers. The surveys were posted back for data input and analysis.
I spent the first six weeks of my working life trying each stage of this process. The door-to-door bit was tough.
In 2025, we have panels of consumers who give their opinions on lots of topics. They are emailed links to surveys. They tick boxes on screen, and the data transmits to a central server. So it is far quicker and cheaper to get survey data, and there are far more surveys conducted.
There are downsides. Many panels offer ‘self-service’ that allows anyone to write and field questionnaires. So the quality of data is mixed. The worst case is untrained people, writing poor questions: “do you think this product is brilliant or just great?” Or writing surveys that are so boring consumers are not engaged: “rate these eight retailers on these 25 attributes.”
So what can you do to get the best from surveys and avoid being influenced by poor design?
The three survey principles
First, make sure the right people are designing your surveys. Whether in-house or in agencies, they should be properly trained in the subtleties of survey design.
It is right that we increasingly ask insight professionals to focus on driving change within organisations, but it’s equally important that they know research techniques. So ask the question – who designed this survey?
Second, test drive it. Ask for a link so you can do the survey for real, experiencing it as your consumer would. Is it interesting enough to keep you engaged? Do the questions and answer options allow you to express what you really think? If you can’t try the survey yourself, make sure someone else does.
Third, listen to the survey output, but do not blindly follow. Yes, be data-driven – but only if the data is expertly designed and sourced. If you’re shown survey data that challenges your beliefs and experience, then absolutely ask yourself if you have been mistaken.
But also stress-test the research method, what the exact question was, and so on. And if a data point still feels off, try another source. If it still says your preconception was wrong, then it probably was!
Well-designed surveys are powerful because they quantify what consumers think and feel about products and services. It’s important we don’t debase that gold with the impurity of rubbish surveys.
Excellent surveys lead to excellent data and excellent decisions. Poor surveys lead to poor data and poor decisions.
Jeremy Garlick, partner at Insight Traction
No comments yet