As health communicators, we deal with survey data on the regular. Surveys can be a powerful tool to help us better understand our priority audiences and their information needs. But without thoughtful interpretation, survey data can be confusing at best — and misleading at worst.
So this week, we’re bringing you some food for thought about a key ingredient in every survey: the respondents. Answering the questions below will help you interpret, apply, and explain survey results with more nuance — and with a better sense of the people behind the data.
- Who took the survey? The first step in understanding survey results is getting a grip on exactly who answered the questions. For example, if 97 percent of your respondents regularly talk to cats, it’s pretty important to know that you only surveyed cat owners. Ideally, your survey’s eligibility requirements will overlap with key characteristics of your priority audience. But if they don’t perfectly align — say, your audience is cat owners but you were only able to survey people with spooky hairless sphynx cats — make sure you take those differences into account as you interpret the results.
- Who didn’t take the survey? If you sent the survey to 1,000 cat owners and only 100 responded, it’s important to think about what that response rate might mean. Start by taking a look at the demographics — were people of a certain race, age, or income level more likely to respond? Then you can think about what these differences mean for your results.
- How many people took the survey? In survey research, size really matters! After all, 90 percent of 1,000 people is very different from 90 percent of 10 people. And a result that seems noteworthy in a sample of 10 people — like 3 people who say that when they talk to their cats, their cats talk back — may not be so impressive if it’s 3 in a sample of 1,000. Larger sample sizes typically mean better-quality survey data — so keep sample size in mind to keep your results in perspective.
- How many respondents answered each question? Some survey questions may only apply to certain respondents. Let’s say you ask, “How often does your beloved cat bring you offerings of dead birds?” People with indoor cats are likely to skip that question (barring any unfortunate overlap with pet parakeets). So it’s important to always take careful note of the base — the number and description of respondents who answered each specific question. In this case, that would be the number of outdoor cat owners in your total sample size.
- Was the sample random or non-random? In a perfect world, we’d always use a random sample. Random samples are the next best thing to interviewing every single person in a population. But random sampling can be expensive and time consuming, especially if you’re researching a very specific population — people who have 12 or more cats, perhaps. So if your survey uses a research panel or another non-random sampling method, just remember that the results may not be generalizable to your whole priority audience.
- What’s the quality of the results? It’s important to take all survey results with a grain of salt. After all, we’re relying on people to self-report answers — and who wants to admit exactly how many times they have tried to dress up their cat as Mufasa from The Lion King? But the size of the grain will vary! A random sample of 10,000 people may yield high-quality results — so maybe just a dash of fine table salt. A non-random sample of 100 is a bit more iffy — think a generous dusting of coarse kosher flakes. Consider the quality of the results before you apply survey findings to your materials — and if you’re communicating survey results to others, make sure your framing mentions how much salt you recommend.
The bottom line: Take a closer look at respondents to better understand and explain survey results.
Browse recent posts