Testing Techniques Part 11: A/B Testing

If there was a fly on your conference room wall, would it ever overhear a conversation like this? “Cool. Let’s do it this way.” “But wait, what if this way is better?” “Well, we have to pick one. So, which do we go with?” (Conversations like this happen a lot here at We ❤ Health Literacy Headquarters.)

Fortunately, in this latest installment of testing techniques, we have a solution for you! It’s called A/B testing, and we really ❤ it. With A/B testing, you put 2 versions of a message or design out into the world — that’s the “A” and the “B” — and gather data about which version is more effective or impactful.

Emails are prime A/B testing material — you can test different subject lines, calls to action, or designs. Let’s say, for example, you want to know which type of email subject line will get more readers to open your e-newsletter.

First, pick the types of subject lines you want to try out. Maybe the first takes a question-style approach and the second is a bit more clickbait-y, like so:

  • Option A: Does someone in your family have allergies or asthma?
  • Option B: 5 ways to help a loved one with allergies or asthma

When you have an A and a B, you’re ready to test them out. Using your email delivery system, send a random sampling of half your subscribers the email with subject line A. Send the other half of your list the same email, but with subject line B.

Then, look at the analytics and you’ll see which subject line led to a higher open rate. Pretty cool, right? The more times you do this, the better sense you’ll get for which type of subject line readers prefer. After that, as they say — lather, rinse, repeat.

Note that it’s best to do A/B testing with large numbers of participants — like your organization’s e-marketing list — to make sure you get a meaningful response.

And one last thing: with A/B testing, it’s important to eliminate variables. For example, if you’re trying to gauge the click rate with a new layout of your e-newsletter, use the same content in both options. Otherwise, your findings could be muddled. Did users respond to the layout or to a different content approach? You’ll never know.

The bottom line: Don’t know which way to go? Use A/B testing to explore 2 ideas at once.

Tweet about it: Don’t know which way to go? A/B testing to the rescue, says @CommunicateHlth: http://bit.ly/2o9MGLE #HealthLit

Health Literacy in the Wild: Feedback Edition

Since the dawn of We ❤︎ Health Literacy, our posts have focused on tips and tricks to help health communicators like you better reach your audiences. But what if we told you that you could apply these same health literacy best practices to interactions at your workplace?

Let’s use giving feedback as an example.

Here at We ❤︎ Health Literacy Headquarters, we’re big fans of giving and receiving feedback. We get feedback from audiences in user testing, we get feedback from our clients, and of course, we give feedback to each other every day. Feedback makes us stronger, smarter, and more motivated to do our best.

Here are a few pointers from the health literacy playbook that you can use to make your feedback more impactful:

  • Be positive. Research shows that people are more likely to change their behavior when they’re rewarded for doing so. If a colleague did an excellent job on a project, tell them! And if you need to give constructive feedback, focus on the benefits of trying a different approach. A little praise and encouragement go a long way.
  • Be specific. When you’re giving feedback, make sure to share concrete examples so your colleague is clear on what they’re doing well and what they need to work on. You may think you’re being clear when you say, “I’d like you to come prepared to meetings,” but it’s clearer to say, “I’d like you to prepare an agenda for our meetings and share it with me ahead of time.”
  • Be actionable. Make sure your colleague walks away from the conversation with a plan for how to apply your feedback. Work together to come up with specific action steps — then check in later to see how it’s going.
  • Make time to listen. Part of giving good feedback in the workplace is hearing the other person’s perspective. Ask questions to try to understand where they’re coming from.

The bottom line: When you’re giving feedback, take a page from the health literacy playbook: be positive, specific, and actionable — and make time to listen.

Tweet about it: How can #HealthLit principles help you give better feedback at work? @CommunicateHlth has tips: http://bit.ly/2nMRrLQ

Useful Theory: Diffusion of Innovations

Imagine you have come up with an awesome new health literacy initiative. You’ve put in the time planning, developing, and testing it. You’re about to educate hundreds — scratch that — thousands. You can see it catching on: health literacy is taking the world by storm.

Now, how do you make your vision a reality? That, dear readers, is a very important question! The sad truth is that most big ideas don’t make it because more attention is given to creating the ideas than spreading them.

Fear not, health literacy warrior! Look to the Diffusion of Innovations Theory. This theory describes many things, but where we see its value is in explaining how new innovations catch on. Understanding this can help you better promote and market your program.

Use the following concepts when you explain the benefits of your program to the masses. (Don’t be intimidated by the fancy buzzwords!)

  • Relative advantage: How much better your innovation is than whatever it’s replacing
  • Compatibility: How your innovation connects with potential users’ experiences and values
  • Complexity: How easy or hard the innovation is to adopt (we’d like to rename this one Simplicity)
  • Trialability: Whether or not users can try out the innovation before committing to it
  • Observability: How easy it is for users (and the people they know) to see results from using the innovation

By applying these concepts when you promote your program, you’ll greatly increase the chance that people will give it a try.

The bottom line: When promoting a new idea, use the Diffusion of Innovations Theory to increase the chance of it catching on.

Tweet about it: Diffusion of Innovations Theory to the rescue! http://bit.ly/2rYimsF via @CommunicateHlth #HealthLit

Sesame Street: An Education in Public Health Messaging

A TV screen shows Big Bird and Oscar the Grouch standing alongside two CommunicateHealth doodles.
Alt: Sesame Street characters share public health messages with kid doodles.

Chances are, dear readers, you’ve seen at least a few episodes of Sesame Street over the years. Or maybe you’ve seen a lot more than that — and some of your earliest lessons in literacy and numeracy came from Elmo, Big Bird, Cookie Monster, and friends. (There are lots of us out there!)

Sesame Street is famous for finding fun ways to teach kids the ABCs and 123s — and even lessons in empathy. So you may not be surprised to hear us say that this show has plenty to teach us about effective public health messaging.

Despite being a show for kids, Sesame Street doesn’t shy away from complex topics like HIV/AIDS, poverty, and racism — and its cast of Muppets (and humans!) is very diverse. The show helps challenge stigma by featuring accurate, sensitive portrayals of characters who are dealing with everything from autism to having a parent who’s in prison. They’ve even created materials to help children deal with the effects of trauma.

Sesame Street also excels at providing kids with super actionable messages. Characters share tips for healthy behaviors — like riding bikes safely and oral health care — with catchy song-and-dance numbers that keep kids’ attention (talk about knowing your audience).

Finally, in using language that kids can understand, Sesame Street is a model clear communicator. Check out how Elmo and former Surgeon General Vivek Murthy tackled the tricky issue of vaccine confidence — we just ❤︎ Dr. Murthy’s funny, compassionate, plain language explanation of how vaccines work.

The bottom line: Keep an eye on Sesame Street to learn strategies for communicating important public health messages.

Tweet about it: Look no further than @sesamestreet to see #HealthLit strategies in action, says @CommunicateHlth. (It can even help us #communicateCOVID!) https://bit.ly/3euamTT

Movie Club: Unrest

Four doodles are gathered in a living room staring forward, as if watching a screen. One is eating popcorn.

Today, dear readers, we want to tell you about a documentary that has some of us at We ❤︎ Health Literacy Headquarters talking. Unrest chronicles filmmaker Jennifer Brea’s experience with a complex and confusing illness: myalgic encephalomyelitis, also called chronic fatigue syndrome or ME/CFS.

Through her own story and the stories of others living with ME/CFS, Brea offers a sensitive, honest look at the inherent challenges of the disease — as well as the accompanying struggles that have nothing to do with ME/CFS itself.

Unrest explores the medical community’s lack of understanding about ME/CFS and the implications of that for people living with the disease. Before being diagnosed, Brea was repeatedly disbelieved by doctors or told that her illness was “all in her head” — something that’s unfortunately very common for people with ME/CFS (and perhaps especially for women).

The film also highlights how people with disabilities or chronic illnesses are using the internet to connect with each other, to find the answers their doctors are unable to provide, and to organize and advocate for themselves.

We talk a lot in these posts about the best ways to communicate health information. In Unrest, Brea chronicles the harm that can result when patients, families, and doctors don’t even have the information — and how empowering it is when people are able to connect and share knowledge. For us as health communicators, it’s an important reminder of why we do what we do.

The bottom line: Unrest is a powerful documentary that offers a window into the experience of living with a mysterious chronic illness. Watch it on PBS (until January 23) or Netflix.

Frequently Asked Question: Should My Site Use Audience-Based Navigation?

A doodle stands on a stage, and a group of doodles stands around it. One doodle in the crowd says, "Where would I go?" and another says, "But...who am I?" The doodle on stage points at the crowd and says "YOU get a navigation, and YOU get a navigation!"

Here at We ❤️︎ Health Literacy Headquarters, one question we get over and over again is: Should my site use topic-based or audience-based navigation?

Let’s back up for a minute and review the basics. Topic-based navigation is when a site is organized into sections based on different topics or user tasks — like HIV.gov. Audience-based navigation, perhaps unsurprisingly, uses sections that are geared toward different audiences — like KidsHealth.org.

The idea behind audience-based navigation is that people will save time by going straight to content designed for them — and they won’t get bogged down in features they don’t find relevant or useful. Great idea, right?

Actually, most of the time it’s not. In practice, audience-based navigation presents a number of problems:

  • It forces people to label themselves. This can be a problem for folks who could be part of more than one audience — or who don’t fit into any of the options you give them.
  • It’s often downright confusing. People may not be sure whether the information in a section is about the specified audience or for it. Case in point: If you’re a patient looking for a new primary care doctor, why wouldn’t you choose a section labeled “Doctors”?
  • It can make people worry that the information they’re seeing is incomplete — and that can mess with your credibility. People may wonder if another section of the site has information theirs doesn’t (think: what do doctors get to know that patients don’t?!). This might cause folks to try switching between sections and end up lost.
  • It’s more work to maintain. If you have content that applies to more than one audience, you’ll need to duplicate it in multiple sections — or create a page that you link to from multiple places. Either way, more work for you!

At the end of the day, audience-based navigation just doesn’t represent how people think. Folks typically come to health websites to complete a specific task or learn about a health topic, and that’s what’s on their minds — not which audience they belong to.

So is audience-based navigation completely off limits? Not exactly, dear readers. An audience-based nav might make sense if your audience groups — and the content designed for them — are truly and totally different. (Think of the students, parents, and faculty sections on a university website.)

The bottom line: Skip audience-based navigation — almost all of the time.


Tweet about it: Most of the time, audience-based navigation doesn’t do your audiences any favors. @CommunicateHlth explains: https://bit.ly/3Lr4Elj #UX #HealthComm

It’s interactive!

2 web pages stand on a stage, as one sings, "I've got your click and drags, sliders and state changes!" and the audience remarks that the other one looks a bit static.

The internet has given us so much: Grumpy Cat, Baby Monkey, and — a recent fave — NPR’s accidental #Ramona phenomenon.

But there’s another reason we’ve been appreciating the web so much lately, dear readers. It offers a ton of options for getting people to pay attention to and engage with health information in new ways. Yep, we’re talking about interactivity.

Unlike static graphics or tables (which, as their name implies, just sit there), interactive graphics and tools do more than just show content or data. They let users make choices about what they see and, often, how they see it.

Interactive elements can give your users the power to:

But before your design team starts learning the latest animation software, make sure that an interactive format is a good fit for what you’re trying to communicate. For example, if you have a very simple message, you may want to consider a static graphic that packs a punch.

And if you decide to go the interactive route, check out Health Literacy Online — a resource we ❤︎ that has guidance for designing intuitive interactive graphics and tools.

The bottom line: Take advantage of the web by using interactivity to engage users and tell them what they want to know.

Happy Holidays from We ❤ Health Literacy!

A blue yeti with yellow horns stands and waves from a frozen landscape. Above its head, text reads, "Happy Holidays from CommunicateHealth."

The holidays are upon us, dear readers. Festive sights, sounds, and smells abound, and we hope you’re enjoying the season as much as we are. To that end, we wanted to bring you a special holiday treat!

Here’s the deal: Our darling Betti the Plain Language Yeti is on a quest for the Gift of Plain Language, and it’s up to you and your plain language know-how to help her! Using your keyboard, follow the path and choose plain language words to get to the center of the maze. And keep an eye out for some of our Doodle friends — following directions isn’t really their strength…

Play the game

The bottom line: Warm wishes and holiday cheer to you and yours from all of us at CommunicateHealth!

Testing Techniques Part 10: Design the Box

Six doodles stand around a table covered with art supplies. One doodle says "Mmhm...nice..." while looking at another doodle's box. That doodle is holding the box — complete with muscle arms, a unicorn, and the word "rad" — above its head, yelling "This is the ultimate box!" Other doodles stare at it and say "Wooow..." and "GASP!"

Is your team lost in the product development process? Do you want to see how someone else might envision this thing you’re making? Maybe you’re interested in getting at the je ne sais quoi of a successful product? Never fear, dear readers! Our latest installment on testing techniques is about “design the box,” a research activity that asks participants to decorate a physical box representing a (digital or tangible) product.

The goal of design the box is for participants to capture the product’s most important qualities and visually represent how they could be displayed — and how they might interact with each other. Folks can do this by writing words on the box, gluing magazine pictures to it, or any other crafty approach they please.

To prepare, you’ll need to gather supplies — including, of course, boxes (shoeboxes work great). You’ll also want to provide things like markers, scissors, tape, magazines…you get the idea. Then divide participants into teams. We find that teams of 3 to 6 people work well, but if you have fewer people, you can also have people work in pairs.

Now it’s time for participants to channel their creativity! Ask them to imagine that the box is the package for the product in question — and their job is to design it in a way that helps sell the product. For example, if you’re working on a new app for tracking physical activity, you can tell participants to think about things like:

  • Who is the app designed for?
  • What’s important to that audience?
  • What will make them want to buy and use the app?
  • What makes this app different from other, similar apps?

Give teams a set amount of time to design their box — say, 30 minutes — and ask them to prepare some talking points about why they designed the box the way they did. When time’s up, have each group present their creation. Then facilitate a sure-to-be enlightening discussion about participants’ choices and their vision for the product.

Keep in mind that participants can be either internal stakeholders or end users of the future product. Each of these groups represents a unique perspective. If you can, do 2 separate sessions with both types of participants!

The bottom line: Design the box taps into creativity, collaboration, and fresh perspectives for product development. It might just be exactly what your research process needs.

Color Me Accessible

Three doodles stand next to one another, wearing blue beanie hats. The first one says, "You mean we've been accessible this WHOLE TIME?!" The second one says, "Good ole #008290!" The third one says, "I...feel so...compliant!"

Here at We ❤︎ Health Literacy Headquarters, we always have accessibility in mind — but on this day each year we think about it with a little extra love. That’s because it’s Blue Beanie Day, a worldwide celebration of the importance of web design that makes content accessible to and usable for people with a diverse range of needs. (Hooray!)

As you may know, dear readers, we’ve shared accessibility tips before — on topics like alt text and labeling, designing for keyboard navigation, and accessibility plug-ins. This week, we’d like to draw your attention (get it?) to color contrast.

Color contrast is the difference in color between elements on a page. Things with low contrast (for example, dark blue text on a black background) can be hard to see. Using text with high contrast (like black text on a white background) makes your content easier for everyone to read — and it’s especially important for people with low vision.

So, how much contrast is enough? We recommend using Section 508 guidelines. Check out these tools to make sure your site passes high-contrast muster:

The bottom line: On Blue Beanie Day (and every day!), a quick color check goes a long way toward making health information as accessible as it can be.