I first posted this essay on my personal blog and have moved it here, with a few small tweaks. This essay gives some insight into why Content Science is pursuing the Content + Credibility Study and why we offer content testing services.
I feel the need to say what should be the obvious. Why? Because recently, while catching up on my Twitter feed, the following statement smacked me like a gauntlet:
IT’S A FACT THAT USERS DON’T READ, AND WE HAVE TO DESIGN FOR IT.
I was too late to join the conversation, but the statement has concerned me ever since. In the user experience and design communities, has an assumption locked our thought about reading so tight that we refer to it as a “fact?”
Based on my experience with designing for and observing users, I am convinced that users read on the web (among other places). Sure, they scan hurriedly through irrelevant or uninteresting content until they arrive at what they want. (For a nice explanation, see pages 2-4 of “Letting Go of the Words.”) THEN, users read.
Why do we forget the reading part?
Think about some of the confining implications. If users don’t ever read, then
And, taking this assumption to its logical conclusion, if users don’t ever read, is there much point to having any words on the web?
These implications seem ridiculous when I shine the light of reason on them. But, when they lurk unexposed with the shrouded assumption that users don’t read, our design and content choices are at risk of suffocating. I think we need to revisit this “fact,” starting with an excavation of Jakob Nielsen’s influential study of How Users Read on the Web.
For background, read Jakob Nielsen’s explanation of the study. Also try to check out a longer version including explanations of related studies. In my searching, I did not find much constructive criticism of this study. I feel it has five limitations:
A tourist trip to Nebraska? I know very few people for whom this topic would be relevant. (No offense to Nebraskans out there! I’m sure it’s a beautiful state. It’s just not on most people’s destination list, even if it should be.) In fact, the topic was chosen specifically because people would likely know little about travel in Nebraska. My concern is that if the topic is not pertinent, people won’ be motivated to read about it.
I could let 1 go if the study recruited a sample of people who expressed interest in a trip to Nebraska. It also would be interesting to test a sample of people with interest and a sample of people with no interest. However, the explanation does not state that the study used such sample criteria.
The options include variations on a bombastic marketing version and a sparse objective version as well as variations on paragraph form and bulleted list form. Some important nuances are missing from the content options. How about a concise, promotional version that doesn’t lie and uses a bulleted list or a simple table? I also wonder whether wording variations and format variations are too many variables in one study. Furthermore, because the success metric focuses largely on remembering the list of tourist attractions, the content option that performs best—a bulleted list of the attractions—is designed to memorize.
The study explanation does not mention the purpose of the content and the overall website. Is the purpose to attract new tourists, to win back past tourists, to encourage tourism business, or something else? Did the study scenarios reflect the context realistically? Also, most of these possible contexts (which I inferred based on reading the original version of the content) seem persuasive, not educational or informational.
The study uses a reading usability metric including comprehension, recall, and time. It also includes a subjective measurement, but it is subjective mostly about usability qualities (how easy was it to find information, etc.). The metric does not address content meaning, influence, likelihood to visit Nebraska, or related measurements. From a user perspective, is the goal to remember the exact names of Nebraska’s tourist attractions? Or is the goal to make a confident decision about whether Nebraska is worth visiting? From a business perspective, is the goal to teach people about Nebraska’s specific tourist attractions? Or is it to convince people that Nebraska deserves to be on their travel itineraries? I believe the study tries to stick strictly to usability. But is it useful to measure success in a persuasive context without touching on meaning, influence, and broader goals?
The description of the metric shows awareness of context, noting that one might add weight to certain elements of the metric for an intranet or a leisure site. However, because the metric elements do not address persuasion, adjusting their weight for a persuasive context would not help.
In short, I believe these limitations stem from the following two mistakes:
Because of the limitations, I don’t feel the study allows us to conclude much more than the following statement: People with unknown interest in visiting Nebraska who are asked to learn about Nebraska’s tourist attractions remember those attractions best when they have little description beyond their name and display in a bulleted list.
We certainly can’t conclude from this study that people don’t read on the web.
Should we cut this study some slack because it happened 12 years ago? Yes and no. I truly appreciate how this study brought corporate attention to writing for the web. I respect the effort to test and measure reading usability at a time when the web was very new. I also am grateful that this and related studies inspired Redish’s useful description of users’ scanning behavior in “Letting Go of the Words.”
However, the approach exemplified in this study limits our thinking about how users read during an interactive experience. We learn only what users quickly find, read, and memorize on command. We do not learn
And that’s just for starters. So, I think this study is overdue to have its slack tightened. And we’re well overdue to elevate our understanding of interactive reading, which will breathe new life into our content and design choices.
Originally published on the now-archived Content Science blog in January 2012.
Make better content decisions with a system of data + insight.
Your content approach makes or breaks your digital transformation. Learn why intelligent content strategy + engineering are critical to your success.
Your content is integral to your product. You might have piloted content strategy and seen promising results. Now what? It’s time to get more strategic so you can sustain and scale. This whitepaper will help you start.
Does your content work? It's a simple question, but getting a clear answer from content analytics or ROI formulas is often anything but easy. This ebook by Colleen Jones will help you overcome the challenges.