Content Science has launched the ContentWRX Index, the first ever index of content effectiveness. I’ve been waiting for more than a year for this day. (That’s a long time in Internet years.) So many people have put in so much work to make this happen. I can’t adequately convey my excitement about finally offering the ContentWRX Index as a resource to advance content practice.

For each ContentWRX Index, we’re publishing an infographic, an article, and an in-depth report here at Content Science Review. (You can see the first in the series, Nonprofit, here.) So, I thought I’d take some time to explain what this index is all about.

Why an Index for Content?

I noticed that our professional world offers indices for customer satisfaction, for digital, for trust, for global technology adoption, and much more. I’m only scratching the surface. When we care about the performance of a function or direction of a trend, we tend to create an index to monitor that performance. It should be no different for content. So, I thought, how can we say content has business value if we can’t monitor its impact in some way? How can we ask our organizations to invest thousands, hundreds of thousands, and even millions of dollars in changing their content approach and, yet, not be bothered to assess the collective current state of content? It’s time to put our money, our effort, and our science where our mouth is. Enter the ContentWRX Index.

Why Focus the Index on Content Effectiveness?

At Content Science, we find the real value of content for an organization is its ability to achieve goals–its effectiveness. If content is highly effective with content users, then usually it brings a return on investment (ROI). Content effectiveness is a strong indicator of ROI.

Also, compared to evaluating content ROI alone, I find assessing effectiveness leads to better overall content intelligence for an organization, which brings more repeatable value. The lessons learned from assessing whether your content is effective can help you with improving your current approach, developing new content ideas, solving problems (content problems and other organizational problems), and much more.

So, we thought, if assessing content effectiveness at the organizational level is so useful, wouldn’t assessing content effectiveness at the industry level be even more so? Wouldn’t it be exciting to improve our collective content intelligence and, as a result, advance content practice?

(All that doesn’t mean content ROI isn’t important…I talk often about ROI …but ROI is more useful to focus on at an organizational level, not an index level.)

How Does the ContentWRX Index Benefit Me?

The ContentWRX Index benefits you in many ways. Here are a few immediate ones.

  • Credibility – Regardless of your role or title, if content matters to you, the fact this index exists adds credibility to your argument that content is important and merits strategy, time, and resources.
  • Industry Benchmarking – You can see how a particular industry performs overall and in specific dimensions such as content accuracy and content influence. (See the next two sections for more about what the index measures and how.) So, if you work for that industry or have a client within that industry, you can see typical problem areas.
  • Data-Informed Ideas + Decisions – The ContentWRX Index and their industry reports are a resource you can use to:
    • Give your clients sound advice.
    • Educate your team(s) on content best practices.
    • Spark ideas for content initiatives or client pitches.
    • Make the case for improving or changing your content approach.

What Does the ContentWRX Index Measure?

The index uses our ContentWRX software platform to measure the effectiveness of a representative sample of content from industry websites in six dimensions including: Discovery / Findability, Accuracy, Polish, Relevancy, Usefulness, and Influence.

contentwrx index

We apply an algorithm to generate an overall score of effectiveness for the industry as well as each website. We then conduct detailed analysis within each dimension to identify what’s working well and what the possible opportunities to improve are.

What’s the Science Behind the ContentWRX Index?

Our approach is the culmination of years of independent research and experimentation with the ContentWRX platform. Let me share a quick summary.

Public Website Focus
Our ContentWRX platform can evaluate a variety of digital experiences. However, for the index, we focus on content available on organization websites, not mobile applications, social channels, or email, because

  • Organizations have a high degree of control over that content.
  • The content is publicly available to anyone using the Internet.
  • That content comprises the majority of the Internet.

Sample Size
In deciding on sample size for the index, we balanced two key considerations: the level of confidence in the data and the feasibility of both collecting and analyzing the data in a timely way. In other words, we tried to narrow the sample size to the smallest number possible and still get valid, useful results.

  • Website Sample Size: 10 Per Industry
    For each industry, we focus on 10 websites. We select the sample based on a combination of critical case sampling, typical case sampling, and extreme case sampling.
  • User Sample Size: 12 Per Website
    For each website selected within each industry, we collect data from 12 users. The only exception is for our first index, ContentWRX Index: Nonprofit, where collected data from 50 users per website. See the next question for more details about how we arrived at 12 users.

Data Collection Method
For each industry, we select a task pertinent to the content, recruit users remotely to accomplish the task, and then ask users to complete a survey of questions addressing the six dimensions of content effectiveness. The survey questions went through a series of validation testing with more than 1600 users as part of our Content + Credibility Study. That means we validated that the questions measure what we intended. Users understand the questions and provide answers that make sense.

Content Type Selection
A website is comprised of content that serves many different purposes. Of course, it’s not possible to evaluate all of that content for each website for our index. So, we choose a specific type of content to focus on within each industry. For example, for our ContentWRX Index: Nonprofit, we focused on thought leadership content, or what we call Expertise. Here are the most common content types we assess for public websites, and we choose the most appropriate one for the industry.

The Most Common Content Types We Assess For Public Websites

Content TypeDescriptionExample
ExpertiseThought leadership, advice, and similar contentBlog, digital magazine, or whitepaper providing insights or tips
SalesContent explaining options, products, or offerings leading to a saleProduct descriptions, solution descriptions
SupportContent instructing users in technical processes, troubleshoot problems, or get answers to technical questionsSupport knowledge base, instruction manual
ServiceContent helping users understand policies, manage their accounts, return or exchange products, and moreReturn and exchange policies, account profiles

(Note that our methodology for the ContentWRX Index is based on but different from our ContentWRX platform. See How Does the ContentWRX Index Differ from the Platform?)

How Did You Arrive at a Sample Size of 120 Users Per Industry?

We conducted an iterative analysis. This is cool, so I’m going to walk you through the main steps. (I love the power of science.)

  1. DETERMINE DISTRIBUTION – We analyzed a large sample (2,258 users) of our ContentWRX client data to determine its distribution. We also found that the data followed a slightly negative skew. This type of skew is common. For example, many (if not most) economic data sets have a negative skew.
    contentwrx index data
    The distribution of our ContentWRX client data set has a slightly negative skew.


  2. DETERMINE CONSISTENCY – We then collected data from a sample of 500 users (50 websites) for one industry (nonprofit). We analyzed that data’s distribution and compared it to our client data to determine whether the distribution was consistent. The great news? Yes, both data sets followed fairly consistent distribution.
  3. DETERMINE MINIMUM SAMPLE – We used bootstrapping analysis to correct for the skew and determine the minimum number of users needed to achieve a reasonable confidence interval. The answer? 12 per website to achieve a margin of error of +/- 16%.

    contentwrx index
    Our analysis determined 12 users per website would give us a reasonable margin of error.

We selected +/- 16% because, similar to quantitative usability testing, we find that margin allows us to obtain a wealth of useful insight for the index in a cost-effective way. And, as Nielsen notes, at least 90% of the time the results will not fall in the margin of error. That’s more than reasonable for a non-academic effort.

What Industries Will the ContentWRX Index Cover?

The ContentWRX Index will span a variety of industries, including but not limited to these.

IndustryFirst Year Availability
Nonprofit (Cause-Based)July 2015
IT Services and SoftwareAugust 2015
HealthOctober 2015
RetailDecember 2015
News + MediaMay 2016
Financial ServicesNovember 2016

How Often Will the ContentWRX Index Publish?

For each industry, we will conduct the analysis and publish the results annually.

In What Formats Will the ContentWRX Index Publish?

For each industry, we will publish:

  • An infographic summarizing the score and key insights
  • 1-2 articles based on a key finding
  • An in-depth report (available to subscribers) that provides
    • Detailed analysis.
    • Explanation of top findings.
    • Recommendations for applying content best practices.

Can I Be Alerted When the ContentWRX Index Publishes?

Yes, if you register with or subscribe to Content Science Review, you will:

  • Be signed up for our monthly email update.
  • Have the option to follow the topic such as content evaluation. When you follow that topic, you’ll see each report for the ContentWRX Index show up in your content feeds.

I Believe in What You’re Doing. What Can I Do to Help Move the ContentWRX Index Forward?

There’s so much you can do to advance the ContentWRX Index and, as a result, content practice. The easiest and quickest way is to help us raise awareness of the index.

  • Share the ContentWRX Index infographics, articles, and reports with others.
  • Add to the conversation by commenting on the infographics and articles.
  • Cite findings from the ContentWRX Index in your presentations and reports.
  • Let us know how the ContentWRX Index has helped you, your employer, or a client.

Another way you can advance the cause is to sponsor a ContentWRX Index study – see the next question below. And, still another way you can help is to let us know what you think about the in-depth reports so we can improve them.

I’m Interested in Applying Your Methodology to a Different Industry or to Something Else Entirely. How Can We Work with You?

There are many possibilities for applying the ContentWRX Index approach. We can work with you in one of three ways.

  • Sponsored Study – If you are a company who wants to support the cause and make the results freely available, then we can collaborate with you on a sponsored study and credit your involvement.
  • Custom Private Study – If you are a company who is interested in conducting a specific study to advance your own content strategy, then you can commission us to conduct private research.
  • Academic Collaboration – If you are an academic institution or nonprofit that has funding to conduct research pertinent to content, communication, user experience, or similar topics, we can collaborate with you on a study.

Contact us to learn more and discuss the best approach.

How Does the ContentWRX Index Differ from the ContentWRX Platform?

The ContentWRX Index is a study methodology using the ContentWRX software platform. So, the ContentWRX platform offers a range of options for studying content. Here’s a breakdown of the key differences.

 ContentWRX IndexContentWRX Platform
User Sample Size12 Per Website / 120 Per IndexUp to 1000 Per Month
Data CollectedSurvey ResponsesSurvey Responses
Behavioral Analytics
Scope of Discovery DimensionOn-Site Findability
Whether users can find content within the site using navigation or search
On-Site Findability
Off-Site Findability, such as
● Visibility in search engines
● Social mentions
● Referrals by experts or colleagues
Types of Digital ExperiencesPublic WebsitesPublic Websites
Websites Requiring Log-In
Mobile Applications

So, there you have the nitty gritty of the ContentWRX Index. I’m excited about the lessons we’ve learned so far, and I have no doubt the index will help us all become even more intelligent about content in the future.

The Author

Colleen Jones is the author of The Content Advantage and founder of Content Science, a content intelligence and strategy firm that has advised or trained hundreds of the world’s leading organizations since 2010. She also is the former head of content at MailChimp, the marketing platform recognized by Inc. as 2017 Company of the Year. A passionate entrepreneur, Colleen has led Content Science to develop the  content intelligence software ContentWRX, publish the online magazine Content Science Review, and offer online certifications through Content Science Academy.

Colleen has earned recognition as an instructor on LinkedIn Learning, one of the Top 50 Most Influential Women in Content Marketing by a TopRank study, a Content Change Agent by Society of Technical Communication’s Intercom Magazine, and one of the Top 50 Most Influential Content Strategists by multiple organizations.

Follow Colleen on Twitter at @leenjones or on LinkedIn.

This article is about
#comments -->

We invite you to share your perspective in a constructive way. To comment, please sign in or register. Our moderating team will review all comments and may edit them for clarity. Our team also may delete comments that are off-topic or disrespectful. All postings become the property of
Content Science Review.