Content Science has launched the ContentWRX Index, the first ever index of content effectiveness. I’ve been waiting for more than a year for this day. (That’s a long time in Internet years.) So many people have put in so much work to make this happen. I can’t adequately convey my excitement about finally offering the ContentWRX Index as a resource to advance content practice.

For each ContentWRX Index, we’re publishing an infographic, an article, and an in-depth report here at Content Science Review. (You can see the first in the series, Nonprofit, here.) So, I thought I’d take some time to explain what this index is all about.

Why an Index for Content?

I noticed that our professional world offers indices for customer satisfaction, for digital, for trust, for global technology adoption, and much more. I’m only scratching the surface. When we care about the performance of a function or direction of a trend, we tend to create an index to monitor that performance. It should be no different for content. So, I thought, how can we say content has business value if we can’t monitor its impact in some way? How can we ask our organizations to invest thousands, hundreds of thousands, and even millions of dollars in changing their content approach and, yet, not be bothered to assess the collective current state of content? It’s time to put our money, our effort, and our science where our mouth is. Enter the ContentWRX Index.

Why Focus the Index on Content Effectiveness?

At Content Science, we find the real value of content for an organization is its ability to achieve goals–its effectiveness. If content is highly effective with content users, then usually it brings a return on investment (ROI). Content effectiveness is a strong indicator of ROI.

Also, compared to evaluating content ROI alone, I find assessing effectiveness leads to better overall content intelligence for an organization, which brings more repeatable value. The lessons learned from assessing whether your content is effective can help you with improving your current approach, developing new content ideas, solving problems (content problems and other organizational problems), and much more.

So, we thought, if assessing content effectiveness at the organizational level is so useful, wouldn’t assessing content effectiveness at the industry level be even more so? Wouldn’t it be exciting to improve our collective content intelligence and, as a result, advance content practice?

(All that doesn’t mean content ROI isn’t important…I talk often about ROI …but ROI is more useful to focus on at an organizational level, not an index level.)

How Does the ContentWRX Index Benefit Me?

The ContentWRX Index benefits you in many ways. Here are a few immediate ones.

  • Credibility – Regardless of your role or title, if content matters to you, the fact this index exists adds credibility to your argument that content is important and merits strategy, time, and resources.
  • Industry Benchmarking – You can see how a particular industry performs overall and in specific dimensions such as content accuracy and content influence. (See the next two sections for more about what the index measures and how.) So, if you work for that industry or have a client within that industry, you can see typical problem areas.
  • Data-Informed Ideas + Decisions – The ContentWRX Index and their industry reports are a resource you can use to:
    • Give your clients sound advice.
    • Educate your team(s) on content best practices.
    • Spark ideas for content initiatives or client pitches.
    • Make the case for improving or changing your content approach.

What Does the ContentWRX Index Measure?

The index uses our ContentWRX software platform to measure the effectiveness of a representative sample of content from industry websites in six dimensions including: Discovery / Findability, Accuracy, Polish, Relevancy, Usefulness, and Influence.

contentwrx index

We apply an algorithm to generate an overall score of effectiveness for the industry as well as each website. We then conduct detailed analysis within each dimension to identify what’s working well and what the possible opportunities to improve are.

What’s the Science Behind the ContentWRX Index?

Our approach is the culmination of years of independent research and experimentation with the ContentWRX platform. Let me share a quick summary.

Public Website Focus
Our ContentWRX platform can evaluate a variety of digital experiences. However, for the index, we focus on content available on organization websites, not mobile applications, social channels, or email, because

  • Organizations have a high degree of control over that content.
  • The content is publicly available to anyone using the Internet.
  • That content comprises the majority of the Internet.

Sample Size
In deciding on sample size for the index, we balanced two key considerations: the level of confidence in the data and the feasibility of both collecting and analyzing the data in a timely way. In other words, we tried to narrow the sample size to the smallest number possible and still get valid, useful results.

  • Website Sample Size: 10 Per Industry
    For each industry, we focus on 10 websites. We select the sample based on a combination of critical case sampling, typical case sampling, and extreme case sampling.
  • User Sample Size: 12 Per Website
    For each website selected within each industry, we collect data from 12 users. The only exception is for our first index, ContentWRX Index: Nonprofit, where collected data from 50 users per website. See the next question for more details about how we arrived at 12 users.

Data Collection Method
For each industry, we select a task pertinent to the content, recruit users remotely to accomplish the task, and then ask users to complete a survey of questions addressing the six dimensions of content effectiveness. The survey questions went through a series of validation testing with more than 1600 users as part of our Content + Credibility Study. That means we validated that the questions measure what we intended. Users understand the questions and provide answers that make sense.

Content Type Selection
A website is comprised of content that serves many different purposes. Of course, it’s not possible to evaluate all of that content for each website for our index. So, we choose a specific type of content to focus on within each industry. For example, for our ContentWRX Index: Nonprofit, we focused on thought leadership content, or what we call Expertise. Here are the most common content types we assess for public websites, and we choose the most appropriate one for the industry.

The Most Common Content Types We Assess For Public Websites

Content TypeDescriptionExample
ExpertiseThought leadership, advice, and similar contentBlog, digital magazine, or whitepaper providing insights or tips
SalesContent explaining options, products, or offerings leading to a saleProduct descriptions, solution descriptions
SupportContent instructing users in technical processes, troubleshoot problems, or get answers to technical questionsSupport knowledge base, instruction manual
ServiceContent helping users understand policies, manage their accounts, return or exchange products, and moreReturn and exchange policies, account profiles

(Note that our methodology for the ContentWRX Index is based on but different from our ContentWRX platform. See How Does the ContentWRX Index Differ from the Platform?)

How Did You Arrive at a Sample Size of 120 Users Per Industry?

We conducted an iterative analysis. This is cool, so I’m going to walk you through the main steps. (I love the power of science.)

  1. DETERMINE DISTRIBUTION – We analyzed a large sample (2,258 users) of our ContentWRX client data to determine its distribution. We also found that the data followed a slightly negative skew. This type of skew is common. For example, many (if not most) economic data sets have a negative skew.
    contentwrx index data
    The distribution of our ContentWRX client data set has a slightly negative skew.

     

  2. DETERMINE CONSISTENCY – We then collected data from a sample of 500 users (50 websites) for one industry (nonprofit). We analyzed that data’s distribution and compared it to our client data to determine whether the distribution was consistent. The great news? Yes, both data sets followed fairly consistent distribution.
  3. DETERMINE MINIMUM SAMPLE – We used bootstrapping analysis to correct for the skew and determine the minimum number of users needed to achieve a reasonable confidence interval. The answer? 12 per website to achieve a margin of error of +/- 16%.

    contentwrx index
    Our analysis determined 12 users per website would give us a reasonable margin of error.

We selected +/- 16% because, similar to quantitative usability testing, we find that margin allows us to obtain a wealth of useful insight for the index in a cost-effective way. And, as Nielsen notes, at least 90% of the time the results will not fall in the margin of error. That’s more than reasonable for a non-academic effort.

What Industries Will the ContentWRX Index Cover?

The ContentWRX Index will span a variety of industries, including but not limited to these.

IndustryFirst Year Availability
Nonprofit (Cause-Based)July 2015
IT Services and SoftwareAugust 2015
HealthOctober 2015
RetailDecember 2015
News + MediaMay 2016
Financial ServicesNovember 2016

How Often Will the ContentWRX Index Publish?

For each industry, we will conduct the analysis and publish the results annually.

In What Formats Will the ContentWRX Index Publish?

For each industry, we will publish:

  • An infographic summarizing the score and key insights
  • 1-2 articles based on a key finding
  • An in-depth report (available to subscribers) that provides
    • Detailed analysis.
    • Explanation of top findings.
    • Recommendations for applying content best practices.

Can I Be Alerted When the ContentWRX Index Publishes?

Yes, if you register with or subscribe to Content Science Review, you will:

  • Be signed up for our monthly email update.
  • Have the option to follow the topic such as content evaluation. When you follow that topic, you’ll see each report for the ContentWRX Index show up in your content feeds.

I Believe in What You’re Doing. What Can I Do to Help Move the ContentWRX Index Forward?

There’s so much you can do to advance the ContentWRX Index and, as a result, content practice. The easiest and quickest way is to help us raise awareness of the index.

  • Share the ContentWRX Index infographics, articles, and reports with others.
  • Add to the conversation by commenting on the infographics and articles.
  • Cite findings from the ContentWRX Index in your presentations and reports.
  • Let us know how the ContentWRX Index has helped you, your employer, or a client.

Another way you can advance the cause is to sponsor a ContentWRX Index study – see the next question below. And, still another way you can help is to let us know what you think about the in-depth reports so we can improve them.

I’m Interested in Applying Your Methodology to a Different Industry or to Something Else Entirely. How Can We Work with You?

There are many possibilities for applying the ContentWRX Index approach. We can work with you in one of three ways.

  • Sponsored Study – If you are a company who wants to support the cause and make the results freely available, then we can collaborate with you on a sponsored study and credit your involvement.
  • Custom Private Study – If you are a company who is interested in conducting a specific study to advance your own content strategy, then you can commission us to conduct private research.
  • Academic Collaboration – If you are an academic institution or nonprofit that has funding to conduct research pertinent to content, communication, user experience, or similar topics, we can collaborate with you on a study.


Contact us to learn more and discuss the best approach.

How Does the ContentWRX Index Differ from the ContentWRX Platform?

The ContentWRX Index is a study methodology using the ContentWRX software platform. So, the ContentWRX platform offers a range of options for studying content. Here’s a breakdown of the key differences.

 ContentWRX IndexContentWRX Platform
User Sample Size12 Per Website / 120 Per IndexUp to 1000 Per Month
Data CollectedSurvey ResponsesSurvey Responses
Behavioral Analytics
Scope of Discovery DimensionOn-Site Findability
Whether users can find content within the site using navigation or search
On-Site Findability
Off-Site Findability, such as
● Visibility in search engines
● Social mentions
● Referrals by experts or colleagues
Types of Digital ExperiencesPublic WebsitesPublic Websites
Websites Requiring Log-In
Intranets
Extranets
Mobile Applications

So, there you have the nitty gritty of the ContentWRX Index. I’m excited about the lessons we’ve learned so far, and I have no doubt the index will help us all become even more intelligent about content in the future.

The Author

Colleen Jones is the founder and CEO of Content Science, a growing content intelligence and strategy company based in Atlanta GA. Content Science owns Content Science Review, Content Science Academy, and the content effectiveness software ContentWRX.  Colleen regularly consults with executives and practitioners to improve their strategy and processes for content. She shares insights and guidance from her experience regularly on Content Science Review, at events around the world, and in highly rated books such as Clout: The Art and Science of Influential Web Content.

Follow Colleen on Twitter at @leenjones or on LinkedIn.

This article is about

Comments

Leave a Reply

4 Comments on "The Rhyme and Reason Behind the ContentWRX Index (CWI)"


Guest
ContentWRX Index: Nonprofit Infographic
1 year 5 months ago

[…] than 500 participants. ask md Read more about the rhyme and reason behind the ContentWRX indexes, here. This nonprofit index infographic gives a visual snapshot of the report and several of its key […]

Guest
ContentWRX Index: IT Services and Software Infographic
1 year 7 months ago

[…] from 120 participants. Read more about the rhyme and reason behind the ContentWRX indexes, here. Our infographic gives a visual snapshot of the index and several of its key […]

Guest
ContentWRX Featured as Innovation Sponsor at CMWorld2015
1 year 8 months ago

[…] than ever. It also recently released the first ever index of content effectiveness. The ContentWRX Index series is a valuable resource for executives and practitioners to benchmark content effectiveness […]

Guest
ContentWRX Releases the First Ever Index of Content Effectiveness
1 year 9 months ago

[…] The ContentWRX Index series is a valuable resource for executives and practitioners to benchmark content effectiveness for their industry. The ContentWRX Index adds credibility to the argument that content is important and merits strategy, time, and resources. The in-depth index is useful for educating team(s) on content best practices, sparking ideas for content initiatives or client pitches, and making the case for improving or changing content. In addition to the nonprofit cause-based index, ContentWRX will also publish indexes on technology, health, financial services, nonprofit association, travel, retail/e-commerce, and news media throughout 2015 and 2016. You can read more about the series, here. […]

 
COMMENT GUIDELINES

We invite you to share your perspective in a constructive way. To comment, please sign in or register. Our moderating team will review all comments and may edit them for clarity. Our team also may delete comments that are off-topic or disrespectful. All postings become the property of
Content Science Review.

Partner Whitepapers

Digital Transformation for Marketing

Your content approach makes or breaks your digital transformation. Learn why intelligent content strategy + engineering are critical to your success.

Content Strategy for Products + Services

Your content is integral to your product. You might have piloted content strategy and seen promising results. Now what? It’s time to get more strategic so you can sustain and scale. This whitepaper will help you start.

Help with Content Analytics + ROI

Does your content work? It's a simple question, but getting a clear answer from content analytics or ROI formulas is often anything but easy. This ebook by Colleen Jones will help you overcome the challenges.

Content Evaluation Made Easier

Frustrated by content evaluation? This whitepaper explains an approach and a tool, ContentWRX, to make evaluating content easier.

SEE ALL