Kaleida launches The Attention Index, an open-source algorithm to measure the impact of stories

Credit: Image from Kaleida

How does the prominence editors and news organisations assign to stories on the homepage compare to what readers actually pay attention to and engage with?

To find out, media research company Kaleida is launching The Attention Index today (21 June), an open-source algorithm aimed at measuring the impact and quality of news articles.

The Attention Index uses data collected by Kaleida using the company's own content analysis tools as well as third party platforms, such as APIs from Google which help identify the subject of an article, and information from a service called Aylien which performs sentiment analysis on stories.

On the publisher side, the data taken into account to measure attention includes an article's headline and URL, as well as the editorial emphasis publishers give to stories, for example whether an article is promoted as a featured story on the homepage and for how long it holds that position for.

"That's a strong indicator the publisher thinks it's an important story, but it might not necessarily be something the public finds interesting," Matt McAlister, chief executive of Kaleida, told Journalism.co.uk.

On the news consumer side, the index collects data based on the public's response, such as their engagement with stories on Facebook. It includes all the metrics provided by the platform, such as likes and shares, which are rolled up into a single number and input into the algorithm.

The data from both the publishers and the public's response is run through logarithmic scaling, McAlister explained, which puts everything into a comparative ranking system which allows editors to see how their organisation compares to others.

"The clearest thing you can do at the moment is to compare articles between themselves, so if you collect a bunch of stories on Trump, or the French election, and put them into groups, you can see how much attention is being given to a particular subject and compare it with other topics."

Since its launch in February, Kaleida has focused on collecting data from publishers to help them determine the topics of interest to their readers, and to identify news consumption patterns and predict the performance of their stories.

The Attention Index is published under a Creative Commons license with support from Google's Digital News Initiative (DNI) fund. The team has made the raw data collected from the month of May publicly available, allowing users to explore 128,000 article headlines, URLs, editorial promotion data, social data and attention scores, as well as the methodology behind the index.

Publisher data and attention scores will continue to be made available on a monthly basis and journalists can join a Google group to provide feedback on the tool and help develop it.

"What we are trying to achieve by open-sourcing [the index] is creating a standard in the market for valuing media. The first step is to show our work and the math behind it, and put real data out there," McAlister said.

"Initially it will be useful to rank publishers and their coverage but we think there is a lot more that can be done with it over time – it could become a standard for value in the same way ComScore is a standard for reach."

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).