Die Welt's in-house analytics tool gives articles a score to measure their impact online

One and a half years ago, German daily newspaper Die Welt decided to change its approach to measuring the performance of its stories online, from focusing on the number of article visits to a model that took into account data from multiple sources.

The outlet, which is owned by Axel Springer, used to send people in the newsroom a list of the top 10 most clicked stories of the day before developing its in-house analytics tool called Article Score.

Die Welt reaches some 15.7 million online readers every month, of which 8.3 million read its articles on mobile, but the title only occupies fourth place in the list of top 10 German publishers, according to AGOF 2016 data cited by Kritsanarat Khunkham, the outlet's managing editor.

"One point we questioned is 'do we want to be number one in this chart?'" Khunkham said, speaking at the WAN-IFRA International Newsroom Summit in London yesterday (21 November).

"We do A/B testing with headlines and photos and we always try to find new ways to optimise content, but one thing we should think about is that not every click is a good click, and not every visitor is a useful visitor."

Article Score awards a score between 0 and 30 to individual stories, by aggregating data from different analytics platforms including Chartbeat, and is integrated into Die Welt's custom management system so it can be easily accessed by the editorial team.

We do A/B testing with headlines and photos and we always try to find new ways to optimise content, but one thing we should think about is that not every click is a good click, and not every visitor is a useful visitorKritsanarat Khunkham, Die Welt

"Looking at how far an article has travelled on social and taking into account the bounce rate, which helps us see determine the quality of a reader, has helped us be more transparent with our values."

The tool takes into account five different elements, which are all weighted differently. Number of visits to an article can have a maximum score of 10, while engagement time, percentage of social media traffic, the number of video views and the bounce rate can only get up to five points each.

Khunkham explained Article Score was built so that a lower number of visits to an article could be outnumbered by the other criteria included, the aim being to promote quality journalism and not "treat clicks as the only currency".

The tool conveys the data to journalists in a friendly and easy to understand way, while encouraging them to question what went right or wrong with their article.

For example, if a story ranks particularly low in the social media department, the editors might look at how often the article was shared on the main Twitter account or the journalist's own account, or whether the headline and image suit that particular piece on a certain platform.

This allows journalists to think about how they can improve their stories in order to keep readers engaged for longer, and it also helps answer some of the questions they might have around data and determining the success of a story.

"We want people to spend time with Die Welt, engage with our articles and maybe, down the road they will even find our work worth paying for, as two months ago we moved from a metered paywall to a freemium model," Khunkham explained.

"But to change, you need to make data an editorial tool, not just something that arrives in your inbox.

"We wanted to build something that gives editors clear indications instead of them having to go through a bunch of Excel files and see how their stories did compared to the previous week."

Using Article Score has changed Die Welt's approach to digital content and has improved engagement. The publisher has an average engagement time of 49.8 seconds per story, based on Chartbeat data cited by Khunkham, while the average engagement time for its competitors ranges between 15 and 45 seconds.

We wanted to build something that gives editors clear indications instead of them having to go through a bunch of Excel files and see how their stories did compared to the previous weekKritsanarat Khunkham, Die Welt

Die Welt readers are also most likely to read more articles when visiting the website and spend more time on the page, and the outlet's stories have longer lifespans on social media.

"The return rate across Chartbeat's network of publishers also shows that the longer a new visitor spends reading, the higher the probability they'll return to the site, so engaged readers matter more than unique visitors who may just come and go."

Since Article Score was developed, the newsroom has also experimented with measuring the performance of stories in real-time, and Khunkham said the tool could be further expanded to include criteria such as: number of comments, backlinks, engagement time compared to expected reading time, social shares versus social traffic, platform-specific measurements and metered versus freemium content.

"Our aim with Article Score was to present this data in a journalist-friendly way and encourage everyone to feel motivated, including people who work on quality stories that would otherwise not make it in the top 10 most clicked articles.

"We want journalists and editors to use this data to improve their content, not to produce click-bait," Khunkham said.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).