I am an editor: what can AI do for me?

The Globe and Mail, the largest newspaper in Canada, has built a tool that helps editors get their priorities right. The machine learning technology, called Sophi, advises on decisions about what audience will value and how to design the homepage.

Like many newsrooms, explains editor-in-chief David Walmsley, the paper was looking to solve the problem of ideas living or dying based on whether an editor buys into them. Although hunches are sometimes right, this also leads to many missed opportunities.

So the publisher progressively hired an army of data scientists and data engineers, who now make up about 10 per cent of staff, and refined the smart tool that can predict audience behaviour.

The team identified around 20 metrics that fit The Globe and Mail’s business model, such as reader’s propensity to pay, location, time of the day they access content or frequently visited topics. The tool also performs A/B testing of headlines every 10 minutes and A/B testing of the homepage design, optimising them for user engagement as it goes.

[Read: I am a community journalist: what can AI do for me?]

One of Sophi's most interesting skills is deciding when to ask a reader for payment. For instance, someone who is interested in markets and investing will be more likely to take a subscription than someone looking up general news and, say, recipes. But if a reader does not fork out money right away, the tool will not pester them with the same message over and over again but may ask them to register with an email instead.

Sophi can come up with other interesting data-based hunches. For instance, after studying audience behaviour, it moved horoscopes an inch up on the homepage. You would assume no one would pay for news because of horoscopes but this small change is now bringing in millions of dollars.

Another great help journalists can get from Sophi is defining when a story is ready to be published instead of working to a deadline. After all, with the 24/7 news cycle, a deadline makes less sense than the time of the day the story is published. As Canada spans over six timezones, The Globe and Mail has the same number of breakfast editions and Sophi can determine at what time which story should be published.

This data-informed approach has led the editorial team to discover several surprising facts about the audience's interests. Politics does not trend on Wednesday afternoons during House of Commons debates but it does on Saturdays at 11 am, so that is when the coverage is published. This also helps with work-life balance as journalists do not have to work ungodly hours to file the story if the best time to publish it may be three days later.

The tool also showed the editorial team that when they publish investigations, the public is as interested in how the story came together as they are in the results. The investigative reporters now write popular pieces about methodology, which is something that they were not doing before.

The algorithm also identified that people want more opinion content, so the publication launched a print opinion section on Saturday that now yields important advertising revenue. Speaking of Saturdays, it also transpired that this is when people are most likely to read the news. When the news site decided to publish more content on the weekend, audience engagement increased by 54 per cent in three weeks.

Other than the satisfaction of fulfilling the audience's needs, Sophi significantly contributes to the bottom line. In simple terms, it puts a score on each piece of journalism that shows its commercial value in the acquisition of paying readers or their retention. This score is not secret - everyone in the newsroom can see how they and their colleagues perform.

Fear not, the score is never down to one individual, it is a collective effort. So rather than singling out journalists and putting them in the dog house, the score helps them decide what they can stop doing as some content is not worth their while.

AI-powered editorial decision-making can be a lot for some to stomach. If a machine knows better than a human editor what, where and when to publish, does that make the human redundant?

"Absolutely not," says Walmsley. Although it took about six years to train the algorithm, it can always be overridden. That is very important when publishing, for instance, an investigation that the tool has never seen, or information about the pandemic that has not happened before.

Humans also make decisions about which content to put out for free no matter what the computer thinks. For instance, when wildfires were raging in Western Canada, the news site put evacuation information out for free, even removing ads from the homepage to make sure they do not hamper loading times. This practice often has a halo effect; when the audience sees that public interest journalism is put in front of any paywall, they decide to take up a subscription to support the organisation.

[Read: I am a local journalist: what can AI do for me?]

Every year, the editor-in-chief looks at Sophi scores and seeks patterns to understand which content audiences value the most. Consistently, the most important work is staff work - about 60 per cent - and then come the wires, press releases and other types of content. Assessing tens of thousands of articles The Globe and Mail publishes every year, it also becomes clear that the articles at the bottom of the pile are generating so little engagement it is probably not worth doing them at all.

Editorial skills and critical thinking are still needed for making sure readers get a balanced news diet. For instance, there was a time when articles about Donald Trump was an audience magnet and the paper could not produce enough of it to satisfy the demand. But editors stepped in and decided - contrary to the algorithm's recommendations - to publish less Trump-related content as it was crowding out other important topics and it was not healthy for the readers.

So this is what the process looks like in practice: a journalist uploads an article to the CMS and sets the time for publication or lets the algorithm decide based on previous publication patterns. If they agree with the recommendation, they confirm and leave. If not, they override it manually, no questions asked.

Letting an algorithm decide what should and should not go on the homepage has one important advantage: it removes promotional bias. In every newsroom, the most prominent reporters sometimes monopolise the front page but with Sophi, the latest junior recruit can get the same exposure as the star writer if their content is valuable to the audience. In a newsroom with some 250 staff on the journalism side, that can make a real difference in career growth.

"Journalists’ work didn’t really change," says Walmsley. For many staffers, it fosters a sense of pride when they see they are bringing new subscribers. But perhaps the biggest gift this AI-powered tool gives them is advice on what to start, stop and continue doing.

This series is supported by United Robots and Utopia Analytics. Neither of them is involved in the editorial process at any stage.

United Robots AB is a Swedish technology company working in automated editorial content. The company leverages structured data to provide publishers with automatically generated content about sports, real estate, traffic, weather, local businesses and the stock market.

Utopia Analytics is a Finnish company that enables automated moderation of reader comments and cuts down the publishing delay. Inappropriate behaviour, bullying, hate speech, discrimination, sexual harassment and spam are filtered out 24/7 so teams can focus on moderation policy management.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).