Who Said What tool brings artificial intelligence to fact-checking audio and video

Who Said What, a tool that will use artificial intelligence to identify quotes in audio and video segments in an effort to help journalists and fact-checkers verify information, received a $50,000 grant from the Knight Foundation in June.

Who Said What will be built to enable users to search either private collections or publicly available audio and video segments, in collaboration with the Internet Archive.

Journalists and fact-checkers will be able to search the clips to find out if a certain person really did say a particular quote circulating online, for example, or to identify the right person who said it.

Delip Rao, founder of AI research start-up Joostware and part of the team behind Who Said What, has been working on tackling the ‘fake news’ problem since 2016, when he noticed the proliferation of misinformation online during the US election campaign.

Who Said What aims to solve the problems of volume and scale that fact-checkers face today. As video and audio content has become easy to produce, the volume of information fact-checkers need to get through in order to find a particular quote they are looking for poses a great challenge.

"We have never taken a position that we can automate fact-checking, but our mission is simply to make sure that fact-checkers have the right sort of tools to make their workflow as streamlined as possible," said Rao.

"We are not automating fact-checking but we are building tools that would make fact-checkers a lot more efficient, or at least that is the goal.

"There aren’t a lot of tools for journalists and fact-checkers that are centred around audio and video content and there's so much of it that being able to find or tackle misinformation problems in audio and video content is an issue that fact-checkers are already facing and they don't have the right set of tools."

The money raised from the Knight Foundation grant will only partly cover the costs of developing and maintaining Who Said What, and Rao explained that the audio and video content analysis tool will also be used for business purposes by Joostware to bring in additional revenue. The goal is to keep it free or at least inexpensive for journalists and fact-checkers.

In collaboration with research scientist Dean Pomerleau, Rao has also been working on the Fake News Challenge, an initiative that brought together around 900 participants from the AI community, journalism and fact-checking organisations.

The first edition started up in November 2016 and finished in June 2017 with the aim of identifying a problem facing fact-checkers and its underlying technical issues, and come up with solutions that would be open-sourced and made available to users worldwide.

Participants looked at tackling the multitude of articles about the same claim, and identifying the stance each article took in regards to that claim (stance detection).

The second run of the Fake News Challenge is currently in the works and ideas can be submitted here.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).