Could crowdsourcing help you fact-check your data?

From Africa to the Baltic States to the plains of South America, public figures try to sneak dodgy claims under the radar of journalists.

Three fact-checking organisations spoke at the Poynter Global Fact-Checking Summit on Monday (9 June) about how they are now trying to change that culture by asking their readers ‘what claims do they think are most in need of being checked out?’.
 
Africa Check

Julian Rademeyer, editor of Africa Check, explained that as Africa is a notoriously hard region to get reliable data for, fact-checking has never been more important there, and that is why they established an African site in the first place.
 
“Take data from the World Health Organisation for example," Rademeyer said, "they rely on individual countries submitting their own data – but there are huge differences between the reliability of the data some African countries submit compared to others.”

Africa Check began crowdsourcing claims for fact-checking to widen their coverage of the region, but found gathering the right information through crowd sourcing a challenge at first.
 
“Many of our freelancers do struggle with the idea of fact-checking opinions," he continued. "There’s a mental shift from reporting to fact-checking and a lot of the initial statements we were getting from people were opinions, rather than facts – we needed to guide people.”
 
They now have a simple guide as to what criteria a submitted claim must meet to be considered for further investigation.
 
A claim must purport to be a fact, be from somebody Africa Check have not featured widely before, and be about one of their three key topic areas: society, politics and justice; the economy; or health, science and the environment.

Moreover, the person submitting a claim has to make a case for why it should be investigated – “What impact would the statement have if it goes unchecked? The greater the impact, the more likely we are to check the comment suggested.”

Crowdsourcing does not equal public opinion
 
The Africa Check team accept that crowdsourcing should not be the only source of the claims they check though.
 
“It’s easy to think you’ve got 50 responses to this, this must the public’s opinion," said Peter Cunliffe-Jones, director of Africa Check, "but no, you’ve got 50 obsessives.
 
“We did one report on race in South Africa and at least half of our 40 comments were negative. We deleted the obviously racist ones but our computer guy who saw all the comments couldn’t believe what a negative reaction we had received. But then he only saw the website, not the 3000 likes on Facebook.

“It’s important that when you interact with a crowd you’ve got to have some knowledge of who is in the crowd.”
 
Become a fact checker with FactCheckEU

Pietro Curatolo, co-founder and co-editor of FactCheckEU, said their site is different as it gives people the choice to submit links to the original sources of a claim or to submit a whole fact check.

“We’ve had a good response from people in the crowd," he said, "there are some regulars, as Africa Check also found out, there are actually about 10 people or so who have submitted multiple fact-checks to us.
 
“We have many more who submit claims to be checked which is very useful, especially in Europe where there are so many languages and countries it’s difficult to monitor them well all the time. But people in the Baltic states for example, can now send in particular statements they want verified – ones that are important to them.”
 
At FactCheckEU a whole fact-checking community has been created, with an incentive and a quality control measure, users rated on the quality of their fact-checks and ranked against other members.

Like with Africa Check, Curatolo said crowdsourcing requires more work at the beginning from the established team, with all fact-checks by users going through a peer review, and then being clearly marked as reader generated material.
It does pay off in the long run though.

“There is a learning curve from when people first submit fact-checks, but as they become better it’s less work for us, and very useful for a small team,” Curatolo said.
 
Chequeado – fact-checking the Argentinain President’s speech in real-time
 
Chequeado, a South American fact-checking organisation, have found crowdsourcing really useful for larger projects. When a team of 40 were organised to fact-check President Christina Fernández de Kirchner’s speech in real time, they had to verify everything from how big the countries natural gas reserves were to how much Coca-Cola the population drinks.
 
The team also regularly ask for data on certain subjects via Twitter using the hashtag #DatoCHQ.
 
Laura Zommer, executive director of Chequeado, said: “We ask for data on certain subjects when we’re not sure what the best source of the data is, but we also accept all the other data that is sent to us via Twitter.
 
“We then verify the reliability of the source of the information and publish the data as a link with a description of the information it contains.”
 
Could crowdsourcing help with your fact-checking?

Zommer also highlighted the pros and cons of crowdsourcing. Their use of Twitter meant people can answer instantly and by widening the type of contributions people can make a greater volume of work can be done. By nurturing users to become full fact checkers, FactCheckEU developed a support network who now have the skills to gather evidence to verify claims.
 
Naturally, crowdsourcing can result in reaching news sources and people you may never have contacted otherwise and increases the transparency and reliability of an organisation, opening up the discussion and providing more eyes on the data.
 
However, setting up a crowdsourcing platform to help with fact-checking does require dedication, and that might be one of the reasons only four out of 48 fact-checking organisations identified as active worldwide have included prominent crowdsourcing features on their sites.

There needs to be a procedure in place to ensure quality control, and it is likely that it be hard to get other people to work for you, but if your crowdsourcing platform does take off you are then still liable to having your crowd manipulated by partisan organisations.
 
Zommer said sometimes you ultimately end up working with just a few people, or even one person – then this raises a further ethical dilemma – “can you still present your work as crowdsourced?”

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).