The bureau, the whistleblower and the data journalist: how WikiLeaks' Iraq war logs made the news

When whistleblowing website WikiLeaks released 400,000 classified US military documents relating to the Iraq war on its website late Friday night, it was a collaborative effort by news organisations, journalists and the site hosting the information to communicate its impact to the public.

WikiLeaks stressed that this release, which follows the disclosure of 92,000 documents relating to the Afghanistan war back in July, should maximise media coverage and reach as many readers as possible for those sources risking their lives and possible incarceration by leaking the information.

The Guardian, Der Spiegel and the New York Times, who produced reports and data analysis of the Afghan war logs, were this time joined by Al Jazeera, Le Monde and Channel 4's Dispatches as well as non-profit media organisation OWNI and voluntary organisation Iraq Body Count.

OWNI designed WikiLeaks' site for the data, while the UK-based Bureau of Investigative Journalism was the driving force behind much of Al Jazeera, Le Monde and Channel 4's coverage. But how did they get involved and begin processing such a vast amount of information? Journalism.co.uk spoke to the bureau's editor Iain Overton and OWNI datajournalist Nicolas Kayser-Bril to find out more.

The Bureau of Investigative Journalism
When it launched in April, the Bureau of Investigative Journalism said it wanted to work on cross-media investigations and build its own programmes and tools for use in data-based projects. The build-up to the release of the Iraq war logs data gave it the opportunity to do both of these things and the organisation helped produce hours of films for Al Jazeera English and Arabic, a programme for Channel 4's Dispatches and stories for French newspaper Le Monde.

In addition the site built its own website, hosting more stories from the data, IraqWarLogs.com. The bureau was working for the past 11 weeks on the data, Overton tells Journalism.co.uk. The day after Dispatches was broadcast the office is empty with staff taking a necessary break from the exhausting, takeaway pizza-fuelled work of the past three months, which has involved up to 20 journalists at some stages, he says.

"We were conscious that we were probably being listened into by state security services, certainly my phone played very oddly during this process. We had a very high number of bizarre crashes on our computer system... I don't think we were being paranoid," he says.

Overton got involved with the investigation after the release of the Afghan war logs and after meeting WikiLeaks' founder Julian Assange. "I don't think any journalist worth his salt would have turned this down," he says, though the much reported issues surrounding the WikiLeaks' set up could put some journalists off.

Overton says he has only spoken with Assange a handful of times: "It's been like working with any other source. We've kept them pretty much at arm's length. There's more to them than just Julian. They're good people, they have good intentions, they want to make the world a better place. I treat them as a source and they didn't have any editorial influence on what we were doing, but they were the conduit through which we understood what other people were doing and when the embargo was lifted.

"We were so caught up in the data that unlike a lot of people I haven't been immersed in the WikiLeaks 'thing'. I've been a lot more focused on what the story is, which is what they want us to do. I think they despair slightly about the whole media circus around who they are and they're very keen themselves that this is about what they're giving to the world and what they're revealing, which is clearly serious questions about coalition forces handling of the Iraq war."

To work with the data, the bureau, led by development producer James Ball, created its own programme to analyse spreadsheets of information. Using this the team applied "good old-fashioned journalism", looking for stories from trends and patterns in the information, anomalies and comparing reports of incidents at the time with reports contained in the logs.

"We typed in the word chocolate: chocolate came up with a sweetshop where an IED had been planted by Al Qaeda and killed lots of children; but also you saw chocolate appearing as chocolate had been given covered in drugs to would-be suicide bombers to create a narcotic euphoric state; and chocolate would also come up as bars of chocolate were thrown from the back of HUMV which resulted in children having a fight, then the mothers of the children having a fight, then the fathers, which resulted in a massive brawl and ended up with one of the children being shot in the leg.  Chocolate, such a benign little word, throws up three different stories," he says.

"We very much had to be directed by the trends within the data. Once we'd established what the trends were or where there were anomalies to what was stated, then we followed it as a story... There were some things that were just so stark and brutal that you couldn't help but be shocked by it."

One drawback when dealing with this volume of data is the stories that are missed, he says. Unfortunately there are stories featured on the bureau's own website, such as an investigation into whether Blackwater was involved in 10 previously unreported civilian deaths, that have not been followed up elsewhere.

"In a way, because there's so much material, if we'd have had that story on its own, that would have been a very big story of the day. As it is it's completely buried, and I sympathise with the reasons why, it is one of the slightly annoying consequences of there being so much attention on the source and the process and not what lies within," says Overton.

Translations of their findings into Chinese and Spanish as an experiment, however, proved hugely successful in increasing international coverage of the investigation, he says. The bureau's stories got more coverage in China than in the rest of the world and were picked up by every major newspaper in Latin America as a result, he suggests.

But the investigation doesn't end there for Overton or the bureau - to maximise the impact of the data and its stories, it needs to be kept alive.

"When Nick Clegg said there needs to be an investigation I'm going to hold him to that. We're putting up a ticker tape, a rolling calendar, for the number of days since Nick Clegg said there would be an investigation till when it happens," says Overton.

"Going forward we're keen to keep this story alive and to continue the redaction process and other processes to make sure that that happens. Just because the story has gone out I don't believe the relationship with WikiLeaks ends here - we will continue to interrogate this."

OWNI
For those that haven't come across it OWNI or OWNI.fr is a non-profit media organisation funded by French web agency 22Mars. OWNI's staff of journalists and developers work on innovative new products that can be sold as solutions to other media businesses - and news applications and data work is a significant part of this.

Off it's own back, and because it was an exciting topic, Kayser-Bril tells Journalism.co.uk how OWNI built an app for the Afghan war logs in a few days to help interested readers trawl through the information. This may have been what alerted Julian Assange to them, as it was only a couple of weeks ago that contact was made and OWNI was asked to create an app and site for WikiLeaks for the Iraq data.

"The idea was, as we did for the Afghan war logs, to let the users browse through and let them create logs as to what they found interesting or not. You rate a log according to its interest and the most interesting documents will float back to the top of the site. It's telling stories that really reflect the daily life in Iraq as it was," says Kayser-Bril.



As part of this process, OWNI tagged each of the 400,000 logs using the OpenCalais system, so users can subscribe to particular tags and be shown other related items according to their interests.

Taking inspiration from the Guardian's crowdsourced analysis of MPs expenses receipts in the UK, OWNI added a 'gaming' element to the app: "The more you contribute to the app the more points you have and we see people having thousands of points so they must really be spending a lot of time using the site. It gives them a reason or reward for participating so much, because everyone sees their name. You spend so much time on the app, there's no reason you shouldn't be credited for your work," says Kayser-Bril.

"What we've found from this and other apps that we've launched is if you find the right community for that app they will stay until the job is done. For instance, we did some crowdsourcing to geolocate polling stations in France, we had a whole bunch of crappy PDF documents, where the addresses had to be typed in by hand. we had 15,000 addresses typed in by hand, it took three months by a community users on a Linux.fr forum, who are really into OpenData. It's just about finding the right community."

Building this application was a tricky process as there's a broad potential audience for the information and it was not possible to predict what the documents would look like. The data had been cleaned and was machine readable, he says, and as a database was not particularly large.

"Dealing with large datasets is not the problem, it's how can we design an interface that might make sense for the user," he adds.

Making the information usable and searchable to as wide an audience as possible gives more potential impact to the stories that lie within - a reason why for this leak, Kayser-Bril suggests, so many organisations were involved.

"We had so many different organisations - NGOs, web-only players like us, traditional media all working together with the same goal of exposing torture in Iraq."

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).