Health data: How the BMJ approaches investigative reporting

The British Medical Journal has been carrying out in-depth data-driven investigations since it appointed an investigations editor three years ago.

Deborah Cohen, who trained as a doctor before becoming a journalist, leads the BMJ's investigations and frequently "pools resources" with other news teams, such as with Newsnight, Channel 4 News, Panorama, Dispatches and the Telegraph.

"What we have access to are statisticians, peer reviewers and academics who will happily help us out," Cohen told Journalism.co.uk. "We understand the tricks and the literature, but what we don't have as we are a small publication is the resource that perhaps Channel 4 News or the BBC has."

And the investigations can be critical of the BMJ's paying subscribers. "We are a little bit unusual for a journal as often we are attacking our own audience," she explained

Cohen and colleagues use a variety of sources of data for the investigations, from asking sources for information, using the Freedom of Information Act, looking at open data, analysing scientific data, and the BMJ also campaigns to open up data to increase transparency.

Asking sources for information

Three years ago the BMJ investigated hospital whistleblowing policies. Most of the policies were obtained simply by calling the hospitals and asking to speak to a person in HR or communications, while others required Freedom of Information requests.

Cohen and the team entered the various details into a spreadsheet, analysed the data and published a story. You can read more about the process here.

Freedom of Information: Accessing emails

Cohen has also used the Freedom of Information Act to access emails from the Department of Health, some in relation to decision-making.

"One of the big stories we have covered is to do with Tamiflu, an influenza drug that was stockpiled during the pandemic," Cohen said.

The BMJ set up an open data campaign to independently scrutinise data from clinical trials. "We wanted to look at the rationale for stockpiling this drug", Cohen said.

The BMJ submitted Freedom of Information requests. "We asked for correspondence between the chief medial officer, the chief executive, various people within the Department of Health about the drug," Cohen explained.

"We also looked at the relationship with the manufacturer Roche, asking for conversations between the drug company and the particular officials in the Department of Health."

Freedom of Information: Facing barriers

One of the challenges the BMJ journalists can come up against is 'policy in progress', Cohen explained. Another barrier can be the 'commercial and confidence' clause in Freedom of Information legislation.

Cohen explained that she has come up against 'commercial and confidence' several times. "I've done quite a few quite heavy-weight investigations into pharmaceutical companies and their drugs and medical implants," she said.

"I've done a lot of work on medical devices, things like hip implants and the PIP breast implants, looking at what data underpins them."

She is interested in finding out the details of the research that has seen a particular medical device approved for use.

"There's a slight loophole in the Freedom of Information legislation that means that you can't get the most basic information about medical devices," she explained. "That's because the medical device 'commercial and confidence' clause trumps the Freedom of Information Act."

Investigating scientific data

The British Medical Journal teamed up with Panorama to look at the the evidence underpinning health claims made by sports products.

Cohen and the other researchers approached the scientific data "very systematically".

"When you talk about data you do have to try and take a systematic approach because you might be then accused of cherry-picking the story that you want. And obviously when you are dealing with multi-billion dollar companies, you have to be quite systematic in how you approach the data."

Working closely with the Centre for Evidence-Based Medicine at Oxford University, the team took the top 10 best selling men's and women's health and fitness magazines in the UK and the US and looked at the adverts.

"We tore out the adverts and looked at what health claims or sports performance enhancements claims that they were making. We looked to see if there were any scientific references, and if there weren't we approached the companies and asked them for their science that underpinned those claims. We then turned that into a huge spreadsheet."

The researchers were then able to judge the quality of the science. "Obviously this isn't something anybody could do, hence we dipped into the expertise of the Centre of Evidence-Based Medicine."

Using the Wayback Machine as an investigative tool

The Wayback Machine is an internet archive of websites going back to 1996. Cohen explained how she uses it as an investigative tool.

"You can pin down what people were saying at a certain time," she said. "It's quite good when you are looking at the effects of lobby, particularly in medicine where you have quite powerful medical societies that are perhaps funded by drugs companies or implant manufacturers or device companies, and they are pushing for a certain product.

"I guess it is an open data cache going back to the late 1990s."

Opening up data

Cohen said that doctors are making big healthcare decisions every day and it is important to understand the science behind those decisions.

The BMJ uses its Tamiflu open data campaign to call for more transparency, for example. The journal also encourages the registration of trials (on clinicaltrials.gov).

The medical journal has also been applying pressure to the regulator, the European Medicines Agency, in relation to opening up its own data as well as data requested from pharmaceutical companies.

"We have argued that that should be available for independent doctors and academics to study themselves."

• You can hear Deborah Cohen discussing investigative data journalism in this Journalism.co.uk podcast.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).