Rashomon Project aims to combat ambiguity in video

The subjectivity of recollection, known as the Rashomon effect, has always been a legal and journalistic bump in the road in terms of getting to the truth, but a team of students and researchers at the CITRIS Data and Democracy Initiative at UC Berkeley is seeking to change that.

"Right now video is kind of the same as photographs were in the 19th century, when they were very mistrusted," said Ken Goldberg, professor of new media at Berkeley and leader of the Rashomon Project, which recently received funding from the Knight Foundation's prototype fund.

"A photograph was considered unreliable as they could be doctored, and video is viewed very sceptically by the courts right now as it may not show the whole story. But if you're showing video from ten different cameras all from different angles then that tends to give you a little more confidence."

Inspiration for the project first came in November 2011 when, during a protest over tuition fees, incidents of students being beaten by police batons at UC Berkeley and using pepper-spray at UC Davis gained widespread media attention.

"This incident was heavily captured with cellphone cameras and all of a sudden everyone was talking about it," said Goldberg, speaking to Journalism.co.uk, "but all of the documentation was spread out on different sites.

"Some people were uploading to Youtube or Vimeo or Facebook and if anyone wanted to look at anything they were always seeing a very fragmented perspective."

The problem with this fragmented perspective is that it can lead to people drawing different conclusions as to the truth behind a particular incident. In the case of the protests at Berkeley, chancellor Birgeneau was absent during the incident and subsequently addressed a letter to campus telling protesters "to accept the consequences of their decisions".

Upon seeing further footage over the following weeks however, he issued a "sincere apology", vowing to ensure that it would not happen again.

"He said his opinion had changed," Goldberg said. "It got me thinking, one of the biggest problems is that we now have these smartphones and we're able to document all kinds of things that we weren't documenting in the past.

"We have the ability to record photos and videos almost instantly, at any time, but you get a huge deluge of material and I was thinking about how we can digest that in a more coherent way, so you can get a coherent picture of what happened."

His answer is the Rashomon Project. Although still in development, the project offers a "multi-perspective chronology" of an event by using metadata in videos to align the footage, therefore allowing viewers to study an incident from a number of different perspectives.

"One of things you'd be able to do is, say, 2 minutes and 33 secs, this policeman whacks the student here, or this is where the pepper spray begins. You'll be able to see it from multiple perspectives or slow it down or create a window around a specific timezone and then it will loop the videos over that."

At present the only publicly available example is a dramatisation of a student protest (video starts automatically) but the team is in contact with activist groups to work with footage from past events and arrange filming for future events.

"I hope to get three different scenarios where we work with groups and take in the data," continued Goldberg, "we process it and do a whole study of what happens and then we modify the interface so it's more effective for them.

"We've talked to people who are connected in Syria but I want to develop it and shake out the bugs before we start using it in a critical situation like that."

One of the key elements to be developed before using it in situations such as Syria is that of protecting the anonymity of the people featured in the video. Goldberg sees this as integral to fostering users' trust in the platform and encouraging them to pass on footage, for both dissidents and authority figures. The team is using tools to remove information that could identify participants and blur faces automatically, but also going in and making sure this process is complete in every individual frame.

Such accuracy is paramount if the project is to roll out to areas with much lower standards of human rights, Goldberg said, because "if we take one false step or we include one piece of metadata or one unblurred face then that could be fatal for someone".

Building that sense of trust and legitimacy is the driving factor in developing features and usability beyond the basic functions. For one, the project is completely not-for-profit to remove any lingering doubts over motive, and also open source, so the code will be available online for groups to use and modify as they see fit.

"What we're really after is that all sides will be able to tell their part of the story. In some cases the police have footage that they don't feel they'll be able to release and the activists have one perspective, one narrative to tell and it's very complex and what we're clear about is trying to get to as fair and unbiased a presentation as possible, saying let's put everything on the table, and let's not hold back."

Ultimately the team hope that the Rashomon Project will be able to take footage from an event and turn it around in a matter of days or less, where currently it could take months or even years for a major incident to be analysed through video footage appearing on sites from across the web. Goldberg cites the Kennedy assassination and 9/11 as historical events where the ability to analyse concurrent video footage would have led to a greater or faster understanding of what happened.

For now, the project remains a prototype, but has already received funding from both the Knight foundation and Mozilla. In the future, Goldberg hopes UC Berkeley's project, and others like it, will enhance the journalistic process like never before.

Free daily newsletter

If you like our news and feature articles, you can sign up to receive our free daily (Mon-Fri) email newsletter (mobile friendly).