World’s first computer-vision system to identify human rights abuse

Computers are being trained to identify human rights abuse through photographs – a move which could have a massive impact on improving the lives of those suffering abuse and in bringing their perpetrators to justice.

A research paper by Essex computer scientists was highlighted as the “most thought-provoking” paper by the MIT Technology Review, published by the Massachusetts Institute of Technology,

Currently the United Nations, and other organisations leading the fight against child labour, police violence and other abuses, use photographs as an important tool in identifying where abuse is taking place and as evidence to bring a case to trial.

But it is a long and slow process – the advent of social media means there are literally hundreds of thousands of images to manually sift through to verify if abuse is taking place and then act on it.

Through the ESRC-funded Human Rights, Big Data and Technology Project, computer scientists at the University of Essex are developing a computer-vision based system, the first of its kind in the world, which would dramatically reduce the workload.

Professor Klaus McDonald-Maier, who is leading the work, explained: “Our aim is to make the lives of those combating abuse much easier, as at the moment they are drowning in data.

“With this system, which can identify abuse and then categorise it according to the type of abuse, they can go through images very quickly to narrow down the field and identify pictures which need to be looked at in more detail.

“We have trained and tested the system using a relatively small database of 5,000 images, and have achieved some very promising results. On average it is 88% accurate.”

The trial will now be extended, using a much wider database of photographs and the system is being refined, with experts investigating whether identifying different objects and actions in images can help improve the systems’ performance. It is hoped in future it can also be trained to deal with video footage.

This research is being carried out in the School of Computer Science and Electronic Engineering’s Embedded and Intelligent Systems Laboratory in collaboration with Grigorios Kalliatakis, Dr Shoaib Ehsan, Professor Ales Leonarids (University of Birmingham), Professor Maria Fasli and Professor Klaus McDonald-Maier.