‘Cyborg working’ can improve decision making

Worried about the march of the machines? Relax. We’ll fare better if humans and Artificial Intelligence work together

We’re all used to dire predictions that Artificial Intelligence (AI) will one day take our jobs. But new research completed at the University of Essex has shown that ‘cyborg-working’ – humans and machines, operating as a team – could significantly improve our decision-making.

The peer-reviewed study, published in PLOS ONE, is the first where humans and machines have acted together as peers to make decisions in a realistic scenario. Its findings suggest potential applications in numerous areas including medicine, finance and counter-terrorism.

The research was completed by Dr Davide Valeriani, previously Senior Research Officer at Essex and now a Research Fellow at Harvard Medical School, and Professor Riccardo Poli, from the School of Computer Science and Electronic Engineering. In this study they focus on facial recognition and build on their previous work on Brain Computer Interfaces (BCIs).

Dr Valeriani said: “Humans and machines already work side-by-side in many areas, but the final decision currently rests with one or the other. The use of a single arbiter has been shown to result in worse decisions. This research shows the benefits available when humans and machines work together, as peers, on the same task. Humans have specific strengths, machines have others – by bringing these different skills to the party we get more than the sum of the parts.”

Participants in the study were shown a sequence of almost three hundred security camera-style images and asked, after each image, if they had spotted a particular face among the many present in each image – what made the task even more difficult was that each image was shown for only a fraction of a second.

The experiment was then repeated, with the human participants supported by two new types of AI, developed specifically for this study.

The first of these is a Brain-Computer Interface (BCI) that uses sensors on the scalp to measure the electrical activity of the user’s brains, and machine-learning algorithms to estimate how likely the decision of each user is to be correct, known as ‘decision confidence’. This information can be used to weigh each team member’s response. The study shows that weighing individual decisions on the basis of this BCI-based confidence improves the accuracy of a group’s collective decision-making.

The second new type of AI is a fully-autonomous deep learning algorithm, called a Residual Neural Network, that uses computer vision to extract faces from the security camera images and compares them with the image of the target individual. The Residual Neural Network operates independently of the human participants and estimates its own decision confidence.

Visual searches, where we look for a specific person or object within a scene, are a common feature of everyday life – examples range from the frantic morning search for your keys, to airport staff inspecting the contents of luggage.

Previous studies have shown humans to be surprisingly fallible in these situations. Factors affecting our underperformance include fatigue and the tendency to focus on one thing to the exclusion of others.

The individual humans in this study achieved an average 72% accuracy. The Residual Neural Network, operating alone, scored 84%. Groups of humans assisted by the new types of AI scored highest of all options studied.

The research paired commercially-available computer vision systems with brain sensors, an increasingly affordable technology. Although classed as ‘cyborgs’ – machines are used to supplement the human brain – in this study the process of merging humans and machines was non-invasive, using external sensors and computers to assist the human participants.

The University of Essex is one of the leaders in the UK for brain-computer interface research and scientists from the Essex Brain-Computer Interfaces and Neural Engineering Laboratory, at the University’s School of Computer Science and Electronic Engineering, were involved in the project.

This research was a precursor of a Bilateral Academic Research Initiative (BARI) funded project. Professor Poli is the UK Principal Investigator for this project: This will further investigate technologies to merge humans and machines. BARI funding is provided by the UK Ministry of Defence and the US Department of Defense.