Conference Workshop

Active Vision and perception in Human(-Robot) Collaboration (AVHRC 2020)

In the middle is a head with a robot face on the left and a human face on the right. Above is a circuit on the left and electric signals from the human brain on the right. Underneath are the letters "AVHRC" in black text.

Held as part of RO-MAN 2020

Humans naturally interact and collaborate in unstructured social environments that produce an overwhelming amount of information and may yet hide behaviourally relevant variables.

Finding the underlying design principles that allow humans to adaptively find and select relevant information is important for Robotics but also other fields, such as Computational Neuroscience, Interaction Design, and Computer Vision.

Current solutions cover specific tasks, e.g. autonomous cars, and usually employ over-redundant, expensive, and computationally demanding sensory systems that attempt to cover the wide set of sensing conditions which the systems may have to deal with. A promising alternative is to take inspiration from the brain. Adaptive control of sensors and the perception process is a key solution found by nature to cope with computational and sensory demands, as shown by the foveal anatomy of the eye and its high mobility.

Alongside this application of “active” vision, collaborative robotics has recently progressed to human-robot interaction in real manufacturing.

Partners’ gaze behaviours are a crucial source of information that humans exploit for collaboration and coordination. Thus measuring and modelling task-specific gaze behaviours seems to be essential for smooth human-robot interaction. Indeed, anticipatory control for human-in-the-loop architectures, which can enable robots to proactively collaborate with humans, could gain much from parsing the gaze and actions patterns of the human partners.

This workshop was held over two days as part of the RO-MAN 2020 conference. The talks and papers delivered covered a range of topics including social interaction and navigation between robots and humans, human-robot collaboration, and robot vision.


Papers

Talks and papers presented during the workshop have been published below on an open access basis.

Selected papers will be published in a dedicated special issue of Frontiers in Neurorobotics a high quality open access journal.

We would like to thank everyone who submitted papers for consideration, as well as our invited speakers and everyone who presented their research during the workshop.

Invited talks

Professor Giulio Sandini - Predictive vision in Human Robot collaboration (.PDF)

Professor Fiora Pirri - Perception, activities and sustained attention when robots help humans (.PDF)

Professor Angelo Cangelosi - Developmental robotics: Language learning, trust, and theory of mind (.PDF)

Dr Tom Foulsham - Attention during social interaction (.PDF)

Professor Giuseppe Boccignone - A probabilistic tour of visual attention and gaze shift computational models (.PDF)

Contributed papers

O. Eldardeer, G. Sandini, F. Rea, Cognitive Models of Multi-sensory Joint Attention in human robot collaborative tasks (.PDF) (presentation PDF).

Dano Roost, Ralph Meier, Giovanni Toffetti Carughi, and Thilo Stadelmann, Combining Reinforcement Learning with Supervised Deep Learning for Neural Active Scene Understanding (.PDF) (presentation PDF).

Jun Kwan, Chinkye Tan, and Akansel Cosgun, Gesture Recognition for Initiating Human-to-Robot Handovers (.PDF) (presentation PDF).

Tianqi Ma, Lin Zhang, Xiumin Diao, and Ou Ma, Human Intention Prediction Using BIL-SCNN. (.PDF) (presentation PDF).

Stefan Fuchs, and Anna Belardinelli, Gaze-based intention recognition for pick-and-place tasks in shared autonomy (.PDF)

Enas Altarawneh, Michael Jenkin, and I. Scott MacKenzie, Is Putting a Face on a Robot Worthwhile? (.PDF) (presentation PDF).

Ziwen. Jiang, Naizheng. Tang, Lixin. Xu, Steffi. Hußlein, Speculating on the behaviors of the blind people in communication with others to AI. (.PDF)

Natalie Friedman, David Goedicke, Vincent Zhang, Dmitriy Rivkin, Michael Jenkin, Ziedune Degutyte, Arlene Astell, Xue Liu, and Gregory Dudek, Out of my way! Exploring Different Modalities for Robots to Ask People to Move Out of the Way. (.PDF) (presentation PDF).

Speakers

More about Professor Sandini

Biography

Giulio Sandini is Director of Research at the Italian Institute of Technology and full professor of bioengineering at the University of Genoa. He was Research Fellow and assistant professor at the Scuola Normale Superiore in Pisa and Visiting Research Associate at the Department of Neurology of the Harvard Medical School. In 1990 he founded the Laboratory for Integrated Advanced Robotics (LIRA-Lab) and in 1996 he was Visiting Scientist at the Artificial Intelligence Lab of MIT.

Since 1980 Giulio Sandini coordinated several international projects in the area of computer vision, cognitive sciences and robotics. This includes the project RobotCub, funded by the "Cognitive Systems" unit of the European Union from 2004 to 2010, where he coordinated the activities of 11 European partners contributing to the realisation of the iCub humanoid platform as a tool to investigate human sensory, motor and cognitive development.

Giulio Sandini is a Founding Director of the Italian Institute of Technology where in 2006 he established the department of Robotics, Brain and Cognitive Sciences.

Relevant publications

Sandini G.iit, Sciutti A.iit, Humane Robots—from Robots with a Humanoid Body to Robots with an Anthropomorphic Mind ACM Transactions on Human-Robot Interaction, vol. 7, (no. 1).

Sciutti A.iit, Mara M., Tagliasco V., Sandini G. Humanizing human-robot interaction: On the importance of mutual understanding, IEEE Technology and Society Magazine, vol. 37, (no. 1), pp. 22-29.

Sandini G.iit, Mohan V., Sciutti A.iit, Morasso P.iit, Social cognition for human-robot symbiosis - challenges and building blocks, Frontiers in Neurorobotics.

Sandini G.iit, Morasso P.iit, Comment on Review by Cristina Becchio et al. "Seeing mental states: An experimental strategy for measuring the observability of other minds", Physics of Life Reviews, vol. In press.

 

More about Professor Pirri

Biography

Fiora Pirri is Professor of Computer Science at Dipartimento di Ingegneria Informatica, Automatica e Gestionale, Sapienza, Università di Roma. She obtained her PhD from the Universitè Pierre et Marie Curie Paris VI. She currently leads lead ALCOR Vision, Perception, and Learning Laboratory which she founded in 1998.

Relevant Publications

M. Sanzari, V. Ntouskos, F. Pirri. Bayesian image based 3D pose estimation. In proceedings of the European Conference on Computer Vision (ECCV), 9912, 566-582, 2016.

F. Natola, V. Ntouskos, F. Pirri, M. Sanzari. Single image object modeling based on BRDF and r-surfaces learning. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4414-4423, 2016.

M. Gianni, M. Ruiz, F. Ferri, F. Pirri. Terrain contact modeling and classification for ATVs. IEEE International Conference on Robotics and Automation (ICRA-2016).

F. Natola, V. Ntouskos, M. Sanzari, F. Pirri. Bayesian non-parametric inference for manifold based MoCap representation. In Proceedings of the International Conference on Computer Vision (ICCV), 4606-4614, 2015.

More about Professor Rudrauf

Biography

David Rudrauf is a Psychologist and Neuroscientist. He obtained a Ph.D. in Cognitive Sciences from Pierre and Marie Curie University in Paris (2005), a Ph.D. in Neurosciences from the University of Iowa in Iowa City, USA (2005), and a Habilitation from the University Joseph Fourier in Grenoble France (2015).

Professor Rudrauf has worked as an Assistant Professor of Neurology & Radiology, in the Department of Neurology of the University of Iowa in Iowa City, USA (2008-2011); a non-permanent research scientist at the Laboratoire d'Imagerie Fonctionelle (INSERM, UMR S 678), Paris, France (2012-2014); a non-permanent research scientist at the Grenoble Institute of Neuroscience GIN (INSERM), Grenoble, France (2014-2016).

He is now (Associate) Professor of Psychology, at the University of Geneva, FAPSE, Section of Psychology, a member of the Swiss Center for Affective Sciences, Campus Biotech, and of the University Computer Science Center. He has been the Director of the Laboratory of Brain Imaging and Cognitive Neuroscience, Division of Behavioral Neurology and Cognitive Neuroscience, Department of Neurology, UIHC, University of Iowa (2008-2011), and is currently since 2016 the Director of the Laboratory of Multimodal Modelling of Emotion & Feeling.

After years of research in neuroscience, neuropsychology, electrophysiology and multimodal neuroimaging in the US and in France, Professor Rudrauf's current research has moved to mathematical psychology, and the development of a computational model of embodied consciousness (Rudrauf et al, 2017), combined with Virtual Reality and robotics, in order to study the normal and pathological mechanisms of the mind, and develop cybernetic frameworks for autonomous systems, with a focus on imagination, social perspective taking and emotion regulation, and their relations to behaviour and the brain.

Relevant publications

  • Rudrauf, D., Bennequin, D., Granic, I., Landini, G., Friston, K., & Williford, K. (2017). A mathematical model of embodied consciousness. Journal of theoretical biology, 428, 106-131.
  • Rudrauf, D., & Debbané, M. (2018). Building a cybernetic model of psychopathology: beyond the metaphor. Psychological Inquiry, 29(3), 156-164.
  • Williford, K., Bennequin, D., Friston, K., & Rudrauf, D. (2018). The Projective Consciousness Model and Phenomenal Selfhood. Frontiers in psychology, 9.
  • Meuleman, B., & Rudrauf, D. (2018). Induction and profiling of strong multi-componential emotions in virtual reality. IEEE Transactions on Affective Computing.
  • Ognibene, D., Giglia, G., Marchegiani, L., & Rudrauf, D. (2019). Implicit Perception Simplicity and Explicit Perception Complexity in Sensorimotor Comunication. Physics of Life Reviews.

More about Professor Cangelosi

Biography

Angelo Cangelosi is Professor of Machine Learning and Robotics at the University of Manchester (UK). He also is Turing Fellow at the Alan Turing Institute. Previously Angelo was Professor of Artificial Intelligence and Cognition, and founding director, at the Centre for Robotics and Neural Systems at Plymouth University (UK).

Professor Cangelosi studied psychology and cognitive science at the Universities of Rome La Sapienza and at the University of Genoa, and was visiting scholar at the University of California San Diego and the University of Southampton. Professor Cangelosi's main research expertise is on language grounding and embodiment in humanoid robots, developmental robotics, human-robot interaction, and on the application of neuromorphic systems for robot learning.

He currently is the coordinator of the EU H2020 Marie Skłodowska-Curie European Industrial Doctorate “APRIL: Applications of Personal Robotics through Interaction and Learning” (2016-2019). He also is Principal investigator for the ongoing projects “THRIVE++” (US Air Force Office of Science and Research, 2014-1023), the H2020 project MoveCare, and the Marie Curie projects SECURE, DCOMM and STRoNA. He has been coordinator of the FP7 projects ITALK and RobotDoc ITN, as well as UK projects BABEL and VALUE. Overall, he has secured over £30m of research grants as coordinator/PI.

Professor Cangelosi has produced more than 250 scientific publications, and has been general/bridging chair of numerous workshops and conferences including the IEEE ICDL-EpiRob Conferences (Frankfurt 2011, Osaka 2013, Lisbon 2017, Tokyo 2018). In 2012-13 he was Chair of the IEEE Technical Committee on Autonomous Mental Development. He has been Visiting Professor at Waseda University (Japan) and at Sassari and Messina Universities (Italy). Cangelosi is Editor (with K. Dautenhahn) of the journal Interaction Studies, and in 2015 was Editor-in-Chief of IEEE Transactions on Autonomous Development.

His latest book “Developmental Robotics: From Babies to Robots” (MIT Press; co-authored with Matt Schlesinger) was published in January 2015, and recently translated in Chinese and Japanese.

Relevant publications

  • A Cangelosi, L Monrouxe, D Joyce, Spatial language, visual attention, and perceptual simulation, KR Coventry, D Lynott, , Brain and language 112 (3), 202-21381, 2010.
  • M Patacchiola, A Cangelosi, Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods, , Pattern Recognition 71, 132-143, 2017.
  • Cangelosi, Angelo, and Matthew Schlesinger. Developmental robotics: From babies to robots. MIT press, 2015.
  • D Marocco, A Cangelosi, K Fischer, T Belpaeme, Grounding action words in the sensorimotor interaction with the world: experiments with a simulated iCub humanoid robot, Frontiers in neurorobotics 4, 7.
  • C Lyon, CL Nehaniv, A Cangelosi, Emergence of communication and language, Springer Science & Business Media, 2010.

More about Dr Foulsham

Biography

Tom Foulsham is a Reader in the Department of Psychology at the University of Essex, where he has been teaching and conducting research since 2011. He obtained BSc and PhD degrees from the University of Nottingham (UK) and was a Commonwealth Postdoctoral Fellow at the University of British Columbia (Canada) from 2008 to 2011.

Dr Foulsham is a cognitive neuroscientist with expertise in the biology and psychology of human eye movements. He is the author of more than 50 peer-reviewed journal articles investigating human perception and cognition and spanning domains from psychology and sports science to human-computer interaction.

He is a fellow of the Psychonomics Society and the Vision Sciences Society. Editorial roles include  being on the editorial board of  the Journal of Experimental Psychology: Human Perception and Performance; and Attention Perception and Psychophysics;, and formerly Associate Editor of Visual Cognition.

Relevant publications

  • Solman, G. J. F., Foulsham, T., & Kingstone, A. (2017). Eye and head movements are complementary in visual selection. Royal Society Open Science , 4 (1).
  • Foulsham, T., & Kingstone, A. (2017). Are fixations in static natural scenes a useful predictor of attention in the real world?. Canadian Journal of Experimental Psychology/Revue canadienne de psychologie experimentale , 71 (2), 172-181.
  • Ho, S., Foulsham, T. and Kingstone, A (2015) Speaking and Listening with the Eyes: Gaze Signaling during Dyadic Interactions. PloS One , 10 (8). e0136905.
  • Foulsham, T. & Lock, M. (2015). How the eyes tell lies: Social gaze during a preference task. Cognitive Science , 39, 1704-1726.
  • Foulsham, T. (2015). Eye movements and their functions in everyday tasks. Eye , 29(2), 196-199.

More about Professor Boccignone

Biography

Giuseppe Boccignone is a Full Professor at the Dept. of Computer Science  of the University of Milano,  where he lectures on Principles and Models of Perception, Natural Interaction, Affective Computing, Probability and StatistIcs. From 1994  to 2008 he has been with the Dipartimento di Ingegneria dell'Informazione e Ingegneria Elettrica of the University of Salerno as an Assistant Professor and since 2002 as an Associate Professor.

He currently is the Director of  the Perceptual computing and Human SEnsing  (PHuSe) Lab, mostly involved in research spanning  the fields of affective computing,  computational vision, Bayesian machine learning, epistemology of the artificial.

Relevant publications

  • Modelling gaze shift as a constrained random walk. G Boccignone, M Ferraro. Physica A: Statistical Mechanics and its Applications 331 (1), 207-218 111, 2004.
  • Bayesian integration of face and low-level cues for foveated video coding. G Boccignone, A Marcelli, P Napoletano, G Di Fiore, G Iacovoni, S Morsa. IEEE Transactions on Circuits and Systems for Video Technology 18 (12), 1727.
  • Ecological sampling of gaze shifts. G Boccignone, M Ferraro. IEEE Transactions on Cybernetics 44 (2), 266-279.
  • Nonparametric Bayesian attentive video analysis. G Boccignone. 2008 19th International Conference on Pattern Recognition, 1-4.

About this workshop 

This workshop was held as part of the The 29th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN 2020), a leading forum where state-of-the-art innovative results, the latest developments as well as future perspectives relating to robot and human interactive communication are presented and discussed.

The conference covered a wide range of topics related to Robot and Human Interactive Communication, involving theories, methodologies, technologies, empirical and experimental studies. Papers related to the study of the robotic technology, psychology, cognitive science, artificial intelligence, human factors, ethics and policies, interaction-based robot design and other topics related to human-robot interaction were presented.

Organisers

We would like to thank everyone listed in the sections below for generously giving their time to help make this workshop a success.

Main organiser

Dr Dimitri Ognibene is a Lecturer in the School of Computer Science and Electronic Engineering at the University of Essex, UK & University of Milano-Bicocca, Italy. His main interest lies in understanding how social agents with bounded sensory and computational resources adapt to complex and uncertain environments. To this end he develops both neural and Bayesian algorithms and applies them both in physical, e.g. robots, and virtual, e.g. social media, settings.

Communication organisers

Francesco Rea, Instituto Italiano di Tecnologia, Italy

Francesco Rea received his Ph.D in Robotics from the University of Genova (Italy) in 2013. He is currently working as Researcher at the RBCS Department of IIT. His research covers science from cognitive models in humans to robotics and aims at fostering effective interaction between humans and humanoid robot.

Francesca Bianco, University of Essex, UK

Francesca Bianco is a neuroscientist who is achieving a PhD in Computer Science & Electronic Engineering to merge her interests in developmental psychology and robotics. She is currently researching the emergence of Theory of Mind (ToM) in human infants, describing the underlying computational processes, and creating human-inspired ToM robotic architectures to equip robots with increasing social skills.

Vito Trianni, ISTC-CNR, Italy

Vito Trianni is a senior researcher at the ISTC-CNR. His research focuses on swarm intelligence, with particular emphasis on the design and analysis of complex self-organising systems, both natural and artificial (e.g., robot swarms). He believes that intelligence can be recognized in a collective of interacting units (from social insects to robots and humans) as a result of the shared problem-solving abilities.

Lucas Paletta, JOANNEUM RESEARCH – Institute for Information and Communication Technologies, Graz. Austria

Lucas Paletta’s main interest lies in computational psychology and human factors based measurement technologies. Modelling, analysing and interfacing human attention and executive functions is applied for human-machine interaction in production as well as in dementia care.

Ayse Kucukyilmaz, University of Nottingham, UK

Ayse Kucukyilmaz is an assistant professor in Computer Science at the University of Nottingham, UK, and a member of the Lincoln Centre for Autonomous Systems (L-CAS) at the School of Computer Science in University of Lincoln, UK. Her research interests include haptics, physical human-robot interaction, assistive robotics, and machine learning.

Review organisers

Angela Faragasso, University of Tokyo, Japan

Angela Faragasso research activities involve different fields of robotics applications, such as medical, rescue and automation. What has always been a constant in her research, is the use of image processing algorithms to perform autonomous control or to understand properties of objects in the environment, i.e. stiffness, in real-time. Additional research involves hybrid control, intelligent construction systems and bio-inspired robotics.

Manuela Chessa, University of Genoa

Manuela Chessa is Assistant Professor in Computer Science at  Dept. of Informatics, Bioengineering, Robotics, and Systems Engineering of the University of Genoa, Italy. Her research interests are focused on the study of biological and artificial vision systems, on the development of bioinspired models,  of natural human-machine interfaces based on virtual, augmented and mixed reality, and on the perceptual and cognitive aspects of interaction in VR and AR.

Fabio Solari, University of Genoa

Fabio Solari is Associate Professor of Computer Science at the Department of Informatics, Bioengineering, Robotics, and Systems Engineering at University of Genoa (Italy). His research activity concerns computational models of visual perception, the design of bio-inspired artificial vision systems, the study of perceptual effects of virtual and augmented reality systems and the development of natural human-computer interactions in mixed reality.

David Rudrauf,  University of Geneva, Switzerland

David Rudrauf, Ph.D., is Associate Professor of Psychology at the University of Geneva, and a member of the Swiss Center for Affective Science (CISA) and Centre Universitaire d’Informatique (CUI). He is the Director of the Laboratory of Multimodal Modelling of Emotion and Feeling on Campus Biotech in Geneva, and concentrates his research on computational modelling of psychology, combined with models of artificial agents and virtual humans in virtual reality and robotics, to study normal and pathological human behaviours.

Yan Wu,  Robotics Department, Institute for Infocomm Research, A*STAR, Singapore

Yan Wu is the Assistant Head of the Robotics and Autonomous Systems Department, A*STAR Institute for Infocomm Research, Singapore. His research interests include service and assistive robotics, imitation learning, tactile sensory learning and human-robot interaction.

Publication organisers

Fiora Pirri, Sapienza - University of Rome, Italy

Fiora Pirri is professor of Computer Science at Dipartimento di Ingegneria Informatica, Automatica e Gestionale, Sapienza, Università di Roma. Obtained the PhD from the Universitè Pierre et Marie Curie Paris VI. She currently leads lead ALCOR Vision, Perception, and Learning Laboratory which she founded in 1998.

Letizia Marchegiani, Aalborg University, Denmark

Letizia Marchegiani is an Assistant Professor in Robotics at Aalborg University (AAU). Her research interests lie in the areas of signal processing, machine learning, and their application to robotics, autonomous systems, cognitive modelling and intelligent healthcare. Before joining AAU, she was a Postdoctoral Researcher at the Oxford Robotics Institute, at the University of Oxford.

Tom Foulsham, University of Essex, UK

Tom Foulsham is a cognitive psychologist with interests in human vision and visual cognition. His work examines how people pay attention in complex, dynamic and social situations using eye tracking and other techniques from experimental psychology.

Giovanni Maria Farinella, University of Catania, Italy

Giovanni Maria Farinella is Associate Professor at the Department of Mathematics and Computer Science, University of Catania, Italy. He is Founder Member of the IPLAB Research Group at University of Catania since 2005, and Associate Member of the Italian National Research Council since 2018. His research interests lie in the fields of Computer Vision, Pattern Recognition and Machine Learning. His group’s most recent effort is related to First Person (Egocentric) Vision.

Dr. Farinella founded (in 2006) and currently directs the International Computer Vision Summer School (ICVSS). He also founded (in 2014) and currently directs the Medical Imaging Summer School (MISS). He is member of the European Laboratory for Learning and Intelligent Systems (ELLIS) and Senior Member of the IEEE Computer Society.

Reviewers

We would like to thank the following colleagues, who have kindly offered to revise the papers:

  • Carmelo Calafiore
  • Konstantinos Chatzilygeroudis
  • Tobias Fischer
  • Yixing Gao
  • Xiaojun Lu
  • Valerio Ortenzi
  • Sarthak Pathak
  • Francesco Ragusa
  • Anuradha Ranasinghe
  • Irem Uygur
  • Xiaoyu Wang
  • Yusheng Wang

Sponsors

We would like to thank our sponsors for supporting this workshop.

A logo showing three cubes on the left and the words "Frontiers in Neurorobotics" in grey text.
Get in touch
Dr Dimitri Ognibene Workshop organiser
Dr Francesco Rea Communication organiser