Computer Science Seminar Series

Abstract

This presentation summarises our work about error situations in human-robot interactions (HRI). Unlike robots that work autonomously, a robot that interacts with a human cannot simply ignore errors, because the human is also noticing them. The first step for a robot to properly address error situations is to recognise them. Therefore, we annotated the behaviour of over 200 videos that show error situations in HRI user studies. We found that the study participants show specific social signals during these error situations and react differently during technical failures and social norm violations by the robot. In a second step, we executed a user study in which the participants interacted either with a faulty or a correctly functioning robot. We recorded the behaviour of our participants and trained different machine learning classifiers with Kinect skeleton data to automatically recognise whether an error situation has occurred. We also asked the participants to rate their interaction with the robot. Come to the talk to find out, what social signals humans show in HRI error situations, whether automatic error detection based on human behaviour works, and whether our participants liked the faulty robot better than the correctly working robot.

If you do not have time to come to the talks and/or want to have more details, please consider reading one of the following papers:

• Pauline Trung, Manuel Giuliani, Michael Miksch, Susanne Stadler, Nicole Mirnig, Gerald Stollnberger, and Manfred Tscheligi. Head and Shoulders: Automatic Error Detection in Human-Robot Interaction. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI 2017), Glasgow, United Kingdom, November 2017
• Nicole Mirnig, Gerald Stollnberger, Markus Miksch, Susanne Stadler, Manuel Giuliani, and Manfred Tscheligi. To Err Is Robot: How Humans Assess and Act toward an Erroneous Social Robot. In Frontiers in Robotics and AI, Vol. 4, pages 21, May 2017
• Nicole Mirnig, Manuel Giuliani, Gerald Stollnberger, Susanne Stadler, Roland Buchner, and Manfred Tscheligi. Impact of Robot Actions on Social Signals and Reaction Times in HRI Error Situations. In Proceedings of 7th International Conference on Social Robotics (ICSR 2015), Paris, France, November 2015
• Manuel Giuliani, Nicole Mirnig, Gerald Stollnberger, Susanne Stadler, Roland Buchner, and Manfred Tscheligi. Systematic Analysis of Video Data from Different Human-Robot Interaction Studies: A Categorisation of Social Signals During Error Situations. In Frontiers in Psychology, 6 (931), July 2015

Bio

Dr. Manuel Giuliani is Professor in Embedded Cognitive AI for Robotics at the Bristol Robotics Laboratory, University of the West of England, Bristol. Before coming to Bristol, he led the Human-Robot Interaction group at the Center for Human-Computer Interaction, Department of Computer Sciences, University of Salzburg. He received a Master of Arts in computational linguistics from Ludwig-Maximilian-University Munich, a Master of Science in computer science from Technische Universität München, and a PhD in computer science from Technische Universität München. He worked on the European projects JAST (Joint Action Science and Technology), JAMES (Joint Action for Multimodal Embodied Social Systems), ReMeDi (Remote Medical Diagnostician) and the Austrian Christian-Doppler-Laboratory "Contextual Interfaces". His research interests include human-robot interaction, social robotics, natural language processing, multimodal fusion, multimodal output generation, and robot architectures.

Host: Verena Rieser

• Back to list