Travel Grant 2020-21

Expressive behaviour as a social signal for socially-competent human-robot interaction


Social agents are graphically or robotically embodied entities designed to act competently and successfully in everyday human social environments. These could be homes, schools, workplaces, or more public spaces, like shopping malls and museums. We know that humans impute social agency to such embodied entities. They therefore assume competence in the social signals humans use themselves to regulate interaction and to manifest what they feel or think. For this reason, researchers in embodied social agents have long worked on agent expressive behaviour - facial expressions, vocalisations, gestures, posture, both for graphical and robotic agents.

Generating expressive behaviour (as distinct from scripting via pre-annotated dialogue items) requires an architecture that can model affective responses. Such architectures are often based on sound psychological theory, especially cognitive appraisal theory, but frequently assume that the social agent merely expresses its own internal state. However, social signals are social because they take into account the social context and the impact such signals may make on interaction partners.A very initial look at the issue can be found in: Ritter, C. and Aylett, R., 2015. The Partial Poker-Face: when agents try to hide their true feelings. In International Conference on Intelligent Virtual Agents (pp. 479-482). Springer, Cham.
A more detailed discussion can be found in:
Aylett, R., Ritter, C., Lim, M.Y., Broz, F., McKenna, P., Keller, I. and Rajendran, G., 2019. An architecture for emotional facial expressions as social signals. IEEE Transactions on Affective Computing.

This travel grant supports work on creating mechanisms for embodied social agents that apply theory of mind (ToM) capabilities to the detection and management by the agent of its own affective state. This would enable the development of more socially competent agents suitable for a wider range of applications. It involves visits to: University of Geneva (David Rudrauf); University of Lisbon (Ana Paiva) and University of Southern California (Jon Gratch).

University of Geneva

Discussion of Bayesian architecture and its use to model changes in the autonomic system associated with affective experiences such as blushing and sweating.

Presentation given over zoom on April 1st, 2020.

  1. pdf of the slides (9+meg)
  2. video of the talk(150meg)


Ruth Aylett
Heriot-Watt University,
E14 4AS.

Tel +44 131 451 4189/3456
Fax +44 131 451 3327

Last Update: 17 April 2020