We attempt to build the Affective Guide based on patterns that are familiar to the user but without explicit labelling of the expressed emotions. We argue that the interpretation of facial expressions depends on individuals and context of occurrence, hence put the task in the hand of the observer. Therefore, we propose a simple novel approach of facial expression mapping onto the arousal and valence emotional space. This approach is flexible and is able to produce an infinite range of expressions.

We mapped meaningful facial features: eyes, mouth and eyebrows onto the two emotional dimensions, valence and arousal. Variation in these values will move the different facial features dynamically in real time producing different emotions. The valence value moving from negative to positive will move the lip curvature from a downturn U to an upturn U as depicted in Figure 1. A value from 0 to 0.5 denotes negative valence while a value greater than 0.5 represents positive valence. In the case of extreme pleasure, the cheek raiser is visible below the eyes. In contrast, for extreme displeasure, wrinkles are formed beside the wing of each nostril due to the action of the naso-labial fold.

Along the arousal dimension, the size of eye opening increases with increasing arousal and reduces with decreasing arousal as shown in Figure 2. As for the eyebrows, they are influenced by both the arousal and valence values. Under positive valence, when arousal is low to medium (less than 0.5), the eyebrows will have a slight V curve. The eyebrows become more and more relaxed and straightened with increasing arousal. When arousal is very high (more than 0.8), the eyebrows will be raised slightly and the raised inner eyebrows cause delicate wrinkles to be formed across the forehead as presented in Figure 3. On the other hand, Figure 4 shows that the opposite takes place when valence is negative. If arousal is very high, the inner eyebrows curve downwards forming a V. The knitting of the inner eyebrows causes furrows to be observed especially when the arousal becomes too high. On the other hand, as arousal decreases, the curve will smoothen and become more and more relaxed. Nevertheless, when arousal falls below neutral (less than 0.5), the curve becomes an upturned V and in the extreme case, forms vertical furrows above the base of the nose, with transverse wrinkles across the forehead.

The resulting facial expressions along the arousal and valence dimensions are shown in Figure 5. From the figure, it can be seen that a wide variation of facial expressions can be generated. Although only simple animation is applied, the nuances of the underlying emotional assessment (arousal and valence) can be reflected. This approach provides a robust and flexible architecture where emotion expressions are not limited to the basic discrete expressions but span a continuous space.

To see the changes of the guide's facial expression in real time with 10 frames per feature, click here.

Figure 1. The lip curvature change along the valence dimension

Figure 2. The eyes opening along the arousal dimension

Figure 3. Movement of the eyebrows along the arousal dimension when valence is positive

Figure 4. Movement of the eyebrows along the arousal dimension when valence is negative

Figure 5. Different facial expressions on the arousal-valence space