Facial expression recognition based on multi-domain norm-referenced encoding.

Abstract

People can easily recognize human facial expressions on unnatural head shapes such as those of cartoon characters or animals. Current machine learning algorithms, however, struggle with out-of-domain transfer in facial expression recognition if not trained with large amounts of data. Here, we show that insights from neuroscience can be integrated into computer vision models to facilitate the transfer of learned expressions to novel head shapes. Specifically, we propose a biologically inspired mechanism based on norm-referenced encoding, which represents inputs as deviations from a domain-specific reference vector. By assuming that deviations from an appropriately chosen reference are preserved across domains, the model is able to generalize to a new domain using only a single additional training image. We conduct experiments on two datasets consisting of facial expressions on highly varying head shapes,demonstrating the model's generalization abilities and data efficiency. In doing so, we show that norm-referenced encod-ing is scalable and can be leveraged effectively by computer vision models, paving the way towards further applications related to faces, and potentially other appropriate classes of patterns.