Multimodal fusion of audio, scene, and face features for first impression estimation

摘要: Affective computing, particularly emotion and personality trait recognition, is of increasing interest in many research disciplines. The interplay shows itself the first impression left on other people. Moreover, ambient information, e.g. environment objects surrounding subject, also affect these impressions. In this work, we employ pre-trained Deep Convolutional Neural Networks to extract facial information from images for predicting apparent personality. We investigate Local Gabor Binary Patterns Three Orthogonal Planes video descriptor acoustic features extracted via popularly used openSMILE tool. subsequently propose classifying using a Kernel Extreme Learning Machine fusing their predictions. proposed system applied ChaLearn Challenge First Impression Recognition, achieving winning test set accuracy 0.913, averaged over “Big Five” traits.