Audio-visual and multimodal speech-based systems

摘要: Communication between humans uses many modalities. We communicate not only via verbal language, but also through our use of intonation, gaze, hand gestures, body and facial expressions. Using these modalities, we can add, modify, substitute information in spoken conversations. Complementary several modalities human-to-human communication ensures high accuracy, few problems occur. When do occur, conversation partners easily recover, using the redundancy complementarity The goal research on multimodal systems is to investigate how human—computer interaction benefit from multiple similar ways.