Experiment with asynchrony in multimodal speech communication

摘要: The purpose of this study was to examine the delay effects in audiovisual speech perception for natural and synthetic faces. main focus on SYNFACE project, development a telephone communication aid hearing impaired persons. In experiments, consequence temporal displacement audio relation visual channel investigated. with vocoder-like distortion simulate loss. Twelve different experimental conditions were presented subjects two separate sessions. face tested audio-leading (negative numbers) as well audio-lagging (positive stimuli, whereas only stimuli. Asynchronies examined 50, 175 300 ms. addition, reference examined: synchrony audio-only. Tests ANOVA including both faces revealed that neither -300 ms nor significantly better than audio-only condition, which implies final product would not be beneficial delays magnitude. -50 however, did show lower intelligibility scores synchronous condition. Unfortunately, measured present prototype is greater this. It would, therefore, interesting investigate asynchronies between -175 see exactly where drops. further showed effect type non-significant, indicating quality close face. Experiment asynchrony multimodal v tolerance larger delays, verified by significant decrease performance late at +300 (the corresponding ms). Even gain found +50 condition compared synchrony. However, significant, statistical analysis within interval [-50, +175] have small spoken message