The SYNFACE project - a status report

摘要: SYNFACE is a European project that aims at developing talking face telephone can assist hard of hearing people in their use an ordinary telephone. The partners come from the Netherlands, Great Britain and Sweden. prototypes will be developed for three languages Dutch, English Swedish. See also information on homepage http://www.speech.kth.se/synface This report describes work performed KTH during first half three-year project. main tasks have been to develop automatic phoneme recognition methods provide results with only very short time delay improve articulation facilitate lip-reading. output recogniser decide movements face. should synchronised delayed speech signal Two different perception tests performed. A multilingual test was run prove gain understanding give. delays between audio visual signals learn about sensitivity delay. synthesis has improved using more recorded data. evaluated keeping mind special demands SYNFACE. 2. Multilingual perceptual studies by partners. aim characterise potential intelligibility derived synthetic head controlled phonetically transcribed speech. Speech materials were simple Swedish, Dutch sentences. degraded simulate severe-toprofound impairment. Degradation produced vocoder-like processing either two or frequency bands, each excited noise. 12 native speakers took part which auditory presented alone, face, natural video original talker. Intelligibility purely conditions low (7% 2band vocoder 30% 3-band vocoder). are shown Figure 1. average increase compared no 20%,