Towards a Robust Speechreading Dialog System

摘要: We describe a hybrid speechreading system that is based on Manifold Learning technique, Neural Networks, and Hidden Markov Models. technique for representing learning smooth nonlinear lowdimensional surfaces embedded in highdimensional abstract feature spaces. The capable of determining the structure surface finding closest manifold point to given query point. use this learn “space lips”. learned used tracking extracting lips, interpolating between frames an image sequence providing features recognition. lip manifold, connectionist phone classifier, Models significantly improves performance acoustic speech recognizers degraded environments. also present preliminary results purely visual reader work progress new face technique.