MiPad: a multimodal interaction prototype

摘要: Dr. Who is a Microsoft research project aiming at creating speech-centric multimodal interaction framework, which serves as the foundation for NET natural user interface. MiPad application prototype that demonstrates compelling advantages wireless personal digital assistant (PDA) devices, fully integrates continuous speech recognition (CSR) and spoken language understanding (SLU) to enable users accomplish many common tasks using interface technologies. It tries solve problem of pecking with tiny styluses or typing on minuscule keyboards in today's PDAs. Unlike cellular phone, avoids speech-only interaction. incorporates built-in microphone activates whenever field selected. As taps screen uses built roller navigate, tapping action narrows number possible instructions word understanding. currently runs Windows CE Pocket PC 2000 machine where performed. The Dr CSR engine unified CFG n-gram model. SLU based robust chart parser plan-based dialog manager. paper discusses MiPad's design, implementation work progress, preliminary study comparison existing pen-based PDA