Spoken Language and Vision for Adaptive Human-Robot Cooperation

摘要: In order for humans and robots to cooperate in an effective manner, it must be possible them communicate. Spoken language is obvious candidate providing a means of communication. previous research, we developed integrated platform that combined visual scene interpretation with speech processing provide input learning model. The system was demonstrated learn rich set sentence-meaning mappings could allow construct the appropriate meanings new sentences generalization task. We subsequently extended not only understand what hears, but also describe sees interact human user. This natural extension knowledge sentence-to-meaning now applied inverse scene-tosentence sense (Dominey & Boucher 2005). current chapter extends this work analyse how spoken can used by users communicate Khepera navigator, Lynxmotion 6DOF manipulator arm, Kawada Industries HRP-2 Humanoid, program robots’ behavior cooperative tasks, such as working together perform object transportation task, or assemble piece furniture. framework, Language Programming (SLP) presented. objectives are 1. Allow impart accomplish task robot, form sensory-motor action plan. 2. To user test modify learned plans. 3. do semi-natural real-time manner using observation/demonstration. 4. When possible, exploit from studies cognitive development making implementation choices. With respect development, addition construction grammar model, concept “shared intentions” developmental cognition goal-directed plans will shared robot during activities. Results several experiments SLP employed on different platforms evaluated terms changes efficiency revealed completion time number command operations required tasks. Finally, language, investingate vision well observe activity able take part observed At interface robotics, results interesting they (1) concrete demonstration science contribute human-robot interaction fidelity, (2) suggest might experiment theories developing human.