Toward imitating object manipulation tasks using sequences of movement dependency graphs

摘要: This extended abstract describes a new representation that can be used for recognizing and imitating object manipulation tasks, which is based on detecting the co-movement patterns between visual features. The representation consists of a sequence of graphs that evolve over time as the activity progresses. The nodes in these graphs correspond to the features that are tracked by the robot. The edges correspond to the movement dependencies between pairs of tracked visual features. The representation was tested on two manipulation tasks in which a human attempted to insert small blocks inside container and non-container objects. The results show that the robot was able to use the graph-based representation to distinguish between these two tasks. Furthermore, the robot was able to relate its own actions with these objects to the human actions through the similarities in the resulting graph sequences. I. INTRODUCTION Imitation learning frameworks in robotics often focus on replicating motor trajectories provided by humans as closely as possible [2][3]. This approach works well when the task is to imitate gross motor movements. When the goal is to imitate object manipulation tasks, however, this is no longer sufficient. In this case, in addition to imitating the motor actions, the robot must also reproduce the spatial and the temporal relations between the objects. When manipulating objects, motor movements that otherwise look the same may result in completely different outcomes [6]. For example, if the robot’s task is to imitate a person dropping a block inside a container, then even if the robot executes perfect motor trajectories the block may …