Eye Movements During Comprehension of Spoken Scene Descriptions

摘要: Eye Movements During Comprehension of Spoken Scene Descriptions Michael J. Spivey Daniel C. Richardson (spivey@cornell.edu) (dcr18@cornell.edu) Melinda Tyler Ezekiel E. Young (mjt15@cornell.edu) (eey1@cornell.edu) Department Psychology Cornell University Ithaca, NY 14853 USA Abstract A recent eyetracking experiment has indicated that, while staring at a blank white display, participants engaged in imagery tend to make eye movements that mimic the directionality spatial expressions speech stream (Spivey & Geng, 2000). This result is consistent with mental models account language comprehension (e.g., Johnson-Laird, 1983), adds motor component evidence for activation perceptual mechanisms during visual Kosslyn, Thompson, Kim, Alpert, 1995), and fits claims regarding embodiment cognition Varela, Rosch, 1991). However, some methodological concerns remain. We report preliminary observations, controlled experiment, which these are resolved. demonstrate even when includes no instructions imagine anything, participants’ eyes closed, same direction (and especially along axis) as described scene listening spatially extended description.. Introduction More than three decades ago, Donald O. Hebb (1968) suggested very eye-movement scanpaths associated viewing an object may be automatically triggered (via transcortical cell assemblies) person imagining -- empirical support this claim recently been reported. When screen being instructed previously-viewed block pattern, observers produced bore resemblance elicited original actual pattern (Brandt Stark, 1997). Such oculomotor behavior absence input notion or remembering event, we often develop representation event distinctly structure it format thus able take advantage properties inherent Cartesian space, such topography metric relationships. construction interrogation Bower Morrow, 1990; Bryant, 1997; 1983, 1996), uses linguistic activate memory representations, then use those representations partially Farah, 1995; Finke, 1986; Kosslyn et al., 1995). The present study demonstrates any stimulus all, simulations (Barsalou, 1999) trigger corresponding responses. In sense, one might say thinking something involves pretending look it. finding contributes developing embodied view mind Ballard, Hayhoe, Pook, Rao, Brooks, 1991), adequate characterization requires special attention repertoire actions available organism agent. Looking Objects Aren't There study, Geng (2000, Experiment 1) had simply listen pre-recorded scenes looking projection wearing headband-mounted eyetracker. Each descriptions specific (rightward, leftward, upward, downward) manner new objects events were introduced scene. addition, control description was presented, particular present. Pilot results methodology movement patterns much accordance description, however most developed rather accurate suspicions our experimental hypothesis. Although relatively automatic, usually not susceptible voluntary control, concern remained have if they hadn't known their recorded. To avoid potential strategy effects, sham task (of following move around on table), referred session break from eyetracker would turned off ( but don't headband because we'd recalibrate tracker return ), two suspected still tracked, closed session, remaining six remarkably descriptions. Figure 1 shows example data Control (left panel) Rightward (right