HARMONY: A Human-Centered Multimodal Driving Study in the Wild

摘要: Effective shared autonomy requires a clear understanding of driver’s behavior, which is governed by multiple psychophysiological and environmental variables. Disentangling this intricate web interactions the state behaviors in different real-world scenarios, longitudinally. Naturalistic Driving Studies (NDS) have shown to be an effective approach behavior scenarios. However, due lack technological computing capabilities, former NDS only focused on vision-based approaches, ignoring important factors such as cognition emotion. The main objective paper introduce HARMONY, human-centered multimodal naturalistic driving study, where states are monitored through (1) in-cabin outside video streams (2) physiological signals including heart rate hand acceleration (IMU data), (3) ambient noise, light, vehicle’s GPS location, (4) music logs, song features tempo. HARMONY first study that collects long-term facial, physiological, data simultaneously. This summarizes HARMONY’s goals, framework design, collection analysis, on-going future research efforts. Through presented case we demonstrate importance longitudinal driver sensing using Kernel Density Estimation Methods. Second, leverage application Bayesian Change Point detection methods how can identify responses conditions fusing information with extracted from streams.