CNN-Based Multistage Gated Average Fusion (MGAF) for Human Action Recognition Using Depth and Inertial Sensors

摘要: Convolutional Neural Network (CNN) provides leverage to extract and fuse features from all layers of its architecture. However, extracting fusing intermediate different CNN structure is still uninvestigated for Human Action Recognition (HAR) using depth inertial sensors. To get maximum benefit accessing the CNN’s layers, in this paper, we propose novel Multistage Gated Average Fusion (MGAF) network which extracts fuses our computationally efficient (GAF) network, a decisive integral element MGAF. At input proposed MGAF, transform sensor data into images called sequential front view (SFI) signal (SI) respectively. These SFI are formed information generated by data. employed feature maps both modalities. GAF extracted effectively while preserving dimensionality fused as well. The MGAF has structural extensibility can be unfolded more than two Experiments on three publicly available multimodal HAR datasets demonstrate that outperforms previous state-of-the-art fusion methods depth-inertial terms recognition accuracy being much efficient. We increase an average 1.5% reducing computational cost approximately 50% over state-of-art.