论文标题
惯性幻觉 - 可穿戴惯性设备开始看到东西时
Inertial Hallucinations -- When Wearable Inertial Devices Start Seeing Things
论文作者
论文摘要
我们为环境辅助生活（AAL）提出了一种新型的多模式传感器融合方法，该方法利用了使用特权信息（LUPI）学习的优势。我们解决了标准多模式方法的两个主要缺点，有限的面积覆盖率和降低的可靠性。我们的新框架将模态幻觉的概念与三胞胎学习融合在一起，以训练具有不同方式的模型，以在推理时处理缺失的传感器。 We evaluate the proposed model on inertial data from a wearable accelerometer device, using RGB videos and skeletons as privileged modalities, and show an improvement of accuracy of an average 6.6% on the UTD-MHAD dataset and an average 5.5% on the Berkeley MHAD dataset, reaching a new state-of-the-art for inertial-only classification accuracy on these datasets.我们通过几项消融研究来验证我们的框架。
We propose a novel approach to multimodal sensor fusion for Ambient Assisted Living (AAL) which takes advantage of learning using privileged information (LUPI). We address two major shortcomings of standard multimodal approaches, limited area coverage and reduced reliability. Our new framework fuses the concept of modality hallucination with triplet learning to train a model with different modalities to handle missing sensors at inference time. We evaluate the proposed model on inertial data from a wearable accelerometer device, using RGB videos and skeletons as privileged modalities, and show an improvement of accuracy of an average 6.6% on the UTD-MHAD dataset and an average 5.5% on the Berkeley MHAD dataset, reaching a new state-of-the-art for inertial-only classification accuracy on these datasets. We validate our framework through several ablation studies.
