Bi-LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection
Bi-LSTM Network for Multimodal Continuous Human Activity Recognition and Fall Detection
复制标题
DOI：
10.1109/jsen.2019.2946095
复制
发表时间：
2020-02-01
影响因子：
4.3
通讯作者：
Fioranelli, Francesco
中科院分区：
文献类型：
--
作者：
Li, Haobo;Shrestha, Aman;Fioranelli, Francesco
本文提出了一种基于多层双LSTM网络（双向长短期记忆）的多模态传感器融合框架，用于感知和分类日常活动模式和高风险事件（如福尔斯）。在这项工作中收集的数据是来自FMCW雷达和手腕，腰部和脚踝上的三个可穿戴惯性传感器的连续活动流。每个活动在数据流中具有可变的持续时间，以便活动之间的转换可以在流中的随机时间发生，而无需采用传统的固定持续时间快照。所提出的bi-LSTM实现了可穿戴传感器和雷达数据之间的软特征融合，以及使用两个传感器的混淆矩阵的两种鲁棒硬融合方法。然后提出了一种新的混合融合方案，将联合收割机软融合和硬融合相结合，在识别连续活动和跌倒事件时，将分类性能提高到约96%的准确率。这些融合方案与建议的bi-LSTM网络实现与传统的滑动窗口方法进行了比较，并与现实的“离开一个参与者”（L1 PO）方法（即测试未知的分类器）进行了验证。所开发的混合融合方法能够稳定不同参与者之间的分类性能，降低精度方差高达18.1%，并将最小，最坏情况下的精度提高到16.2%。
This paper presents a framework based on multil-ayer bi-LSTM network (bidirectional Long Short-Term Memory) for multimodal sensor fusion to sense and classify daily activities' patterns and high-risk events such as falls. The data collected in this work are continuous activity streams from FMCW radar and three wearable inertial sensors on the wrist, waist, and ankle. Each activity has a variable duration in the data stream so that the transitions between activities can happen at random times within the stream, without resorting to conventional fixed-duration snapshots. The proposed bi-LSTM implements soft feature fusion between wearable sensors and radar data, as well as two robust hard-fusion methods using the confusion matrices of both sensors. A novel hybrid fusion scheme is then proposed to combine soft and hard fusion to push the classification performances to approximately 96% accuracy in identifying continuous activities and fall events. These fusion schemes implemented with the proposed bi-LSTM network are compared with conventional sliding window approach, and all are validated with realistic "leaving one participant out" (L1PO) method (i.e. testing subjects unknown to the classifier). The developed hybrid-fusion approach is capable of stabilizing the classification performance among different participants in terms of reducing accuracy variance of up to 18.1% and increasing minimum, worst-case accuracy up to 16.2%.