Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection
Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection
复制标题
DOI：
10.1145/3550319
复制
发表时间：
2022-09
影响因子：
--
通讯作者：
Kei Tanigaki;Tze Chuin Teoh;Naoya Yoshimura;T. Maekawa;Takahiro Hara
中科院分区：
文献类型：
--
作者：
Kei Tanigaki;Tze Chuin Teoh;Naoya Yoshimura;T. Maekawa;Takahiro Hara
基于机器学习的人体活动识别系统的开发主要包括数据采集、模型训练和评估三个阶段。在数据收集期间，HAR开发人员从佩戴惯性传感器的参与者那里收集标记的数据。在模型训练阶段，开发人员根据收集的训练数据训练HAR模型。在评估阶段，开发人员根据收集的测试数据评估训练后的HAR模型。当HAR模型不能达到目标识别精度时，开发者通过采取一定的措施迭代上述过程，包括收集额外的训练数据，直到重新训练的模型达到目标精度。然而，收集HAR的标签数据需要额外的时间，并会产生高昂的金钱成本。此外，很难确定为在降低成本的同时实现目标准确性而收集的数据量和数据类型。为了解决这个问题，本文提出了一种新的方法来预测当前HAR模型的性能改进，即在收集额外的数据之前，通过使用额外的数据重新训练HAR模型来确定可以实现的性能改进的水平。因此，该方法使得HAR开发者能够通过提供建议来建立用于额外数据收集的策略，例如，如果从另外两个参与者收集用于步行和跑步活动的标签数据，则当前用于步行的HAR模型的HAR精度将提高20%。为了实现这一点，提出了一种称为AIP-Net的神经网络，通过分析现有HAR模型的特征空间，利用所提出的基于熵的注意机制来估计性能的改善。在8个HAR数据集上，采用留一数据集剔除交叉验证方法对AIP-Net的性能进行评估。
The development of a machine-learning-based human activity recognition (HAR) system using body-worn sensors is mainly composed of three phases: data collection, model training, and evaluation. During data collection, the HAR developer collects labeled data from participants wearing inertial sensors. In the model training phase, the developer trains the HAR model on the collected training data. In the evaluation phase, the developer evaluates the trained HAR model on the collected test data. When the HAR model cannot achieve the target recognition accuracy, the developer iterates the above procedures by taking certain measures, including collecting additional training data, until the re-trained model achieves the target accuracy. However, collecting labeled data for HAR requires additional time and incurs high monetary costs. In addition, it is difficult to determine the amount and type of data to collect for achieving the target accuracy while reducing costs. To address this issue, this paper proposes a new method that predicts the performance improvement of the current HAR model, i.e., it determines the level of performance improvement achievable by re-training the HAR model with additional data, before collecting the additional data. Thus, the method enables the HAR developer to establish a strategy for additional data collection by providing advice such as "If labeled data for the Walking and Running activities from two additional participants is collected, the HAR accuracy of the current HAR model for Walking will improve by 20%." To achieve this, a neural network called AIP-Net is proposed to estimate the improvement in performance by analyzing the feature space of the current HAR model using the proposed entropy-based attention mechanism. The performance of AIP-Net was evaluated on eight HAR datasets using leave-one-dataset-out cross-validation.