LASO: Exploiting Locomotive and Acoustic Signatures over the Edge to Annotate IMU Data for Human Activity Recognition
LASO: Exploiting Locomotive and Acoustic Signatures over the Edge to Annotate IMU Data for Human Activity Recognition
复制标题
DOI：
10.1145/3382507.3418826
复制
发表时间：
2020-10
期刊：
影响因子：
--
通讯作者：
S. Chatterjee;Avijoy Chakma;A. Gangopadhyay;Nirmalya Roy;Bivas Mitra;Sandip Chakraborty
中科院分区：
文献类型：
--
作者：
S. Chatterjee;Avijoy Chakma;A. Gangopadhyay;Nirmalya Roy;Bivas Mitra;Sandip Chakraborty
来自智能设备和可穿戴设备的注释IMU传感器数据对于开发用于细粒度人类活动识别的监督模型至关重要，尽管为不同环境下的各种人类活动生成足够的注释数据具有挑战性。现有的方法主要使用基于人在回路的技术，包括主动学习;然而，它们是繁琐的，昂贵的，耗时的。利用从嵌入式麦克风的声学数据的可用性在数据收集设备，在本文中，我们提出了LASO，一种多模式的方法，从声学和机车信息的自动数据注释。LASO在边缘设备本身上工作，确保只收集注释的IMU数据，丢弃来自设备本身的声学数据，从而保护用户的音频隐私。在没有任何预先存在的标记信息的情况下，这种自动注释是具有挑战性的，因为IMU数据需要以完全无监督的方式针对不同的时间尺度活动进行会话化。我们使用一个变点检测技术，同时同步从IMU数据与声学数据的机车信息，然后使用预先训练的基于音频的活动识别模型标记的IMU数据，同时处理的声学噪声。LASO有效地注释IMU数据，没有任何明确的人为干预，平均精度为0.93 $（$\pm 0.04$）和0.78 $（$\pm 0.05$）的两个不同的现实生活中的数据集，从车间和厨房环境，分别。
Annotated IMU sensor data from smart devices and wearables are essential for developing supervised models for fine-grained human activity recognition, albeit generating sufficient annotated data for diverse human activities under different environments is challenging. Existing approaches primarily use human-in-the-loop based techniques, including active learning; however, they are tedious, costly, and time-consuming. Leveraging the availability of acoustic data from embedded microphones over the data collection devices, in this paper, we propose LASO, a multimodal approach for automated data annotation from acoustic and locomotive information. LASO works over the edge device itself, ensuring that only the annotated IMU data is collected, discarding the acoustic data from the device itself, hence preserving the audio-privacy of the user. In the absence of any pre-existing labeling information, such an auto-annotation is challenging as the IMU data needs to be sessionized for different time-scaled activities in a completely unsupervised manner. We use a change-point detection technique while synchronizing the locomotive information from the IMU data with the acoustic data, and then use pre-trained audio-based activity recognition models for labeling the IMU data while handling the acoustic noises. LASO efficiently annotates IMU data, without any explicit human intervention, with a mean accuracy of $0.93$ ($\pm 0.04$) and $0.78$ ($\pm 0.05$) for two different real-life datasets from workshop and kitchen environments, respectively.