CoDEm: Conditional Domain Embeddings for Scalable Human Activity Recognition
CoDEm: Conditional Domain Embeddings for Scalable Human Activity Recognition
复制标题
DOI：
10.1109/smartcomp55677.2022.00017
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
A. Faridee;Avijoy Chakma;Zahid Hasan;Nirmalya Roy;Archan Misra
中科院分区：
文献类型：
--
作者：
A. Faridee;Avijoy Chakma;Zahid Hasan;Nirmalya Roy;Archan Misra
我们探讨了辅助标签在提高基于可穿戴传感器的人类活动识别（HAR）系统的分类精度方面的作用，该系统主要是在活动标签的监督下训练的（例如跑步、行走、跳跃）。在数据收集过程中，通常可以获得补充元数据，如可穿戴传感器的身体位置、受试者的人口统计信息（如性别、年龄）和使用的可穿戴设备类型（如智能手机、智能手表）。这些信息虽然与活动分类任务没有直接关系，但仍然可以提供辅助监督，并有可能通过提供关于如何处理来自域（即位置、人员或传感器）变化的引入样本异质性的额外指导，显著提高HAR的准确性，特别是在有限活动标签存在的情况下。然而，在分类管道中集成这样的元数据信息是非常重要的——(i)活动和领域标签空间之间的复杂交互很难用简单的多任务和/或对抗性学习设置来捕获，（ii）元数据和活动标签可能无法同时用于所有收集的样本。为了解决这些问题，我们提出了一个新的框架条件域嵌入（CoDEm）。从可用的未标记原始样本及其域元数据中，我们首先使用对比学习方法学习一组域嵌入，以处理域间可变性和域间相似性。为了对活动进行分类，CoDEm然后以一种对比的方式学习标签嵌入，以一种新的注意力机制为条件的领域嵌入，强制模型学习复杂的领域-活动关系。我们在三个基准数据集中针对许多多任务和对抗性学习基线广泛评估CoDEm，并在每个途径中实现最先进的性能。
We explore the effect of auxiliary labels in improving the classification accuracy of wearable sensor-based human activity recognition (HAR) systems, which are primarily trained with the supervision of the activity labels (e.g. running, walking, jumping). Supplemental meta-data are often available during the data collection process such as body positions of the wearable sensors, subjects' demographic information (e.g. gender, age), and the type of wearable used (e.g. smartphone, smart-watch). This information, while not directly related to the activity classification task, can nonetheless provide auxiliary supervision and has the potential to significantly improve the HAR accuracy by providing extra guidance on how to handle the introduced sample heterogeneity from the change in domains (i.e positions, persons, or sensors), especially in the presence of limited activity labels. However, integrating such meta-data information in the classification pipeline is non-trivial - (i) the complex interaction between the activity and domain label space is hard to capture with a simple multi-task and/or adversarial learning setup, (ii) meta-data and activity labels might not be simultaneously available for all collected samples. To address these issues, we propose a novel framework Conditional Domain Embeddings (CoDEm). From the available unlabeled raw samples and their domain meta-data, we first learn a set of domain embeddings using a contrastive learning methodology to handle inter-domain variability and inter-domain similarity. To classify the activities, CoDEm then learns the label embeddings in a contrastive fashion, conditioned on domain embeddings with a novel attention mechanism, enforcing the model to learn the complex domain-activity relationships. We extensively evaluate CoDEm in three benchmark datasets against a number of multi-task and adversarial learning baselines and achieve state-of-the-art nerformance in each avenue.