Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data.
Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data.
复制标题
DOI：
10.1109/jiot.2021.3139038
复制
发表时间：
2022-07-15
影响因子：
10.6
通讯作者：
Turaga, Pavan
中科院分区：
文献类型：
--
作者：
Jeon, Eun Som;Som, Anirudh;Shukla, Ankita;Hasanaj, Kristina;Buman, Matthew P.;Turaga, Pavan
关键词：
深度神经网络由数千或数百万个参数进行参数化，并在许多分类问题上取得了巨大的成功。然而，大量的参数使得这些模型很难集成到智能手机和可穿戴设备等边缘设备中。为了解决这个问题，知识蒸馏（KD）被广泛采用，它使用预先训练的高容量网络来训练更小的网络，适用于边缘设备。在本文中，我们首次研究了将 KD 用于可穿戴设备时间序列数据的适用性和挑战。 KD 的成功应用需要在训练过程中具体选择数据增强方法。然而，目前尚不清楚在 KD 期间是否存在选择增强方法的连贯策略。在本文中，我们报告了一项详细研究的结果，该研究比较和对比了基于 KD 的人类活动分析中的各种常见选择和一些混合数据增强策略。该领域的研究通常受到限制，因为公共领域中可穿戴设备的综合数据库并不多。我们的研究考虑了从小规模公开可用的数据库到源自对人类活动和久坐行为的大规模干预研究的数据库。我们发现 KD 期间数据增强技术的选择对最终性能有不同程度的影响，并且发现最佳网络选择以及数据增强策略特定于手头的数据集。然而，我们还得出了一组通用的建议，可以提供跨数据库的强大基准性能。
Deep neural networks are parametrized by several thousands or millions of parameters, and have shown tremendous success in many classification problems. However, the large number of parameters makes it difficult to integrate these models into edge devices such as smartphones and wearable devices. To address this problem, knowledge distillation (KD) has been widely employed, that uses a pre-trained high capacity network to train a much smaller network, suitable for edge devices. In this paper, for the first time, we study the applicability and challenges of using KD for time-series data for wearable devices. Successful application of KD requires specific choices of data augmentation methods during training. However, it is not yet known if there exists a coherent strategy for choosing an augmentation approach during KD. In this paper, we report the results of a detailed study that compares and contrasts various common choices and some hybrid data augmentation strategies in KD based human activity analysis. Research in this area is often limited as there are not many comprehensive databases available in the public domain from wearable devices. Our study considers databases from small scale publicly available to one derived from a large scale interventional study into human activity and sedentary behavior. We find that the choice of data augmentation techniques during KD have a variable level of impact on end performance, and find that the optimal network choice as well as data augmentation strategies are specific to a dataset at hand. However, we also conclude with a general set of recommendations that can provide a strong baseline performance across databases.
登录
查看更多内容
DOI：
10.1109/tnnls.2014.2308321
发表时间：
2014-12-01
影响因子：
10.4
作者：
Cao, Hong;Tan, Vincent Y. F.;Pang, John Z. F.
通讯作者：
Pang, John Z. F.
DOI：
10.1109/taslp.2015.2438544
发表时间：
2015-09-01
影响因子：
5.4
作者：
Cui, Xiaodong;Goel, Vaibhava;Kingsbury, Brian
通讯作者：
Kingsbury, Brian
DOI：
10.1016/j.engappai.2020.103679
发表时间：
2020-06-01
影响因子：
8
作者：
Gil-Martin, Manuel;San-Segundo, Ruben;Ferreiros-Lopez, Javier
通讯作者：
Ferreiros-Lopez, Javier
DOI：
10.3390/s18092892
发表时间：
2018-08-31
期刊：
Sensors (Basel, Switzerland)
影响因子：
--
作者：
Steven Eyobu O;Han DS
通讯作者：
Han DS
DOI：
10.1109/taslp.2014.2339736
发表时间：
2014-10-01
影响因子：
5.4
作者：
Abdel-Hamid, Ossama;Mohamed, Abdel-Rahman;Yu, Dong
通讯作者：
Yu, Dong