Topological Knowledge Distillation for Wearable Sensor Data.
Topological Knowledge Distillation for Wearable Sensor Data.
复制标题
DOI：
10.1109/ieeeconf56349.2022.10052019
复制
发表时间：
2022-10
期刊：
影响因子：
--
通讯作者：
Turaga, Pavan
中科院分区：
文献类型：
--
作者：
Jeon, Eun Som;Choi, Hongjun;Shukla, Ankita;Wang, Yuan;Buman, Matthew P.;Turaga, Pavan
近年来，将可穿戴传感器数据转换为可操作的健康见解引起了人们的广泛兴趣。深度学习方法已在涉及可穿戴设备领域的各种应用中得到运用并取得了很多成功。然而，可穿戴传感器数据具有与受试者之间的灵敏度和变异性以及对分析采样率的依赖性相关的独特问题。为了缓解这些问题，使用拓扑数据分析的不同类型的分析也显示出了前景。拓扑数据分析 (TDA) 通过持久同源算法捕获复杂数据中的鲁棒特征，例如持久图像 (PI)，这有望提高机器学习性能。然而，由于TDA方法对大规模数据所需的计算量较大，集成和实施相对滞后。此外，许多涉及可穿戴设备的应用要求模型足够紧凑，以允许部署在边缘设备上。在这种背景下，知识蒸馏（KD）被广泛应用于使用预先训练的大容量网络（教师模型）生成小型模型（学生模型）。在本文中，我们提出了一种使用两种教师模型的新 KD 策略——一种使用原始时间序列，另一种使用时间序列中的持久性图像。然后，这两位老师使用 KD 培训一名学生。本质上，学生向提供不同知识的异质教师学习。为了考虑来自教师的特征的不同属性，我们在 KD 中应用了退火策略和自适应温度。最后，提炼出一个稳健的学生模型，该模型仅利用时间序列数据。我们发现，通过第二个老师结合持久性特征可以显着提高性能。这种方法提供了一种将深度学习与拓扑特征融合以开发有效模型的独特方法。
Converting wearable sensor data to actionable health insights has witnessed large interest in recent years. Deep learning methods have been utilized in and have achieved a lot of successes in various applications involving wearables fields. However, wearable sensor data has unique issues related to sensitivity and variability between subjects, and dependency on sampling-rate for analysis. To mitigate these issues, a different type of analysis using topological data analysis has shown promise as well. Topological data analysis (TDA) captures robust features, such as persistence images (PI), in complex data through the persistent homology algorithm, which holds the promise of boosting machine learning performance. However, because of the computational load required by TDA methods for large-scale data, integration and implementation has lagged behind. Further, many applications involving wearables require models to be compact enough to allow deployment on edge-devices. In this context, knowledge distillation (KD) has been widely applied to generate a small model (student model), using a pre-trained high-capacity network (teacher model). In this paper, we propose a new KD strategy using two teacher models – one that uses the raw time-series and another that uses persistence images from the time-series. These two teachers then train a student using KD. In essence, the student learns from heterogeneous teachers providing different knowledge. To consider different properties in features from teachers, we apply an annealing strategy and adaptive temperature in KD. Finally, a robust student model is distilled, which utilizes the time series data only. We find that incorporation of persistence features via second teacher leads to significantly improved performance. This approach provides a unique way of fusing deep-learning with topological features to develop effective models.
登录
查看更多内容
DOI：
10.1109/cvprw50498.2020.00425
发表时间：
2020-06
期刊：
Conference on Computer Vision and Pattern Recognition Workshops. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Workshops
影响因子：
--
作者：
Som A;Choi H;Ramamurthy KN;Buman MP;Turaga P
通讯作者：
Turaga P
影响因子：
3.3
作者：
Yang, Xin-She
通讯作者：
Yang, Xin-She
影响因子：
10.6
作者：
Jeon, Eun Som;Som, Anirudh;Shukla, Ankita;Hasanaj, Kristina;Buman, Matthew P.;Turaga, Pavan
通讯作者：
Turaga, Pavan
影响因子：
10.6
作者：
Pachauri D;Hinrichs C;Chung MK;Johnson SC;Singh V
通讯作者：
Singh V
影响因子：
0.8
作者：
Edelsbrunner, H;Letscher, D;Zomorodian, A
通讯作者：
Zomorodian, A