Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications
Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applications
复制标题
DOI：
10.1145/3453142.3491285
复制
发表时间：
2021-10
期刊：
影响因子：
--
通讯作者：
Young D. Kwon;Jagmohan Chauhan;Abhishek Kumar;Pan Hui;C. Mascolo
中科院分区：
文献类型：
--
作者：
Young D. Kwon;Jagmohan Chauhan;Abhishek Kumar;Pan Hui;C. Mascolo
持续学习方法通过尝试解决灾难性遗忘问题，帮助深度神经网络模型逐步适应和学习。然而，这些传统上应用于基于图像的任务的现有方法是否对移动或嵌入式传感系统生成的顺序时间序列数据具有相同的功效仍然是一个悬而未决的问题。为了解决这一空白，我们进行了首次全面的实证研究，量化了三种主要持续学习方案（即正则化、重放和示例重放）在来自三个移动和嵌入式传感应用程序的六个数据集上在具有不同学习复杂性的一系列场景中的性能。更具体地说，我们在边缘设备上实现了端到端的持续学习框架。然后我们研究不同持续学习方法的通用性以及性能、存储、计算成本和内存占用之间的权衡。我们的研究结果表明，即使在复杂的场景中，使用基于样本的方案（例如 iCaRL）进行重放也具有最佳的性能权衡，但会牺牲一些训练样本的存储空间（几 MB）（1% 到 5%）。我们还首次证明，在内存预算有限的情况下在设备上运行持续学习是可行且实用的。特别是，两种类型的移动和嵌入式设备上的延迟表明，跨数据集的增量学习时间（几秒 - 4 分钟）和训练时间（1 - 75 分钟）都是可以接受的，因为训练可以在嵌入式设备充电时在设备上进行，从而确保完整的数据隐私。最后，我们为想要将持续学习范式应用于移动传感任务的从业者提供了一些指南。
Continual learning approaches help deep neural network models adapt and learn incrementally by trying to solve catastrophic forgetting. However, whether these existing approaches, applied traditionally to image-based tasks, work with the same efficacy to the sequential time series data generated by mobile or embedded sensing systems remains an unanswered question. To address this void, we conduct the first comprehensive empirical study that quantifies the performance of three predominant continual learning schemes (i.e., regularization, replay, and replay with examples) on six datasets from three mobile and embedded sensing applications in a range of scenarios having different learning complexities. More specifically, we implement an end-to-end continual learning framework on edge devices. Then we investigate the generalizability, trade-offs between performance, storage, computational costs, and memory footprint of different continual learning methods. Our findings suggest that replay with exemplars-based schemes such as iCaRL has the best performance trade-offs, even in complex scenarios, at the expense of some storage space (few MBs) for training examples (1% to 5%). We also demonstrate for the first time that it is feasible and practical to run continual learning on-device with a limited memory budget. In particular, the latency on two types of mobile and embedded devices suggests that both incremental learning time (few seconds - 4 minutes) and training time (1 - 75 minutes) across datasets are acceptable, as training could happen on the device when the embedded device is charging thereby ensuring complete data privacy. Finally, we present some guidelines for practitioners who want to apply a continual learning paradigm for mobile sensing tasks.