SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry from Wearable Sensors
SyncWISE: Window Induced Shift Estimation for Synchronization of Video and Accelerometry from Wearable Sensors
复制标题
SyncWISE：用于同步可穿戴传感器的视频和加速度测量的窗口引起的偏移估计
DOI：
10.1145/3411824
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Alshurafa, Nabil
中科院分区：
文献类型：
--
作者：
Zhang, Yun C.;Zhang, Shibo;Liu, Miao;Daly, Elyse;Battalio, Samuel;Kumar, Santosh;Spring, Bonnie;Rehg, James M.;Alshurafa, Nabil
使用可穿戴设备检测日常人类行为(例如，吃饭、吸烟、刷牙)的计算模型的开发和验证需要从自然现场环境收集的带标签的数据，并且具有微行为(例如，在吸烟或进食手势期间的手对嘴手势的开始/结束时间)和相关联的标签的紧密时间同步。视频数据越来越多地被用于这种标签收集。不幸的是，可穿戴设备和带有独立(和漂移)时钟的摄像机使得严格的时间同步具有挑战性。为了解决这个问题，我们提出了用于同步的窗口诱导移位估计方法(SyncWISE)。我们通过同步可穿戴相机和可穿戴加速度计的时间戳来证明我们方法的可行性和有效性，这些视频来自来自21名参与真实世界戒烟研究的21名参与者的163个视频，代表45.2小时的数据。我们的方法比最先进的方法有显著的改进，即使在存在高数据丢失的情况下，在700毫秒的同步容限下也达到了90%的同步精度。我们的方法还在CMU-MMAC数据集上实现了最先进的同步性能。
The development and validation of computational models to detect daily human behaviors (e.g., eating, smoking, brushing) using wearable devices requires labeled data collected from the natural field environment, with tight time synchronization of the micro-behaviors (e.g., start/end times of hand-to-mouth gestures during a smoking puff or an eating gesture) and the associated labels. Video data is increasingly being used for such label collection. Unfortunately, wearable devices and video cameras with independent (and drifting) clocks make tight time synchronization challenging. To address this issue, we present the Window Induced Shift Estimation method for Synchronization (SyncWISE) approach. We demonstrate the feasibility and effectiveness of our method by synchronizing the timestamps of a wearable camera and wearable accelerometer from 163 videos representing 45.2 hours of data from 21 participants enrolled in a real-world smoking cessation study. Our approach shows significant improvement over the state-of-the-art, even in the presence of high data loss, achieving 90% synchronization accuracy given a synchronization tolerance of 700 milliseconds. Our method also achieves state-of-the-art synchronization performance on the CMU-MMAC dataset.