Vi-Fi: Associating Moving Subjects across Vision and Wireless Sensors
Vi-Fi: Associating Moving Subjects across Vision and Wireless Sensors
复制标题
DOI：
10.1109/ipsn54338.2022.00024
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Hansi Liu;Abrar Alali;Mohamed Ibrahim;Bryan Bo Cao;Nicholas Meegan;Hongyu Li;M. Gruteser;Shubham Jain;Kristin J. Dana;A. Ashok;Bin Cheng;Hongsheng Lu
中科院分区：
文献类型：
--
作者：
Hansi Liu;Abrar Alali;Mohamed Ibrahim;Bryan Bo Cao;Nicholas Meegan;Hongyu Li;M. Gruteser;Shubham Jain;Kristin J. Dana;A. Ashok;Bin Cheng;Hongsheng Lu
在本文中，我们提出了Vi-Fi，一种多模式系统，利用用户的智能手机WiFi精细定时测量（FTM）和惯性测量单元（IMU）传感器数据，将相机镜头上检测到的用户与其相应的智能手机标识符（例如WiFi MAC地址）相关联。我们的方法使用了一个递归的多模态深度神经网络，该网络利用FTM和IMU测量值沿着用户和相机之间的距离（深度信息）来学习亲和矩阵。作为比较的基线方法，我们还提出了一种使用二分图匹配的传统非深度学习方法。为了便于评估，我们收集了一个多模态数据集，其中包括具有深度信息（RGB-D）的相机视频，WiFi FTM和IMU测量，用于不同现实环境中的多个参与者。使用关联准确度作为评估Vi-Fi将摄像头上的人类用户与其手机ID相关联的保真度的关键指标，我们发现Vi-Fi的关联准确度在81%（实时）到91%（离线）之间。
In this paper, we present Vi-Fi, a multi-modal system that leverages a user's smartphone WiFi Fine Timing Measurements (FTM) and inertial measurement unit (IMU) sensor data to associate the user detected on a camera footage with their corresponding smartphone identifier (e.g. WiFi MAC address). Our approach uses a recurrent multi-modal deep neural network that exploits FTM and IMU measurements along with distance between user and camera (depth information) to learn affinity matrices. As a baseline method for comparison, we also present a traditional non deep learning approach that uses bipartite graph matching. To facilitate evaluation, we collected a multi-modal dataset that comprises camera videos with depth information (RGB-D), WiFi FTM and IMU measurements for multiple participants at diverse real-world settings. Using association accuracy as the key metric for evaluating the fidelity of Vi-Fi in associating human users on camera feed with their phone IDs, we show that Vi-Fi achieves between 81% (real-time) to 91% (offline) association accuracy.