Towards a General Video-based Keystroke Inference Attack
Towards a General Video-based Keystroke Inference Attack
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Zhuolin Yang;Yuxin Chen;Zain Sarwar;Hadleigh Schwartz;Ben Y. Zhao;Haitao Zheng
中科院分区：
文献类型：
--
作者：
Zhuolin Yang;Yuxin Chen;Zain Sarwar;Hadleigh Schwartz;Ben Y. Zhao;Haitao Zheng
大量的研究文献已经确定了使用统计模型提取键盘上键入的内容的恶意推断攻击的隐私风险。然而，现有的攻击无法在现实环境中操作，并且依赖于对标记的训练数据、键盘布局知识、精心放置的传感器或来自其他侧通道的数据的强烈假设。本文描述了开发和评估一种通用的基于视频的反推理攻击的经验，该攻击使用单一商品相机手机在公共环境中运行，没有预训练，没有键盘知识，没有本地传感器，也没有侧信道。我们表明，使用自监督方法，可以处理、标记和过滤来自视频的嘈杂手指跟踪数据，以训练DNN神经网络推理模型，这些模型可以在同一视频上准确运行。使用IRB批准的用户研究，我们在各种环境，键盘和内容以及具有不同打字行为和能力的用户中验证攻击功效。我们的项目网站位于：https://sandlab.cs.uchicago.edu/keystroke/。
A large collection of research literature has identified the privacy risks of keystroke inference attacks that use statistical models to extract content typed onto a keyboard. Yet existing attacks cannot operate in realistic settings, and rely on strong assumptions of labeled training data, knowledge of keyboard layout, carefully placed sensors or data from other side-channels. This paper describes experiences developing and evaluating a general, video-based keystroke inference attack that operates in common public settings using a single commodity camera phone, with no pretraining, no keyboard knowledge, no local sensors, and no side-channels. We show that using a self-supervised approach, noisy finger tracking data from a video can be processed, labeled and filtered to train DNN keystroke inference models that operate accurately on the same video. Using IRB approved user studies, we validate attack efficacy across a variety of environments, keyboards, and content, and users with different typing behaviors and abilities. Our project website is located at: https://sandlab.cs.uchicago.edu/keystroke/ .