CAREER: Investigating and Combating Micro Signal Attacks in Video Conferencing

视频会议应用程序已被广泛用于连接地理上相距遥远的人们进行工作、学校和社交互动。因为他们使用音频和视频，这些可以显示参与者的行为和环境的信息，当人们有相关的隐私问题时，他们会选择关闭麦克风或摄像头。然而，麦克风和摄像头可能会泄露超出所见所闻语义的其他类型的信息，增加了人们没有意识到的隐私风险。该项目的目标是检查这些风险，查看通过视频会议的合法视频和音频渠道在网络上发送的许多敏感的“微信号”。这些信号太小，人类无法识别，但机器可以检测到，通过仔细的信号处理，它们可能会无意中泄露关于人们的位置、镜头外行为、与计算机的交互以及其他人们可能想要保密的信息。通过更好地了解风险并制定缓解风险的方法，该项目将促进对侧渠道攻击的理解，并增加在线会议的隐私。该项目还通过课程开发、演示平台实施、研究生/本科生培训、K-12参与、公众宣传和未被充分代表的学生参与研究，为网络安全教育做出贡献。该项目促进了声音传感和视觉传感的知识，并将视频会议的安全保护降至微信号水平。它利用视频会议的双向音频通道远程发送恶意声音信号，这些信号感知用户当前的物理环境，并以特定位置的回声信号返回给攻击者。深度学习算法被开发来绕过音频流系统强制执行的回声消除机制，最大限度地为攻击者检索敏感回声。该项目进一步揭示了用户在视频通话期间的屏幕输入，这些输入不在网络摄像头的视野内，被认为是安全的，包括在线投票选择、触摸屏设备输入和物理键盘输入。用户的眼睛运动、击键引起的相机振动和监视器的屏幕灯光被用于隐私推断。为了防止微信号泄露隐私，该项目开发了针对每个攻击的有针对性的防御方法，以及基于压缩和解压缩的通用微信号去除技术，可以处理不同的微信号攻击及其变体。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Video conferencing applications have been broadly used to connect geographically distant people for work, school, and social interactions. Because they use audio and video, and these can show information about a participant's behavior and environment, people choose to turn off microphones or cameras when they have related privacy concerns. However, microphones and cameras may leak other kinds of information beyond the semantics of what is seen and heard, raising privacy risks people are unaware of. This project's goal is to examine those risks, looking at the many sensitive "micro signals" that are sent over the network through the legitimate visual and acoustic channels of video conferencing. These signals are too tiny for humans to recognize, but detectable by machines, and through careful signal processing they might inadvertently reveal information about people's location, off-camera behavior, interaction with their computer, and other things people might want to keep private. Through better understanding the risks and developing methods to mitigate them, this project will advance understanding of side channel attacks and increase online meeting privacy. The project also contributes to cyber security education through curriculum development, demo platform implementation, graduate/undergraduate student training, K-12 involvement, public outreach, and underrepresented student engagement in research.This project advances the knowledge of acoustic sensing and visual sensing and brings the security protection of video conferencing down to the micro signal level. It exploits the two-way audio channel of video conferencing to send malicious acoustic signals remotely, which sense the user's current physical surroundings and return to the attacker with location-specific echo signals. Deep learning algorithms are developed to circumvent the echo cancellation mechanisms enforced by audio streaming systems, maximizing the retrieval of sensitive echoes for attackers. The project further uncovers the user's on-screen inputs during video calls, which are out of the webcam's view and believed to be safe, including online voting choices, touchscreen device inputs, and physical keyboard typing. The user's eye motions, keystroke-induced camera vibrations, and the monitor's screen lights are exploited for privacy inference. To prevent privacy leakage from micro signals, this project develops both targeted defense approaches, which address each attack separately, and general micro-signal removal techniques based on compression and decompression, which can handle different micro-signal attacks and their variants.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.