Privacy-preserving Reflection Rendering for Augmented Reality
Privacy-preserving Reflection Rendering for Augmented Reality
复制标题
DOI：
10.1145/3503161.3548386
复制
发表时间：
2022-07
期刊：
影响因子：
--
通讯作者：
Yiqin Zhao;Sheng Wei;Tian Guo
中科院分区：
文献类型：
--
作者：
Yiqin Zhao;Sheng Wei;Tian Guo
当虚拟物体由反光材料构成时，渲染此类物体所需的光照信息可能包含当前相机视野之外的隐私敏感信息。在本文中，我们首次表明，追求准确性的多视角环境光照会泄露相机视野外的场景信息并损害隐私。我们提出了一种简单而有效的隐私攻击方法，该方法能在多种应用场景下从渲染的物体中提取敏感的场景信息，比如人脸和文字。为了抵御此类攻击，我们开发了一种新颖的IPC2S防御方法和一种有条件的R2防御方法。我们的IPC2S防御方法与一种通用的光照重建方法相结合，在保留场景几何结构的同时混淆隐私敏感信息。作为概念验证，我们利用现有的光学字符识别（OCR）和人脸检测模型从过去的相机观测中识别文字和人脸，并模糊与检测区域相关的彩色像素。我们通过将渲染的虚拟物体与使用通用多光照重建技术、ARKit和R2防御方法渲染的物体进行比较，来评估我们的防御方法对视觉质量的影响。我们的视觉和定量结果表明，我们的防御方法在各种渲染场景下能产生结构相似的反射，结构相似性（SSIM）得分高达0.98，同时通过将自动提取成功率降低至至多8.8%来保护敏感信息。
When the virtual objects consist of reflective materials, the required lighting information to render such objects can consist of privacy-sensitive information outside the current camera view. In this paper, we show, for the first time, that accuracy-driven multi-view environment lighting can reveal out-of-camera scene information and compromise privacy. We present a simple yet effective privacy attack that extracts sensitive scene information such as human faces and text from rendered objects under several application scenarios. To defend against such attacks, we develop a novel IPC2S defense and a conditional R2 defense. Our IPC2S defense, combined with a generic lighting reconstruction method, preserves the scene geometry while obfuscating the privacy-sensitive information. As a proof-of-concept, we leverage existing OCR and face detection models to identify text and human faces from past camera observations and blur the color pixels associated with detected regions. We evaluate the visual quality impact of our defense by comparing rendered virtual objects to ones rendered with a generic multi-lighting reconstruction technique, ARKit, and R2 defense. Our visual and quantitative results demonstrate that our defense leads to structurally similar reflections with up to 0.98 SSIM score across various rendering scenarios while preserving sensitive information by reducing the automatic extraction success rate to at most 8.8%.