Interacting with Augmented Reality

计算技术现在无处不在，数字环境提供了不断扩展的服务和便利。然而，计算机及其数字内容在很大程度上仍然与真实的物理世界分离——用户通过各种屏幕和输入设备明确地与计算机交互，导致物理世界和数字世界之间存在差距。沉浸式增强现实的兴起预示着未来物理和虚拟之间的界限将越来越模糊，整个物理环境将成为通过投影、头戴式显示器甚至智能眼镜显示数字内容的丰富画布。然而，要实现这样一个未来，即数字内容能够与物理现实和谐共存，而不是仅仅存在于物理现实之中，还有很多工作要做。我建议研究我们如何与现实世界的数字内容互动。我的研究将利用计算和界面技术的新发展，包括高性能计算平台的小型化，投影技术和增强现实的最新进展，以及机器学习和人工智能的使用来学习和预测用户行为，意图和输入-我在所有领域都做出了实质性的贡献。我将通过三个目标来探索交互技术：稳健地感知输入，设计实际交互和部署集成系统。为了更强大的感知输入，我建议探索各种非传统的感官通道——包括电磁传感、多普勒雷达、声学和振动传感——作为理解用户行为和行为的潜在途径。同时，我将通过将深度学习的最新进展应用于信号处理问题，以及通过巧妙的硬件和软件设计有针对性地减少延迟和错误率，来提高现有传感系统的鲁棒性。为了设计实际的交互，我建议探索现实世界用户交互的设计空间，使用户能够使用增强现实系统的全部功能，并使用发现的交互来设计更有针对性的输入传感方法。最后，我将探讨该技术的实际部署，设计一个集成平台，该平台可以作为进一步深入探索该主题的基础。我建议探索将普通灯泡转变为一种计算“信息灯泡”的可行性，将投影、传感和计算能力结合到一个适合标准灯具的设备中，能够将交互界面投射到环境中，并为无处不在的增强现实铺平道路。

Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world - users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content. My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input - all areas in which I have made substantial prior contributions. I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels - including electromagnetic sensing, Doppler radar, acoustic and vibration sensing - as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational "info-bulb", combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.