Interacting with Augmented Reality

计算技术现在无处不在，数字环境提供了不断扩展的服务和便利。然而，计算机及其数字内容在很大程度上仍然与真实的物理世界分离，用户通过各种屏幕和输入设备明确地与计算机交互，导致物理世界和数字世界之间存在差距。沉浸式增强现实的兴起预示着未来物理和虚拟之间的界限将越来越模糊，整个物理环境将成为通过投影、头戴式显示器甚至智能眼镜显示数字内容的丰富画布。然而，要实现这样一个未来，即数字内容能够与物理现实和谐共存，而不是仅仅存在于物理现实之中，还有很多工作要做。我建议研究我们如何与现实世界的数字内容互动。

Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content. My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input all areas in which I have made substantial prior contributions. I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels including electromagnetic sensing, Doppler radar, acoustic and vibration sensing as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational “info-bulb”, combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.