Interacting with Augmented Reality

计算技术现在无处不在，数字环境提供了不断扩展的服务和便利。然而，计算机及其数字内容仍然在很大程度上与真实的物理世界分离，用户通过各种屏幕和输入设备明确地与计算机交互，导致物理世界与数字世界之间的差距。沉浸式增强现实的兴起指向了一个未来，在这个未来中，物理和虚拟之间的界限将越来越模糊，整个物理环境将成为通过投影、头戴式显示器甚至智能眼镜显示的数字内容的丰富画布。然而，要实现一个数字内容可以与物理现实和谐共存的未来，而不仅仅是在物理现实中，还有很多工作要做。 我的研究将利用计算和接口技术的新发展，包括高性能计算平台的小型化，投影技术和增强现实的最新进展，以及使用机器学习和人工智能来学习和预测用户行为，意图和输入我之前做出了大量贡献的所有领域。 我将通过三个目标探索交互技术：鲁棒感测输入，设计实用的交互和部署集成系统。为了更强大的感觉输入，我建议探索各种非传统的感官渠道，包括电磁传感，多普勒雷达，声学和振动传感作为了解用户的行动和行为的潜在途径。同时，我将通过将深度学习的最新进展应用于信号处理问题，以及通过巧妙的硬件和软件设计有针对性地降低延迟和错误率，来提高现有传感系统的鲁棒性。为了设计实用的交互，我建议探索世界上用户交互的设计空间，使用户能够使用增强现实系统的全部功能，并使用发现的交互来设计更有针对性的输入感知方法。最后，我将探索该技术的实际部署，设计一个集成平台，作为进一步深入研究该主题的基础。我建议探索将普通灯泡转变为一种计算“信息灯泡”的可行性，将投影，传感和计算能力结合到一个适合标准灯具的设备中，能够将交互式界面投射到环境中，并为无处不在的增强现实铺平道路。

Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content. My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input all areas in which I have made substantial prior contributions. I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels including electromagnetic sensing, Doppler radar, acoustic and vibration sensing as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational “info-bulb”, combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.