Interacting with Augmented Reality

计算技术现在无处不在，数字环境提供了越来越多的服务和便利。然而，计算机及其数字内容在很大程度上仍然与真实的物理世界分离，用户通过各种屏幕和输入设备与计算机进行明确的交互，导致物理世界和数字世界之间的差距。沉浸式增强现实的兴起指向了这样一个未来：物理和虚拟之间的界限将越来越模糊，整个物理环境将成为通过投影、头盔显示器甚至智能眼镜显示数字内容的丰富画布。然而，要实现这样一个未来，数字内容可以与物理现实如此和谐地共存，而不仅仅是在物理现实中，还有很多工作要做。我建议研究我们将如何与现实世界的数字内容互动。*我的研究将利用计算和接口技术的新发展，包括高性能计算平台的微型化、投影技术和增强现实的最新进展，以及使用机器学习和人工智能来学习和预测用户行为、意图和输入，这些领域都是我之前做出重大贡献的领域。*我将通过三个目标来探索交互技术：稳健地感知输入，设计实际的交互和部署集成系统。为了更有力地感知输入，我建议探索各种非传统的感知渠道，包括电磁感知、多普勒雷达、声学和振动感知，作为了解用户行为和行为的潜在途径。同时，我将通过将深度学习的最新进展应用于信号处理问题，以及通过巧妙的硬件和软件设计有针对性地减少延迟和错误率，来提高现有传感系统的健壮性。为了设计实际的交互，我建议探索世界上用户交互的设计空间，使用户能够使用增强现实系统的全部功能，并使用发现的交互来设计更有针对性的输入感知方法。最后，我将探索该技术的实际部署，设计一个集成平台，作为进一步深入探讨该主题的基础。我建议探索将普通灯泡转变为一种计算“信息灯泡”的可行性，将投影、传感和计算能力结合到一种适合标准灯具的设备中，能够将交互界面投影到环境中，并为无处不在的增强现实铺平道路。

Computing technology is now ubiquitous, with the digital environment offering an ever-expanding array of services and conveniences. However, computers and their digital contents have remained largely separated from the real, physical world users explicitly interact with computers through various screens and input devices, leading to a gap between the physical and digital worlds. The rise of immersive augmented reality points towards a future in which the lines between physical and virtual will increasingly blur, and the entire physical environment will become a rich canvas for digital content displayed via projection, head-mounted displays, or even smart glasses. Yet, there is much work to be done to achieve a future in which digital content can live so harmoniously alongside physical reality, instead of merely inside of it. I propose to investigate how we would interact with real-world digital content.***My research will leverage new developments in computing and interface technology, including the miniaturization of high-performance computational platforms, recent advances in projection technologies and augmented reality, and the use of machine learning and artificial intelligence to learn and predict user behaviour, intent and input all areas in which I have made substantial prior contributions.***I will explore interaction techniques through three objectives: robustly sensing input, designing practical interactions and deploying integrated systems. To more robustly sense input, I propose exploring various non-traditional sensory channels including electromagnetic sensing, Doppler radar, acoustic and vibration sensing as potential avenues for understanding the actions and behaviour of users. Simultaneously, I will improve the robustness of existing sensing systems by applying recent advances in deep learning to the signal processing problem, as well as applying targeted reductions to latency and error rate through clever hardware and software design. To design practical interactions, I propose to explore the design space of on-world user interactions, enabling users to use the full capabilities of augmented reality systems, and using the discovered interactions to design more targeted input sensing approaches. Finally, I will explore a real-world deployment of the technology, designing an integrated platform which can serve as a base for further high-level explorations into this topic. I propose exploring the feasibility of transforming the ordinary light bulb into a kind of computational “info-bulb”, combining projection, sensing, and computational capabilities into a device that fits into standard light fixtures, able to project interactive interfaces onto the environment and paving the way for ubiquitous augmented reality.