Human-Computer Interaction in Spatial Augmented Reality

点击翻译按钮获取中文摘要

We are at the dawn of the next computing paradigm, where everything around us will be augmented with digital information. Instead of an augmented reality viewable only through a phone or special glasses, multiple projector-camera units will place addressable pixels on arbitrary objects and surfaces to create a Spatial Augmented Reality (SAR). This will mean a revolutionary change in how we access digital information. For example: a sterile and clutter-free way for surgical pre-visualizations to be accessed during an operation; a ubiquitous display method for smart homes and the internet of things; a more flexible and accessible use of display space in offices and design studios; and new possibilities for training and customization in human-powered manufacturing and maintenance. Companies are developing SAR display technologies now, but displaying information is only part of the solution. There is a fundamental open question for Human-Computer Interaction (HCI) research: “How should human input be integrated into SAR?” The candidate has already secured funds to build a $1.8 million facility to conduct HCI research in non-immersive, interactive SAR (which we call Fully Interactive Physio-digital Spaces, or “FIPS”). The proposed research program supports HQP to use this state-of-the-art facility to specifically explore SAR input and interaction when performed at a distance. The two objectives are: contribute empirical results which characterise and model distant input; and invent distant interaction techniques that are practical, usable, comfortable, and efficient. These objectives will be pursued as three streams of work, each focused on one input context. Stream 1 focuses on device-based input, and explores using a smartwatch to manipulate content in SAR (as a technology that is currently practical), as well as inventing a totally new type of hand-held device inspired by the candidate's contact-sensing “Conté”. The focus of stream 2 is hand-based input, which investigates key research problems when using bare hands to interact in SAR: mode-switching, visual feedback, and the best style of hand-based interaction. Stream 3 looks at how implicit whole-body poses (like sitting or standing) and explicit body movements (like foot taps) can augment and refine techniques from streams 1 and 2. Finally, a capstone topic ties all results together in an exemplar SAR demonstration application and encodes findings in a HCI SAR development toolkit.The proposed research program will train 4 PhD, 3 Master's, and multiple undergraduates in core HCI research skills by inventing the future of HCI in a cutting-edge technology facility. The results of the proposed research will be a greater understanding for the potential of SAR interaction based on human needs and capabilities, and ultimately, the invention of new interaction techniques and methods to lead the way towards commercial applications.