Reappearing Interfaces in an IoT World

点击翻译按钮获取中文摘要

As computers become increasingly embedded into the world, the process of interacting with them increases in complexity for the user. The computation that users encounter (embedded technologies) and the computation that users bring with them (personal devices) would, ideally, support a seamless user experience that reveals itself intelligently, effectively supports learning both about services available and ways to access those services, and non-intrusively presents feedback to users, all while preserving the aesthetics of embedding in objects, artifacts and environments that we encounter. However, this compelling, intuitive experience often remains elusive, leaving users with questions such as 'what can I do?', 'how do I do it?' and 'did it work?'. Within the broad area of embedded interaction, the primary area of research of this grant is gestural input to augmented environments. Since Bolt's seminal work 30 years ago, gestures have been perceived to be a natural way to interact with augmented environments, but challenges in realizing rich interactions via gestures. I aim to explore challenges of both unencumbered interaction (where the user's bare hand motion is captured via external sensors in the environment, e.g., a Kinect, an optical camera, or some surface augmentation) and encumbered interaction (where a wearable or mobile device or custom hand-worn hardware captures input via on-board sensors). Over the five-year period of this grant, my primary focus will be on interaction in Projective Augmented Reality (P-AR) and ubiquitous computing (UbiComp) or Internet-of-Things (IoT) spaces. Early applications of this work will focus on P-AR and limited instances of UbiComp/IoT. As IoT artifacts increase in maturity, my research will naturally generalize. The long-term goal of my research is to explore gesture as a first-class input modality to augmented environments. With that overall goal in mind, there are three primary research objectives that I aim to pursue over the coming five-year period: 1. Gestural Interaction in Augmented Environments, including research on scalability, interactions with the physical world, and impacts of context on metrics for success. 2. Gesture Learning and Expertise, including research on the potential for mnemonics to enhance learning, the leveraging of kinematics to model expertise, and better techniques to activate learning. 3. Perception in and of Augmented Environments, including the use of ambient information and techniques from information visualization to enhance awareness of augmentation and perceptions of ownership and  an exploration of the uncanny valley of augmentation. Each of these objectives contributes directly to the goal of enriching gestural interaction in augmented spaces.