Reappearing Interfaces in an IoT World

随着计算机越来越多地嵌入到世界中，用户与计算机交互的过程也越来越复杂。理想情况下，用户遇到的计算（嵌入式技术）和用户带来的计算（个人设备）将支持无缝的用户体验，智能地展示自己，有效地支持学习可用的服务和访问这些服务的方式，并以非侵入性的方式向用户提供反馈，同时保留嵌入在我们遇到的对象，工件和环境中的美学。然而，这种引人注目的直观体验往往难以实现，给用户留下了诸如“我能做什么？”'，'我怎么做？以及成功了吗'.在嵌入式交互的广泛领域内，这项资助的主要研究领域是增强环境的手势输入。自从30年前Bolt的开创性工作以来，手势一直被认为是与增强环境交互的自然方式，但通过手势实现丰富的交互是一个挑战。我的目标是探索不受阻碍的交互（用户的徒手动作是通过环境中的外部传感器捕获的，例如，Kinect、光学相机或某种表面增强）和受阻碍的交互（其中可穿戴或移动终端或定制的手持硬件经由机载传感器捕获输入）。在这项资助的五年期间，我的主要重点将是投影增强现实（P-AR）和无处不在的计算（UbiComp）或物联网（IoT）空间中的交互。这项工作的早期应用将集中在P-AR和UbiComp/IoT的有限实例上。随着物联网产品的成熟，我的研究自然会推广。我研究的长期目标是探索手势作为增强环境的一流输入方式。在这个总体目标下，我在未来五年内的主要研究目标有三个：1。增强环境中的手势交互，包括可扩展性研究，与物理世界的交互，以及上下文对成功指标的影响。2.手势学习和专业知识，包括研究记忆术的潜力，以加强学习，利用运动学建模的专业知识，以及更好的技术来激活学习。3.增强环境中的感知和增强环境的感知，包括使用环境信息和信息可视化技术来增强增强意识和所有权感知，以及探索增强的神秘谷。这些目标中的每一个都直接有助于增强空间中丰富手势交互的目标。

As computers become increasingly embedded into the world, the process of interacting with them increases in complexity for the user. The computation that users encounter (embedded technologies) and the computation that users bring with them (personal devices) would, ideally, support a seamless user experience that reveals itself intelligently, effectively supports learning both about services available and ways to access those services, and non-intrusively presents feedback to users, all while preserving the aesthetics of embedding in objects, artifacts and environments that we encounter. However, this compelling, intuitive experience often remains elusive, leaving users with questions such as 'what can I do?', 'how do I do it?' and 'did it work?'. Within the broad area of embedded interaction, the primary area of research of this grant is gestural input to augmented environments. Since Bolt's seminal work 30 years ago, gestures have been perceived to be a natural way to interact with augmented environments, but challenges in realizing rich interactions via gestures. I aim to explore challenges of both unencumbered interaction (where the user's bare hand motion is captured via external sensors in the environment, e.g., a Kinect, an optical camera, or some surface augmentation) and encumbered interaction (where a wearable or mobile device or custom hand-worn hardware captures input via on-board sensors). Over the five-year period of this grant, my primary focus will be on interaction in Projective Augmented Reality (P-AR) and ubiquitous computing (UbiComp) or Internet-of-Things (IoT) spaces. Early applications of this work will focus on P-AR and limited instances of UbiComp/IoT. As IoT artifacts increase in maturity, my research will naturally generalize. The long-term goal of my research is to explore gesture as a first-class input modality to augmented environments. With that overall goal in mind, there are three primary research objectives that I aim to pursue over the coming five-year period: 1. Gestural Interaction in Augmented Environments, including research on scalability, interactions with the physical world, and impacts of context on metrics for success. 2. Gesture Learning and Expertise, including research on the potential for mnemonics to enhance learning, the leveraging of kinematics to model expertise, and better techniques to activate learning. 3. Perception in and of Augmented Environments, including the use of ambient information and techniques from information visualization to enhance awareness of augmentation and perceptions of ownership and  an exploration of the uncanny valley of augmentation. Each of these objectives contributes directly to the goal of enriching gestural interaction in augmented spaces.