Reappearing Interfaces in an IoT World

随着计算机越来越多地融入世界，与它们交互的过程对用户来说增加了复杂性。理想情况下，用户遇到的计算(嵌入式技术)和用户随身携带的计算(个人设备)将支持智能地揭示自身的无缝用户体验，有效地支持对可用服务和访问这些服务的方式的了解，并以非侵入性的方式向用户提供反馈，同时保持嵌入到我们遇到的对象、人工制品和环境中的美学。然而，这种令人信服的直观体验往往令人难以捉摸，给用户留下了这样的问题：我能做什么？我该怎么做？以及“它起作用了吗？”在嵌入式交互的广泛领域内，这项拨款的主要研究领域是对增强环境的手势输入。自30年前博尔特的开创性工作以来，手势一直被认为是与增强环境互动的一种自然方式，但通过手势实现丰富的互动面临挑战。我的目标是探索无障碍交互(通过环境中的外部传感器，如Kinect、光学摄像头或一些表面增强功能来捕捉用户的徒手动作)和有障碍交互(可穿戴或移动设备或定制手持硬件通过车载传感器捕获输入)的挑战。在这笔赠款的五年期间，我的主要重点将是投影增强现实(P-AR)和普适计算(UbiComp)或物联网(IoT)空间中的交互。这项工作的早期应用将集中在P-AR和UbiComp/IoT的有限实例上。随着物联网产品日益成熟，我的研究自然会得出结论。我研究的长期目标是探索手势作为增强环境的一流输入方式。考虑到这一总体目标，我打算在未来五年内追求三个主要研究目标：1.增强环境中的手势交互，包括研究可伸缩性、与物理世界的交互以及环境对成功衡量标准的影响。2.手势学习和专门知识，包括研究助记法增强学习的潜力，利用运动学模拟专业知识，以及激活学习的更好技术。3.增强环境中的感知和对增强环境的感知，包括利用环境信息和来自信息可视化的技术，以提高对增强的认识和对所有权的认识，并探索增强的神秘山谷。这些目标中的每一个都直接促进了在增强空间中丰富手势互动的目标。

As computers become increasingly embedded into the world, the process of interacting with them increases in complexity for the user. The computation that users encounter (embedded technologies) and the computation that users bring with them (personal devices) would, ideally, support a seamless user experience that reveals itself intelligently, effectively supports learning both about services available and ways to access those services, and non-intrusively presents feedback to users, all while preserving the aesthetics of embedding in objects, artifacts and environments that we encounter. However, this compelling, intuitive experience often remains elusive, leaving users with questions such as 'what can I do?', 'how do I do it?' and 'did it work?'. Within the broad area of embedded interaction, the primary area of research of this grant is gestural input to augmented environments. Since Bolt's seminal work 30 years ago, gestures have been perceived to be a natural way to interact with augmented environments, but challenges in realizing rich interactions via gestures. I aim to explore challenges of both unencumbered interaction (where the user's bare hand motion is captured via external sensors in the environment, e.g., a Kinect, an optical camera, or some surface augmentation) and encumbered interaction (where a wearable or mobile device or custom hand-worn hardware captures input via on-board sensors). Over the five-year period of this grant, my primary focus will be on interaction in Projective Augmented Reality (P-AR) and ubiquitous computing (UbiComp) or Internet-of-Things (IoT) spaces. Early applications of this work will focus on P-AR and limited instances of UbiComp/IoT. As IoT artifacts increase in maturity, my research will naturally generalize. The long-term goal of my research is to explore gesture as a first-class input modality to augmented environments. With that overall goal in mind, there are three primary research objectives that I aim to pursue over the coming five-year period: 1. Gestural Interaction in Augmented Environments, including research on scalability, interactions with the physical world, and impacts of context on metrics for success. 2. Gesture Learning and Expertise, including research on the potential for mnemonics to enhance learning, the leveraging of kinematics to model expertise, and better techniques to activate learning. 3. Perception in and of Augmented Environments, including the use of ambient information and techniques from information visualization to enhance awareness of augmentation and perceptions of ownership and  an exploration of the uncanny valley of augmentation. Each of these objectives contributes directly to the goal of enriching gestural interaction in augmented spaces.