User interface feedforward and feedback supporting and enabling body tracking technologies

计算机视觉的最新进展使用户能够以比以前更自然的方式向系统提供输入。基于摄像头的用户身体跟踪允许系统识别手势、身体姿势变化和移动。尽管这些能力令人印象深刻，但此类系统未能实现广泛的工业应用。此前，这种缺乏使用可能是由于技术的限制。然而，已经取得了重大改进，这种系统现在能够实现接近传统输入设备的识别率。因此，我们将有限的商业使用归因于两个一般知识领域的不足：(1)缺乏对这些传感器在人类能力和需求方面的真正效用的了解；(2)促进其使用所需的用户界面的组成。 这一研究项目旨在扩大目前对这两个领域的理解，具体目标是提供能够支持使用身体跟踪技术进行交互的全新用户界面。为了实现对人类能力的理解，我们将应用经验评估和人种学技术来：1)测试和确定人类能力的极限；2)确定哪些能力最能与传感器的能力相对应；3)了解这些能力在什么情况下得到最好的利用。这将产生对两个关键问题的更全面的了解，即每种输入方式在哪里和何时最适合，以及什么输入手段最适合这一背景。在此基础上，我们将构建适合这些上下文的用户界面，允许用户以全新的方式向设备提供输入。其结果将是更好地了解这些技术的潜力、人类的需求和能力，并最终了解将这些技术应用于工业用途的效用和方法。

Recent advances in computer vision have made it possible for users to provide input to systems in far more natural ways than before. Camera-based tracking of a user's body allows systems to recognize hand gestures, body posture changes, and movements. While these are impressive capabilities, such systems have failed to make the leap to extensive industrial application. Previously, this lack of use could be attributed to the limitations of the technology. However, significant improvements have been made, and such systems are now capable of achieving recognition rates approaching those of traditional input devices. Therefore, we attribute the limited commercial use to shortfalls in two general areas of knowledge: (1) the lack of understanding of the true utility of these sensors with respect to human capabilities and needs; and (2) the composition of user interfaces required to facilitate their use. This research project is designed to broaden the current understanding of these two areas with the specific goal of providing fundamentally new user interfaces capable of supporting interaction using body-tracking technology. To achieve an understanding of human capabilities, we will apply empirical evaluation and ethnographic techniques to: 1) test and determine the limits of human abilities; 2) identify those capabilities that can best be mapped to the capabilities of the sensors; and 3) understand the contexts under which these capabilities are best leveraged. This will yield a more comprehensive understanding of two vital issues, i.e., where and when each input modality is best suited and what means of input is best suited to that context. From this understanding, we will construct user interfaces suitable for those contexts, allowing the user to provide input to devices in fundamentally new ways. The result will be a far greater understanding of the potential of these technologies, of human needs and capabilities, and ultimately of the utility and methods for the application of these technologies to industrial uses.