User interface feedforward and feedback supporting and enabling body tracking technologies

点击翻译按钮获取中文摘要

Recent advances in computer vision have made it possible for users to provide input to systems in far more natural ways than before. Camera-based tracking of a user's body allows systems to recognize hand gestures, body posture changes, and movements. While these are impressive capabilities, such systems have failed to make the leap to extensive industrial application. Previously, this lack of use could be attributed to the limitations of the technology. However, significant improvements have been made, and such systems are now capable of achieving recognition rates approaching those of traditional input devices. Therefore, we attribute the limited commercial use to shortfalls in two general areas of knowledge: (1) the lack of understanding of the true utility of these sensors with respect to human capabilities and needs; and (2) the composition of user interfaces required to facilitate their use. This research project is designed to broaden the current understanding of these two areas with the specific goal of providing fundamentally new user interfaces capable of supporting interaction using body-tracking technology. To achieve an understanding of human capabilities, we will apply empirical evaluation and ethnographic techniques to: 1) test and determine the limits of human abilities; 2) identify those capabilities that can best be mapped to the capabilities of the sensors; and 3) understand the contexts under which these capabilities are best leveraged. This will yield a more comprehensive understanding of two vital issues, i.e., where and when each input modality is best suited and what means of input is best suited to that context. From this understanding, we will construct user interfaces suitable for those contexts, allowing the user to provide input to devices in fundamentally new ways. The result will be a far greater understanding of the potential of these technologies, of human needs and capabilities, and ultimately of the utility and methods for the application of these technologies to industrial uses.