CAREER: Developing Contextual Cues for Just-in-Time Information Retrieval on Wearable Computers

这个项目的假设是，随着越来越多的交互性和个性化的计算系统，即时信息检索代理（JITIR），主动检索和呈现信息的基础上，一个人的本地上下文在一个可访问的，但非侵入性的方式，并在可穿戴计算机上实现，将成为下一阶段的计算功能。 为了恢复用户的上下文，可穿戴计算机与用户的物理接近度将在设计从用户的角度“看”和“听”的感知系统时被利用。 为此，PI将创建一个可穿戴计算机研讨会，用于快速原型设计新的可穿戴传感器和系统。 虽然移动的环境对于实验室中开发的感知系统和算法来说是极其苛刻的，但是使用标准视频和音频进行输入的系统将是优选的，因为它们提供了诸如自动创建用于搜索的新媒体数据库、易于注释、传感器的可用性以及结果对其他领域的适用性等优点。 PI希望适应和利用其他领域先前研究的有希望的结果，包括人脸识别算法，手语手势识别，与对象交互或执行任务时涉及的手部运动，话语中的手势以及用户定位系统。 为了支持所需的基于模型的识别系统，一种新的形式的注释来自以前的工作在分析叙述将开发的数据收集在用户的日常生活。 最终，PI将创建一个由其系统的日常用户组成的社区，并进行研究，以确定JITIR的访问如何影响佩戴者对知识的使用。 PI预计这项工作将产生广泛的影响，包括开发全新的可定制可穿戴平台，该平台具有与新传感器技术接口的能力，旨在捕捉用户日常生活的传感器包，原型可穿戴人脸识别和手势识别系统，以及将传统信息检索算法扩展到松散定义的多媒体查询搜索。

The hypothesis underlying this project is that, with increasing interactivity and personalization of computing systems, just-in-time information retrieval agents (JITIRs) which proactively retrieve and present information based on a person's local context in an accessible yet non-intrusive manner, and which are implemented on wearable computers, will become a feature of the next stage of computing. To recover the user's context, the physical proximity of wearable computers to the user will be leveraged in designing perceptual systems that "see" and "hear" from the user's perspective. To these ends, the PI will create a Wearable Computer Workshop for rapid prototyping of new wearable sensors and systems. Although the mobile environment is extremely harsh for perceptual systems and algorithms that have been developed in the laboratory, systems that use standard video and audio for input will be preferred because they afford advantages such as automatic creation of new media databases for search, ease of annotation, availability of sensors, and applicability of results to other fields. The PI expects to adapt and exploit promising results from prior research in other domains, including face recognition algorithms, recognition of sign language gestures, hand movements involved in interacting with objects or performing tasks, gestures made in discourse, and user location systems. To support the desired model-based recognition systems, a new form of annotation derived from prior work in analyzing narration will be developed for the data collected during a user's everyday life. Ultimately, the PI will create a community of everyday users of his systems and perform studies to determine how access to JITIRs affects the wearer's use of knowledge. The PI expects this work to have broad impacts, including the development of radically new customizable wearable platforms with the ready ability to interface to new sensor technology, a sensor package directed at capturing a user's everyday life, prototype wearable face recognition and gesture recognition systems, and the expansion of traditional information retrieval algorithms to loosely defined multimedia query searches.