CAREER: Developing Contextual Cues for Just-in-Time Information Retrieval on Wearable Computers

这个项目背后的假设是，随着计算系统的交互性和个性化的增加，即时信息检索代理（jit）将以一种可访问但非侵入的方式主动检索和呈现基于个人本地环境的信息，并在可穿戴计算机上实现，将成为下一阶段计算的一个特征。为了恢复用户的环境，可穿戴计算机与用户的物理接近性将被用于设计从用户角度“看”和“听”的感知系统。为此，PI将创建一个可穿戴计算机车间，用于快速制作新的可穿戴传感器和系统的原型。尽管移动环境对于在实验室中开发的感知系统和算法来说是极其苛刻的，但使用标准视频和音频作为输入的系统将是首选，因为它们具有诸如自动创建搜索新媒体数据库，易于注释，传感器可用性以及结果对其他领域的适用性等优势。PI希望适应和利用其他领域先前研究的有希望的结果，包括人脸识别算法、手语手势识别、与物体交互或执行任务时涉及的手部运动、话语中的手势和用户定位系统。为了支持所需的基于模型的识别系统，将为用户日常生活中收集的数据开发一种新的注释形式，这种注释来源于先前分析叙述的工作。最终，PI将创建一个由其系统的日常用户组成的社区，并进行研究，以确定访问JITIRs如何影响佩戴者对知识的使用。PI预计这项工作将产生广泛的影响，包括开发全新的可定制可穿戴平台，该平台具有与新传感器技术接口的能力，用于捕获用户日常生活的传感器包，原型可穿戴面部识别和手势识别系统，以及将传统信息检索算法扩展到松散定义的多媒体查询搜索。

The hypothesis underlying this project is that, with increasing interactivity and personalization of computing systems, just-in-time information retrieval agents (JITIRs) which proactively retrieve and present information based on a person's local context in an accessible yet non-intrusive manner, and which are implemented on wearable computers, will become a feature of the next stage of computing. To recover the user's context, the physical proximity of wearable computers to the user will be leveraged in designing perceptual systems that "see" and "hear" from the user's perspective. To these ends, the PI will create a Wearable Computer Workshop for rapid prototyping of new wearable sensors and systems. Although the mobile environment is extremely harsh for perceptual systems and algorithms that have been developed in the laboratory, systems that use standard video and audio for input will be preferred because they afford advantages such as automatic creation of new media databases for search, ease of annotation, availability of sensors, and applicability of results to other fields. The PI expects to adapt and exploit promising results from prior research in other domains, including face recognition algorithms, recognition of sign language gestures, hand movements involved in interacting with objects or performing tasks, gestures made in discourse, and user location systems. To support the desired model-based recognition systems, a new form of annotation derived from prior work in analyzing narration will be developed for the data collected during a user's everyday life. Ultimately, the PI will create a community of everyday users of his systems and perform studies to determine how access to JITIRs affects the wearer's use of knowledge. The PI expects this work to have broad impacts, including the development of radically new customizable wearable platforms with the ready ability to interface to new sensor technology, a sensor package directed at capturing a user's everyday life, prototype wearable face recognition and gesture recognition systems, and the expansion of traditional information retrieval algorithms to loosely defined multimedia query searches.