CAREER: Developing Contextual Cues for Just-in-Time Information Retrieval on Wearable Computers

该项目的假设是，随着计算系统的互动性和个性化的增加，即时信息检索代理(JITIR)将成为下一阶段计算的一个特征，JITIR以可访问但非侵入性的方式根据人的本地上下文主动检索和呈现信息，并在可穿戴计算机上实现。为了恢复用户的环境，可穿戴计算机与用户的物理接近程度将被用于设计从用户的角度来“看”和“听”的感知系统。为此，PI将创建一个可穿戴计算机研讨会，用于快速制作新的可穿戴传感器和系统的原型。尽管移动环境对实验室开发的感知系统和算法极其苛刻，但使用标准视频和音频进行输入的系统将是首选，因为它们提供的优势包括自动创建用于搜索的新媒体数据库、易于注释、传感器的可用性以及结果可应用于其他领域。PI希望适应和利用之前在其他领域的研究成果，包括人脸识别算法、手语手势识别、与物体互动或执行任务的手部动作、话语中的手势以及用户定位系统。为了支持所需的基于模型的识别系统，将为用户日常生活中收集的数据开发一种新的注释形式，该形式来自于分析叙事的先前工作。最终，PI将创建他的系统的日常用户社区，并进行研究以确定访问JITIR如何影响佩戴者对知识的使用。PI预计这项工作将产生广泛的影响，包括开发全新的可定制的、具有与新传感器技术接口的能力的可穿戴平台、旨在捕捉用户日常生活的传感器包、可穿戴人脸识别和手势识别系统的原型，以及将传统信息检索算法扩展到定义松散的多媒体查询搜索。

The hypothesis underlying this project is that, with increasing interactivity and personalization of computing systems, just-in-time information retrieval agents (JITIRs) which proactively retrieve and present information based on a person's local context in an accessible yet non-intrusive manner, and which are implemented on wearable computers, will become a feature of the next stage of computing. To recover the user's context, the physical proximity of wearable computers to the user will be leveraged in designing perceptual systems that "see" and "hear" from the user's perspective. To these ends, the PI will create a Wearable Computer Workshop for rapid prototyping of new wearable sensors and systems. Although the mobile environment is extremely harsh for perceptual systems and algorithms that have been developed in the laboratory, systems that use standard video and audio for input will be preferred because they afford advantages such as automatic creation of new media databases for search, ease of annotation, availability of sensors, and applicability of results to other fields. The PI expects to adapt and exploit promising results from prior research in other domains, including face recognition algorithms, recognition of sign language gestures, hand movements involved in interacting with objects or performing tasks, gestures made in discourse, and user location systems. To support the desired model-based recognition systems, a new form of annotation derived from prior work in analyzing narration will be developed for the data collected during a user's everyday life. Ultimately, the PI will create a community of everyday users of his systems and perform studies to determine how access to JITIRs affects the wearer's use of knowledge. The PI expects this work to have broad impacts, including the development of radically new customizable wearable platforms with the ready ability to interface to new sensor technology, a sensor package directed at capturing a user's everyday life, prototype wearable face recognition and gesture recognition systems, and the expansion of traditional information retrieval algorithms to loosely defined multimedia query searches.