ITR: Augmented Reality and Computer Vision for Enhancing Human Assembly Skills

这是一个为期三年的连续奖励的第一年资助。该项目旨在解决使用计算机视觉实现增强现实界面的基本问题。增强现实的目标是增强一个人对周围世界的感知，为计算机提供整合到用户活动中的潜力，作为个性化的助手。要实现这种接口，有两个关键挑战。第一个挑战是“感知”，这将使增强与用户与之交互时的世界状态相匹配;第二个挑战是开发系统的“增强方案”，从而产生以用户为中心的信息流。为了帮助概念化的问题和实验验证，该项目的主要重点是一个“组装域”。这是一个人从零部件组装机械物体的场景。集中在“组装域”，使我们能够适当地制定和解决的传感，增强，和其他问题，在新的人机界面。同时，它允许我们使用增强现实来检查特定的交互式装配任务，例如装配过程中的指导和培训以及原型装配序列的评估。 为了解决跟踪“上下文”的关键问题，该项目旨在推进用于识别装配状态的计算机视觉技术的发展。基于外观和基于CAD的方法相结合，将用于解决同时跟踪大量已知的装配部件的问题。提出了一种概率方法，以提高性能的装配状态恢复随着时间的推移，装配任务的进展。另一个重点是寻找有效的方法来建立模型空间的零件数量更大。装配域的几何建模和分析将用于系统增强流程的开发，以帮助完成各种装配任务。将使用两个实验装置，包括一个透明的头戴式显示器和一个计算机显示器，图像覆盖在实时视频上。具体的装配任务场景将被设计，以确定增强现实接口的装配域的实际可行性。 另一个目标是通过实验评估增强现实界面作为培训工具与其他指令手段相比的有效性。由此产生的增强现实界面预计将影响许多应用程序，这些应用程序受益于多模态信息的在线，场景相关的呈现，例如装配原型设计，飞机维护，航天器维修，工厂工人的成本效益培训，以及指导没有经验的用户进行复杂的机械维修。

This is the first year funding of a three-year continuing award. This project aims at addressing basic issues for enabling an augmented reality interface using computer vision. Augmented Reality has the goal of enhancing a person's perception of the surrounding world, offering the potential for the computer to be integrated into the activities of a user, serving as a personalized helper. There are two key challenges to enable such an interface. The first is "sensing", which would allow the augmentation to be matched to the state of the world as the user interacts with it. The second challenge is that of developing systematic "augmentation schemes" that result in user-centered information flow. To aid in the conceptualization of the problem and for experimental verification, the main focus of the project is on an "assembly domain". The context is that of a human engaged in assembling a mechanical object from its components. The focus on the "assembly domain", allows us to suitably formulate and address the sensing, augmentation, and other issues in the novel human-computer interface. At the same time, it allows us to examine specific interactive assembly tasks using augmented reality, such as guiding and training during assembly and for evaluation of prototype assembly sequences. To address the key problem of tracking the "context", the project seeks to advance the state of art of computer vision techniques for recognizing assembly states. A combination of appearance-based and CAD-based approach will be used for addressing the problem of simultaneously tracking a large number of known assembly parts. A probabilistic approach is proposed to improve the performance of assembly state recovery over time as the assembly task progresses. Another focus is to look for efficient approaches for building the model spaces for subassemblies with larger number of parts. Geometric modeling and analysis of the assembly domain will be utilized in the development of systematic flow of augmentation to aid various assembly tasks. Two experimental setups will be used, involving a see-through head-mounted display and a computer monitor with graphics overlaid on live video. Specific assembly task scenarios will be devised to determine the practical feasibility of augmented reality interface for the assembly domain. Another goal will be to experimentally evaluate the effectiveness of augmented reality interface as a training tool compared to other means of instructions. The resulting augmented reality interface is expected to impact many applications that benefit from the on-line, scene-dependent presentation of multimodal information, such as assembly prototyping, aircraft maintenance, repair of space vehicles, cost-effective training of factory workers, and perhaps guiding inexperienced users through complex repair of machinery.