SGER - Vision and RFID for Multimodal Tracking of Working Teams

该项目提出了在拥挤的空间中自动捕获团队合作的新策略。最初的目标是开发新的方法，使用不同的方式，包括机器视觉和射频识别（RFID）的人和工件的位置采集。接下来，将开发软件，以整合这些观察结果，并随着时间的推移跟踪团队成员和工件。最终，概率推理将用于根据这些统一的观察来确定团队任务。创伤复苏的代表性领域是这项工作的理想环境，因为球员的角色和任务是明确定义的，工作流程遵循一个一般的模式，无论病人的伤害。该系统将使用机器人人体模型患者在模拟创伤场景中进行测试。这项工作有两个主要好处。首先，计算机化的团队支持系统的系统需求的推导过程中需要分析大量的观察当前的做法。自动转录和标记团队协作将有助于有效地捕捉和解释事件，比专家进行的更繁琐和容易出错的观察更可取。第二，团队活动的自动跟踪是开发“智能室”的必要的第一步，智能室为团队合作提供计算机化支持。该项目的核心贡献将是一个概念验证系统，该系统集成了使用计算机视觉和RFID跟踪的演员和医疗对象的跟踪。所提出的方法将开发新的算法和方法，用于（i）在拥挤的协作环境中进行基于视觉的人员跟踪，以及（ii）融合来自多模态传感器的不可靠数据，以实现对人类活动的可靠识别。更广泛的影响这项工作的科学重要性在于需要标记视频观察。许多形式的视频都是重复的行为，无论是在监控应用程序中，工作情况下，还是其他用途。在所有这些情况下，将语法应用于视频，并将动作和声音与该语法相匹配，有可能大大简化工作分析的工作-这是计算机支持复杂，高风险人类活动的开发过程中的关键阶段。这项工作还将为在缺乏有效的仪器跟踪工作方法的环境中（如创伤复苏和相关医疗领域）实施决策辅助提供基础。

This project proposes novel strategies for the automatic capture of teamwork in crowded spaces. The initial goal is to develop new methods for acquisition of people and artifact locations using diverse modalities, including machine vision and radiofrequency identification (RFID). Following, software will be developed in order to integrate these observations and track team members and artifacts over time. Ultimately probabilistic reasoning will be used to identify team tasks based on these unified observations. The representative domain of trauma resuscitation is an ideal environment for this work since the roles and tasks of players are well-defined and the flow of work follows a general schema regardless of the patient's injuries. The system will be tested in simulated trauma scenarios using a robotic mannequin patient. There are two key benefits of this work. First, the process of deriving system requirements for computerized teamwork support systems demands analyzing a large number of observations of current practices. Automatic transcription and tagging of teamwork will allow for efficient capture and interpretation of events and is preferable to more tedious and error-prone observations by experts. Second, automatic tracking of team activities is a needed initial step in the development of "smart rooms" that provide computerized support of teamwork. The core contribution of this project will be a proof-of-concept system integrating tracking of actors and medical objects using computer vision and RFID tracking. The proposed approach will develop novel algorithms and methods for (i) vision-based person tracking in crowded collaborative environments and (ii) fusion of unreliable data from multimodal sensors to achieve reliable recognition of human activities. Broader ImpactsThe scientific importance of this work is in the need to tag video observations. Many forms of videos are of repetitive behaviors, whether in surveillance applications, work situations, or other uses. In all such cases, applying a grammar to the video, and matching actions and sounds to that grammar, has the possibility of greatly simplifying the job of work analysis - a critical phase in the development process of computer support for complex, high-risk human activities. This work will also provide the foundation for implementing decision aids in environments such as trauma resuscitation and related medical domains that lack effective methods for instrument tracking of work.