SGER - Vision and RFID for Multimodal Tracking of Working Teams

该项目提出了在拥挤空间中自动捕获团队合作的新策略。最初的目标是开发新的方法来获取人员和工件的位置，使用不同的模式，包括机器视觉和射频识别（RFID）。接下来，将开发软件，以便随着时间的推移集成这些观察结果和田径队成员和工件。最终，概率推理将用于基于这些统一的观察来确定团队任务。创伤复苏的代表性领域是这项工作的理想环境，因为参与者的角色和任务是明确定义的，工作流程遵循一般模式，无论患者的受伤情况如何。该系统将在模拟创伤场景中使用机器人人体模型进行测试。这项工作有两个主要好处。首先，导出计算机化团队协作支持系统的系统需求的过程需要分析大量当前实践的观察结果。团队合作的自动转录和标记将允许有效地捕获和解释事件，并且比专家更繁琐和容易出错的观察更可取。第二，团队活动的自动跟踪是开发“智能房间”所需的第一步，它为团队合作提供计算机支持。该项目的核心贡献将是一个概念验证系统，该系统将使用计算机视觉和RFID跟踪集成对演员和医疗对象的跟踪。提出的方法将开发新的算法和方法，用于(i)在拥挤的协作环境中基于视觉的人员跟踪和（ii）融合来自多模态传感器的不可靠数据，以实现对人类活动的可靠识别。更广泛的影响这项工作的科学重要性在于需要标记视频观测。许多形式的视频都是重复的行为，无论是在监控应用、工作环境还是其他用途中。在所有这些情况下，将语法应用于视频，并将动作和声音与该语法相匹配，有可能大大简化工作分析的工作-这是计算机支持复杂，高风险人类活动的开发过程中的关键阶段。这项工作还将为在创伤复苏和相关医疗领域等缺乏有效仪器跟踪工作方法的环境中实施决策辅助提供基础。

This project proposes novel strategies for the automatic capture of teamwork in crowded spaces. The initial goal is to develop new methods for acquisition of people and artifact locations using diverse modalities, including machine vision and radiofrequency identification (RFID). Following, software will be developed in order to integrate these observations and track team members and artifacts over time. Ultimately probabilistic reasoning will be used to identify team tasks based on these unified observations. The representative domain of trauma resuscitation is an ideal environment for this work since the roles and tasks of players are well-defined and the flow of work follows a general schema regardless of the patient's injuries. The system will be tested in simulated trauma scenarios using a robotic mannequin patient. There are two key benefits of this work. First, the process of deriving system requirements for computerized teamwork support systems demands analyzing a large number of observations of current practices. Automatic transcription and tagging of teamwork will allow for efficient capture and interpretation of events and is preferable to more tedious and error-prone observations by experts. Second, automatic tracking of team activities is a needed initial step in the development of "smart rooms" that provide computerized support of teamwork. The core contribution of this project will be a proof-of-concept system integrating tracking of actors and medical objects using computer vision and RFID tracking. The proposed approach will develop novel algorithms and methods for (i) vision-based person tracking in crowded collaborative environments and (ii) fusion of unreliable data from multimodal sensors to achieve reliable recognition of human activities. Broader ImpactsThe scientific importance of this work is in the need to tag video observations. Many forms of videos are of repetitive behaviors, whether in surveillance applications, work situations, or other uses. In all such cases, applying a grammar to the video, and matching actions and sounds to that grammar, has the possibility of greatly simplifying the job of work analysis - a critical phase in the development process of computer support for complex, high-risk human activities. This work will also provide the foundation for implementing decision aids in environments such as trauma resuscitation and related medical domains that lack effective methods for instrument tracking of work.