CHS: Medium: Collaborative Research: Activity Recognition for Reducing Delays in Fast-Response Teamwork

CHS：中：协作研究：减少快速响应团队合作延迟的活动识别

• 批准号: 1763827
• 负责人: Ivan Marsic
• 金额: $ 70万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1763827&HistoricalAwards=false
• 关键词: CHS; Medium; Collaborative; Research; Activity

Human performance in time-critical teamwork settings relies on appropriate and timely task completion. Time perception, a critical cognitive function that influences team performance, is often skewed in these settings and is impacted by cognitive workload. This project will address timeliness errors by automatically and unobtrusively modeling and tracking cross-disciplinary task performance through analysis of verbal communication and the use of information artifacts. The resulting model will be used to display alerts about timeliness of critical tasks in a way that supports team members' information needs without increasing their cognitive workload. The benefits and costs of this approach will be evaluated in a simulation setting, measuring its impact on team performance and overall goal accomplishment, as well as its impact on workload and distraction. The application domain for this research is trauma resuscitation, the early evaluation and management of injured patients in the emergency department; the goal is for increased temporal awareness to improve both trauma team efficiency and patient outcomes, saving money and lives. Further, the project will provide opportunities for interdisciplinary education involving students from computer science and medicine. This project will develop techniques for monitoring the progress of teamwork and displaying alerts about timeliness of critical tasks. The key system components will include recognizing activities, modeling process deviations and delays, and displaying process information in a way that does not divert attention from the work. Novel techniques for activity recognition in fast-paced and crowded collaborative settings will be based on passive RFID, speech recognition, and computer vision, supplemented by other sensors and digital devices. The proposed research will develop (1) temporal models of verbal communication and digital document interaction during complex activities in fast-paced teamwork for the purpose of automatic activity recognition; (2) approaches for real-time recognition of over a hundred different activities in the presence of up to a thousand RFID tags; and (3) approaches for displaying delay information for rapid assimilation under high task load that learn from workers' responses to improve their usefulness to the team. Together, the work will provide building blocks for computerized support of teams not just in the trauma domain, but in other domains with complex, interleaved tasks such as surgery, traffic control, and disaster management.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

时间紧迫的团队合作环境中的人员绩效依赖于适当且及时的任务完成。时间感知是影响团队绩效的关键认知功能，在这些环境中通常会出现偏差，并受到认知工作量的影响。 该项目将通过分析口头交流和使用信息工件，自动且不引人注目地建模和跟踪跨学科任务绩效，从而解决及时性错误。由此产生的模型将用于显示有关关键任务及时性的警报，以支持团队成员的信息需求，而不增加他们的认知工作量。该方法的收益和成本将在模拟环境中进行评估，衡量其对团队绩效和总体目标实现的影响，以及对工作量和分心的影响。本研究的应用领域是创伤复苏、急诊科受伤患者的早期评估和管理；目标是提高时间意识，提高创伤团队的效率和患者的治疗效果，从而节省金钱和生命。 此外，该项目还将为计算机科学和医学专业的学生提供跨学科教育的机会。该项目将开发用于监控团队合作进度并显示关键任务及时性警报的技术。关键的系统组件将包括识别活动、对流程偏差和延迟进行建模，以及以不分散工作注意力的方式显示流程信息。在快节奏和拥挤的协作环境中进行活动识别的新技术将基于无源 RFID、语音识别和计算机视觉，并辅以其他传感器和数字设备。拟议的研究将开发（1）快节奏团队合作中复杂活动期间口头交流和数字文档交互的时间模型，以实现自动活动识别； (2) 在存在多达一千个 RFID 标签的情况下实时识别一百多种不同活动的方法； (3) 显示延迟信息的方法，以便在高任务负荷下快速同化，从工人的反应中学习，以提高他们对团队的有用性。 总之，这项工作将为团队的计算机化支持提供基础，不仅在创伤领域，而且在具有复杂、交叉任务的其他领域，如手术、交通控制和灾害管理。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Real-time Context-Aware Multimodal Network for Activity and Activity-Stage Recognition from Team Communication in Dynamic Clinical Settings

实时上下文感知多模态网络，用于动态临床环境中团队沟通的活动和活动阶段识别

• DOI: 10.1145/3580798
• 发表时间: 2022
• 期刊: Wearable and Ubiquitous Technologies
• 作者: Gao, Chenyang;Marsic, Ivan;Sarcevic, Aleksandra;Gestrich-Thompson, Waverly;Burd, Randall S.
• 通讯作者: Burd, Randall S.

TubeR: Tubelet Transformer for Video Action Detection

• DOI: 10.1109/cvpr52688.2022.01323
• 发表时间: 2021-04
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Jiaojiao Zhao;Yanyi Zhang;Xinyu Li;Hao Chen;Shuai Bing;Mingze Xu;Chunhui Liu;Kaustav Kundu;Yuanjun Xiong;Davide Modolo;I. Marsic;Cees G. M. Snoek;Joseph Tighe

Improving Label Assignments Learning by Dynamic Sample Dropout Combined with Layer-wise Optimization in Speech Separation

• DOI: 10.21437/interspeech.2023-1172
• 发表时间: 2023-08
• 作者: Chenyu Gao;Yue Gu;I. Marsic