MRI: Development of Real-time 3D Social Signal Imaging System (SSIS)

MRI：实时 3D 社交信号成像系统 (SSIS) 的开发

• 批准号: 1919965
• 负责人: Hyun Soo Park
• 金额: $ 55万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1919965&HistoricalAwards=false
• 关键词: MRI; Development; Real; time; 3D

This project represents a step toward a computational model capable of detecting early social behavioral markers in children at risk for autism spectrum disorder, schizophrenia, and obsessive-compulsive disorder. The real-time 3D Social Signal Imaging System (SSIS) will be designed to precisely measure social signals utilizing cameras producing billions of pixels dozens of times per second. The infrastructure will be designed to enable reconstruction of the 3D geometry of gaze, face, finger, body, and physical appearance. The system is expected to be capable of generating a vast amount of multiple perspective visual data to reconstruct high fidelity 3D signals, needed to enable social intelligence that can decode every nuance of human expression.The ability to discern subtle social signals (e.g., gaze following) can be computationally modeled by leveraging a massive camera system. The Social Signal Imaging System (SSIS) facilitates quantitative measurements of the social signals in 3D at unprecedented temporal and spatial resolutions. This development involves the following steps: (i) Design a distributed visual computing architecture to efficiently process the Multiview visual data streams; (ii) Build a new high-fidelity 3D representation of the view-invariant social signals (gaze, face, finger, body, appearance); (iii) Create a novel 3D dataset of social signals for use in discovering behavioral markers; and (iv) Develop new computer vision algorithms (recognition, matching, tracking, reconstruction) tailored to social signal imaging that minimize computational latency while maintaining accuracy. The system provides a unique characterization of microscopic social signals that enable overcoming fundamental limitations of existing approaches in behavioral assessment of at-risk children. This work impacts diverse disciplines such as robotics, neuroscience, psychology, psychiatry, and medicine. The outcomes will be disseminated through K-12 students from under-represented groups via workshops, machine learning and technology summer camps, and other activities.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目代表了一个计算模型，能够检测自闭症谱系障碍，精神分裂症和强迫症风险儿童的早期社会行为标志物。实时3D社会信号成像系统（SSIS）将被设计用于精确测量社会信号，利用每秒数十次产生数十亿像素的相机。该基础设施将被设计为能够重建凝视、面部、手指、身体和身体外观的3D几何形状。该系统预计能够生成大量的多视角视觉数据，以重建高保真3D信号，这是实现能够解码人类表达的每一个细微差别的社会智能所需要的。注视跟随）可以通过利用大规模相机系统来计算建模。社会信号成像系统（SSIS）有助于以前所未有的时间和空间分辨率对3D社会信号进行定量测量。该开发涉及以下步骤：（i）设计分布式视觉计算架构以有效地处理多视图视觉数据流;（ii）构建新的视图不变社会信号的高保真3D表示（iii）创建用于发现行为标记的社交信号的新颖3D数据集;以及（iv）开发新的计算机视觉算法（识别、匹配、跟踪、重建），以适应社会信号成像，最大限度地减少计算延迟，同时保持准确性。 该系统提供了一个独特的表征微观社会信号，使克服现有的方法在行为评估的风险儿童的根本局限性。这项工作影响了不同的学科，如机器人，神经科学，心理学，精神病学和医学。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。