CAREER:Towards Perceptual Agents That See and Reason Like Humans

计算机视觉系统的最新进展使其能够在社交媒体，医疗保健，机器人和生态等领域广泛部署。虽然这些应用程序在改善我们的福祉和推进科学发现方面具有特殊的前景，但这些智能系统的普遍存在为其大规模采用带来了新的技术，社会和文化挑战。该项目领导研究，教学和推广的综合努力，以解决其中一些挑战。该项目开发的架构更加准确，能够从不同模态的感知数据中提取详细信息。这项工作的重点是开发计算机视觉系统，可以以人类可解释的方式对数据进行推理。此外，本研究还通过研讨会，促进多样性，让高中生、大学生、研究生参与到研究活动中来，并与产业界和生态学、生物学等领域的研究人员开展合作。本研究探索了提高图像数据、时空数据、深度数据的视觉感知和推理系统的能力的新方向。该研究开发了一类新的基于图形和因子分解的架构，用于3D形状和时空分析，与现有模型相比，这些架构在计算成本、内存开销和准确性之间提供了更好的权衡。该研究开发了弱监督技术，用于从大量未标记数据中学习形状和运动表示。该研究还开发了一类新的技术，用于将视觉数据转换为属性，自然语言和符号程序等语义表示。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Recent advancements in computer vision systems have enabled their widespread deployment in areas like social media, healthcare, robotics, and ecology, among many others. While such applications hold exceptional promise for improving our well-being and advancing scientific discovery, the ubiquity of these intelligent systems presents new technical, social, and cultural challenges for their wide-scale adoption. This project leads an integrated effort of research, teaching, and outreach to address some of these challenges. The project develops architectures that are substantially more accurate and capable of extracting detailed information from perceptual data across different modalities. An emphasis of this work is to develop computer vision systems that can reason about data in ways that are interpretable by humans. This project also promotes diversity, engages high school, undergraduate, and graduate students in research activities, and fosters collaborations with industry and researchers in areas such as ecology and biology through workshops.This research explores new directions that improve the capabilities of visual perception and reasoning systems for analyzing image data, spatio-temporal data, and depth data. The research develops a novel class of graph-based and factorized architectures for 3D shape and spatio-temporal analysis that provide better tradeoffs between computational cost, memory overhead, and accuracy than existing models. The research develops weakly supervised techniques for learning shape and motion representations from large amounts of unlabeled data. The research also develops a novel class of techniques for transforming visual data to semantic representations such as attributes, natural language, and symbolic programs. These techniques will improve the interpretability of machine learning models and enable collaborative learning and inference between humans and AI agents.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.