CCRI: Research Infrastructure: Planning-M: Multi-Modal Infrastructure for Enabling Social AI Research

用于理解社交交流的人工智能(AI)可以在家用机器人等重要应用领域发挥关键作用。然而，目前在自然语言处理和计算机视觉方面的进展主要集中在分析文本和视频的表层结构上，缺乏深入理解社交交流的能力。为了构建能够更好地理解社交互动的大规模人工智能系统，这个规划项目旨在开发一种流程，以负责任和可持续的方式收集和标注社交沟通的多模式和跨文化数据集。规划活动还包括围绕多模式社交数据收集组织研讨会和小组讨论，以表征这一新兴社交人工智能社区的需求。这些规划活动将为建立下一代社交人工智能模型提供基础，并确保数据的收集和使用符合道德和法律考虑。由此产生的数据基础设施将促进未来关于心理理论、团队合作和群体决策的研究。这个规划项目将展示自然语言处理、计算机视觉、社会科学、法律和伦理学之间的跨学科研究的潜力。该项目将通过以下网站托管：Social alai-data.org，在项目期间将在该网站上积极更新和维护数据、代码和数据使用协议。该奖项反映了NSF的法定使命，并已通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Artificial intelligence (AI) for understanding social communication can play a key role in important application domains such as household robotics. However, current advancements in natural language processing and computer vision mainly focus on analyzing the surface structure of text and videos, lacking the ability to understand social communication deeply. To build large-scale AI systems that can better understand social interaction, this planning project aims at how to develop a process for collecting and annotating multi-modal and cross-cultural datasets of social communication in a responsible and sustainable way. The planning activities also include organizing workshops and panels around multi-modal social data collection to characterize the needs of this emerging social AI community. These planning activities will provide the foundation towards building the next generation of social AI models and ensuring that collection and use of the data complies with ethical and legal considerations. The resulting data infrastructure will facilitate future research on theory of mind, teamwork, and group decision-making. This planning project will demonstrate the potential of interdisciplinary research between natural language processing, computer vision, social science, law, and ethics. This project will be hosted via this website: socialai-data.org, where data, code, and data use protocols will be actively updated and maintained during the project period.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.