SGER: Social-Emotional Intelligence Prosthetic

SGER：社交情绪智力假肢

• 批准号: 0555411
• 负责人: Rosalind Picard
• 金额: --
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2005
• 资助国家: 美国
• 起止时间: 2005-11-01 至 2007-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0555411&HistoricalAwards=false
• 关键词: SGER; Social; Emotional; Intelligence; Prosthetic

This research will develop the first known wearable device capable of perceiving and reporting on social-emotional information in real-time human interaction. Using a small wearable camera and customized video and pattern analysis algorithms, the system will automatically read and interpret the facial expressions and head movements of the person the wearer is interacting with and will communicate its inferences to the wearer via sound or tactile feedback. The system builds on and extends the computational model of "mind-reading" developed by Rana el Kaliouby, which forms the basis of an automated system for the inference of mental states from head and facial displays in a video stream in real time. Bringing together and expanding upon recent advances and methodologies in three key scientific and technical areas - affective computing, wearable computing, and real time machine perception - the project will significantly advance the nascent ability of machines to infer cognitive-affective states in real-time from nonverbal expressions of people engaged in a range of natural interactions. The proposed work is for the first prototype of what is expected to become a sequence of increasingly intelligent prosthetics. The design and strategy will open up new lines of inquiry in understanding the role of social-emotional cues in natural interaction. The automated perception of nonverbal cues and inference of mental states are problems that continue to challenge researchers in the domains of machine vision and machine learning. These problems become even more complex if the system is required to execute in real-time and to generalize to novel situations and people, a necessary requirement in the case of an assistive technology. The project addresses open research questions pertaining to whether machine perception can be reliable enough and comfortable enough to augment social interactions in a way that improves human to human communication. The challenge is further complicated by the requirement of operating in a mobile environment with varied lighting conditions and limited processing and power. Being the first prosthetic assistive technology that is designed to augment a person's impairment in socio-emotional intelligence skills, the project has the potential to lead to a line of new research. A system with social-emotional perceptual abilities can potentially assist the growing number of individuals diagnosed with Autism Spectrum Disorder, in perceiving communication in a natural rather than a structured environment, bootstrapping their ability to learn and develop in social settings. The system will also provide a new tool for researchers on collaborative systems to more carefully examine the communication of nonverbal cues in natural everyday communication. The results of this interdisciplinary work can be leveraged in human-computer interaction, wearable computing, robotics and in future technologies with social-emotional intelligence. Results will be distributed in all of these communities through peer-reviewed publications and conference presentations, as well as through forums for attracting more minorities to scientific and technical fields. This work promotes the training and education of students by deeply involving them in the proposed research activities, with a commitment to growing the participation of underrepresented minorities in cutting-edge scientific research

这项研究将开发第一个已知的可穿戴设备，能够感知和报告实时人类互动中的社交情感信息。使用小型可穿戴相机和定制的视频和模式分析算法，系统将自动读取和解释与佩戴者互动的人的面部表情和头部动作，并通过声音或触觉反馈将其推论传达给佩戴者。该系统建立在并扩展了 Rana el Kaliouby 开发的“读心术”计算模型的基础上，该模型构成了根据视频流中的头部和面部显示实时推断心理状态的自动化系统的基础。该项目汇集并扩展了情感计算、可穿戴计算和实时机器感知这三个关键科学技术领域的最新进展和方法，将显着提高机器从参与一系列自然交互的人们的非语言表达中实时推断认知情感状态的新生能力。拟议的工作是第一个原型，预计将成为一系列日益智能的假肢。 该设计和策略将为理解社会情感线索在自然互动中的作用开辟新的探究途径。非语言线索的自动感知和心理状态的推断是机器视觉和机器学习领域继续挑战研究人员的问题。如果系统需要实时执行并推广到新的情况和人员（对于辅助技术来说这是必要的要求），这些问题就会变得更加复杂。该项目解决了一些开放性的研究问题，即机器感知是否足够可靠、足够舒适，能够以改善人与人之间沟通的方式增强社交互动。由于需要在照明条件变化且处理能力和功率有限的移动环境中运行，这一挑战变得更加复杂。 作为第一个旨在增强人的社会情绪智力技能损伤的假肢辅助技术，该项目有可能引发一系列新的研究。 具有社交情感感知能力的系统可以帮助越来越多的被诊断患有自闭症谱系障碍的人在自然环境而不是结构化环境中感知交流，从而增强他们在社交环境中学习和发展的能力。该系统还将为协作系统的研究人员提供一个新工具，以更仔细地检查自然日常交流中非语言线索的交流。这项跨学科工作的成果可用于人机交互、可穿戴计算、机器人技术以及具有社交情感智能的未来技术。结果将通过同行评审的出版物和会议演示以及通过吸引更多少数群体进入科学和技术领域的论坛在所有这些社区中分发。这项工作通过让学生深入参与拟议的研究活动来促进学生的培训和教育，致力于增加代表性不足的少数群体对尖端科学研究的参与