I-Corps: Nurture Neurodiversity

这个i-Corps项目的更广泛的影响/商业潜力是开发一种软件应用程序，可以被自闭症患者用来识别他们谈话伙伴的情绪。该软件可以被来自不同文化背景的人使用。这个为自闭症儿童提供的工具可能使他们能够理解和学习人们如何通过面部、肢体手势和语言表达不同的情绪，以提高他们成年后的沟通和互动技能。拟议的项目结合了语音语言病理学家和工程师的专业知识。这项技术及其商业化可能会使自闭症患者、他们的家庭成员和他们的治疗师受益。社交互动对自闭症患者来说可能是一种挑战。这个i-Corps项目寻求将机器和深度学习模型整合到一个应用程序中，该应用程序可以识别并提供检测到的情绪的表情反馈。自闭症儿童将使用这款应用程序来理解和识别与他们互动的人的情绪状态，作为识别情绪的发展工具。该应用程序使麦克风和摄像头能够进行必要的视觉和听觉输入。人脸表情模型将根据在不同光照条件和角度下训练的面部特征对情绪进行分类。身体-手势模型将识别身体动作的序列来分类人的情感表达，而语音-情感模型将提取频率特征来分类语音语调所呈现的情感。挑战是组装这些模型以提供准确的整体情绪分类，将机器和深度学习模型集成到代码开发中，并通过屏幕向儿童提供关于自闭症谱系的表情反馈。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a software application that can be used by people on the autism spectrum to identify the emotions of their conversational partners. The software can be used by people from different cultural backgrounds. This tool for children on the autism spectrum may enable them to understand and learn how people express different emotions through facial, body gestures, and speech to improve their communication and interaction skills into adulthood. The proposed project combines the expertise of speech-language pathologists and engineers. The technology and its commercialization may benefit people on the autism spectrum, their family members, and their therapists. Social interactions can be challenging for people on the autism spectrum. This I-Corps project seeks to integrate machine and deep learning models in a single app that recognizes and provides emoticon feedback of emotions detected. The app will be utilized by children on the autism spectrum to understand and recognize emotional states with people they interact with as a developmental tool in recognizing emotions. The app enables the microphone and camera for the necessary visual and auditory inputs. The facial expression model will classify the emotion based on the facial features, which was trained in different lighting conditions and angles. The body-gesture model will recognize the sequence of body movements to classify the emotional expression of the person and the speech-emotion model will extract frequency features to classify the emotion presented by the tone of speech. The challenge is to assemble these models to provide an accurate overall emotion classification, integrate machine and deep learning models to the code development, and provide emoticon feedback to the child on the autism spectrum through the screen.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.