I-Corps: Nurture Neurodiversity

这个I-Corps项目更广泛的影响/商业潜力是开发一个软件应用程序，可以被自闭症谱系的人用来识别他们的对话伙伴的情绪。该软件可供来自不同文化背景的人使用。这种针对自闭症儿童的工具可以使他们了解和学习人们如何通过面部，身体姿势和言语表达不同的情绪，以提高他们的沟通和互动技能。 拟议的项目结合了语音语言病理学家和工程师的专业知识。这项技术及其商业化可能会使自闭症患者、他们的家庭成员和他们的治疗师受益。社会交往对自闭症谱系的人来说可能是一个挑战。这个I-Corps项目旨在将机器和深度学习模型集成到一个应用程序中，该应用程序可以识别并提供检测到的情绪的表情反馈。该应用程序将被自闭症儿童用来理解和识别与他们互动的人的情绪状态，作为识别情绪的发展工具。该应用程序启用麦克风和摄像头，以进行必要的视觉和听觉输入。人脸表情模型将根据不同光照条件和角度下训练的人脸特征对情绪进行分类。 身体-手势模型将识别身体动作序列以分类人的情感表达，并且语音-情感模型将提取频率特征以分类由语音语调呈现的情感。挑战是将这些模型组合起来，提供准确的整体情感分类，将机器和深度学习模型集成到代码开发中，并通过屏幕向自闭症谱系儿童提供表情反馈。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

The broader impact/commercial potential of this I-Corps project is the development of a software application that can be used by people on the autism spectrum to identify the emotions of their conversational partners. The software can be used by people from different cultural backgrounds. This tool for children on the autism spectrum may enable them to understand and learn how people express different emotions through facial, body gestures, and speech to improve their communication and interaction skills into adulthood. The proposed project combines the expertise of speech-language pathologists and engineers. The technology and its commercialization may benefit people on the autism spectrum, their family members, and their therapists. Social interactions can be challenging for people on the autism spectrum. This I-Corps project seeks to integrate machine and deep learning models in a single app that recognizes and provides emoticon feedback of emotions detected. The app will be utilized by children on the autism spectrum to understand and recognize emotional states with people they interact with as a developmental tool in recognizing emotions. The app enables the microphone and camera for the necessary visual and auditory inputs. The facial expression model will classify the emotion based on the facial features, which was trained in different lighting conditions and angles. The body-gesture model will recognize the sequence of body movements to classify the emotional expression of the person and the speech-emotion model will extract frequency features to classify the emotion presented by the tone of speech. The challenge is to assemble these models to provide an accurate overall emotion classification, integrate machine and deep learning models to the code development, and provide emoticon feedback to the child on the autism spectrum through the screen.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.