CHS:Small: A Kinder, Gentler Technology: Enhancing Human-Machine Symbiosis Using Adaptive, Personalized Affect-Aware Systems

人工智能的一个长期目标是开发与人类良好互动的智能系统。 传感和机器学习的进步越来越多地允许计算机推断精神状态，这引发了关于代理如何使用这些推断来适应人类伴侣的问题。 该项目将系统地解决如何设计和评估“情感感知”系统，根据对用户情感体验的估计来调整其行为。 该团队将首先研究当前策略的有效性，这些策略根据推断的影响来改变教育任务和游戏的难度。 然后，他们将制定新的战略，考虑到个人的个性和物理环境的动态特性。 最后，他们将评估这些策略，特别注意当系统根据关于影响的不正确推断采取行动时会发生什么。 这些研究将有助于为自动驾驶汽车、会话助理和虚拟现实角色铺平道路，这些角色在与人互动时会考虑影响，理想情况下会带来更好的体验和结果。 该团队还将开发人的因素和人机交互方面的新的跨学科课程，与工业合作伙伴建立联系，帮助培养学生对这些自适应系统的实践和研究。 此外，他们将对这些系统进行公共宣传，并利用它们为K-12和社区大学的学生提供夏季研究经验，重点关注那些传统上在计算方面代表性不足的群体。使用空间认知游戏和机器人辅助的运动康复任务作为测试平台，允许团队直接操纵任务难度并测量乐趣。参与和绩效/学习成果。 该团队将首先收集训练数据，其中人们使用随机选择的难度水平的测试平台，并报告感知的难度水平为太容易（无聊），太难（沮丧）或大约正确，同时捕获心率信号，皮肤电导和温度，脑电图（EEG）数据以及环境因素，包括光线，一天中的时间和室温。 这些将用于使用各种机器学习方法训练情感识别器：线性判别分析（包括卡尔曼自适应版本），支持向量机，神经和贝叶斯网络以及随机森林。 使用一种通用的适应策略，将难度向上或向下调整一步，该团队将测量影响意识识别器在考虑和不考虑环境因素的情况下所实现的享受和性能结果，并将其与仅基于任务性能来适应难度的基线策略进行比较。 在这些实验中，该团队还将收集有关用户个性特征的数据，并利用这些数据开发针对不同个性类型的个性化识别模型和适应策略。这些个性化的模型和策略将通过与第一次实验的基线数据进行比较来进行评估。 最后，他们将这些系统的结果与人类控制的“最佳情况”系统和随机选择适应策略的“最坏情况”易出错系统的结果进行比较，观察这些诱导错误率沿着其他实验中捕获的自然错误率，以确定识别和适应错误对满意度和任务结果的影响。

A longstanding goal in artificial intelligence is to develop smart systems that interact well with humans. Advances in sensing and machine learning are increasingly allowing computers to infer mental states, raising questions about how agents might use those inferences to adapt to human partners. This project will systematically address how to design and evaluate "affect-aware" systems that adapt their behavior based on estimates of their users' emotional experiences. The team will first look at the effectiveness of current strategies that vary the difficulty of educational tasks and games based on inferred affect. They will then develop new strategies that take into account both individual personality and dynamic characteristics of the physical environment. Finally, they will evaluate these strategies, paying particular attention to what happens when systems act on incorrect inferences about affect. These studies will help pave the way toward self-driving cars, conversational assistants, and virtual reality characters that consider affect when interacting with people, ideally leading to better experiences and outcomes. The team will also develop new interdisciplinary courses in human factors and human-computer interaction, connecting with industrial partners to help train students in both the practice and research of these kinds of adaptive systems. Further, they will do public outreach about these systems and use them to provide summer research experiences for K-12 and community college students, focusing on those from groups traditionally underrepresented in computing.The project will be structured as a series of lab studies, using spatial cognition games and robot-assisted motor rehabilitation tasks as testbeds that allow the team to directly manipulate task difficulty and measure enjoyment/engagement and performance/learning outcomes. The team will first collect training data with people using the testbeds at randomly selected difficulty levels and reporting the perceived level of difficulty as too easy (bored), too hard (frustrated), or about right, while capturing heart rate signals, skin conductance and temperature, electroencephalogram (EEG) data, and environmental factors including light, time of day, and room temperature. These will be used to train affect recognizers using a variety of machine learning methods: linear discriminant analysis (including a Kalman adaptive version), support vector machines, neural and Bayesian networks, and random forests. Using a common adaptation strategy that adjusts difficulty up or down one step, the team will measure the enjoyment and performance outcomes that affect-aware recognizers achieve both with and without considering environmental factors, comparing those to a baseline strategy that adapts difficulty based only on task performance. During these experiments, the team will also collect data about users' personality characteristics and use those to develop individualized recognition models and adaptation strategies for different personality types. These individualized models and strategies will be evaluated by comparing them to the baseline data from the first experiment. Finally, they will compare the outcomes of these systems with those from a "best-case" system controlled by humans and a "worst-case" error-prone system that chooses adaptation strategies randomly, looking at those induced error rates along with the natural error rates captured during the other experiments to determine the effect of recognition and adaptation error on satisfaction and task outcomes.