SHF: Small: A Chip of Happiness: Device-to-System Developments of Affective Computing for Human-in-the-loop Computer System

虽然近年来人工智能（AI）和低功耗可穿戴电子产品的技术进步给人们的生活带来了巨大的改善，但现有的计算机系统仍然缺少一个元素，即用户的个人感受与计算设备的硬件操作之间的深度合作。随着人工智能技术的发展趋向于“以人为中心的计算”，现在是时候重塑计算设备的角色，将人类的感知融入到操作的循环中。最近，所谓的“情感计算”或情感人工智能的研究显示了一种很有前途的新计算范式，其中利用对用户情感（例如情感）的了解来显著提高对用户的服务质量。不幸的是，到目前为止，硬件层面对人类实时感受的支持和参与非常少。该项目旨在开发一种新型的情感计算硬件技术，通过先进的微电子设备实时跟踪人类的情绪、情绪等情感，并进一步融入现代计算系统的操作中，从而为人们的日常活动提供前所未有的支持，提高计算设备的效率。通过将先进的计算硬件与用户的实时感受相结合，创造新一代智能人机界面，让人站在现代可穿戴电子设备的中心。该项目将对在线业务、社交媒体、电子学习、医疗保健等人类服务产生广泛影响。通过举办高级讲座、讲习班和研讨会，该项目还将为大学生和更广泛的受众提供重要的教育和培训机会。这个项目将在缩小人类实时感受与现代可穿戴计算设备操作之间的差距方面迈出一大步。将开发跨层方法和技术，以实现硬件级别的情感计算，范围从微电子设备的设计到数据管理，从先进的计算模型到系统级软件和硬件集成。更具体地说，在设备层面，利用最近兴起的基于硅的神经处理器技术（新型“幸福加速器”）将被开发出来，在高度受限的低功耗可穿戴设备上实现实时影响推断。在算法层面，该项目将开发先进的机器学习模型，不仅可以提高情感分类和相关认知任务的准确性，还可以在超低功耗边缘设备上高效部署情感计算。此外，在架构层面，该项目将开发一系列新颖的基于影响的存储器、数据和电源管理技术，以提高现代计算设备的能源效率。作为示范，该项目将使用预制硅芯片和紧凑型可穿戴设备来创建先进的情感驱动计算系统，为在线服务、增强/虚拟现实（AR/VR）和课堂学习等现实应用提供新的人类辅助水平。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

While the recent technology advancements in artificial intelligence (AI) and low-power wearable electronics have created tremendous improvements to people’s lives, there is still a missing element in existing computer systems, namely, a deep cooperation between user’s personal feeling and the hardware operation of computing devices. As current developments of AI technology are trending towards “human-centric computing”, it is time to reshape the role of computing devices by bringing human’s perception into the loop of the operations. Recently, research on so-called “affective computing” or emotional AI has shown a promising new computing paradigm in which the knowledge of users’ affects, e.g. emotion, are utilized to significantly enhance the quality of service to the users. Unfortunately, as of today, the support and engagement to human’s real-time feelings at the hardware level is very little. This project aims at developing a new class of affective-computing hardware technology where human affects, e.g. mood, emotions, etc., are being real-time tracked by advanced microelectronic devices and further incorporated into the operations of modern computing systems, leading to unprecedented support to people’s daily activities and enhanced efficiency of computing devices. By linking the advanced computing hardware with users’ real-time affects, a new generation of intelligent human-machine interface can be created allowing human to stay at the center of modern wearable electronic devices. This project will create broad impacts to human services such as online business, social media, e-learning, healthcare, etc. By creating advanced lectures, workshops and seminars, significant educational and training opportunities will also be delivered from this project to college students and the broader audience. This project will take a big step towards closing the gap between human’s real-time feelings and operation of modern wearable computing devices. Cross-layer methodology and techniques will be developed to enable affective computing at the hardware level, ranging from design of microelectronic devices to data management, from advanced computing models to system-level software and hardware integration. More specifically, at the device level, leveraging the recent boom of silicon-based neural-processor techniques (novel "happiness accelerators") will be developed enabling real-time affect inference on highly constrained low-power wearable devices. At the algorithm level, this project will develop advanced machine-learning models to not only improve the accuracy of affect classification and related cognition tasks but also enable efficient deployment of affective computing at ultra-low-power edge devices. Furthermore, at the architecture level, this project will develop a series of novel affect-based memory, data and power management techniques to enhance the energy efficiency of modern computing devices. As a demonstration, this project will use fabricated silicon chips and compact wearable devices to create advanced affect-driven computing system providing a new level of human assistance for real-life applications such as online services, Augmented/Virtual Reality(AR/VR) and classroom learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.