Towards next-generation context-aware affective human-machine interfaces

直到最近，与数字信息交互的普遍方式还是通过键盘和鼠标。然而，随着移动设备的普及，触摸成为人机界面（hmi）的主要输入方式，紧随其后的是语音。然而，未来十年将见证数据生成的巨大增长（例如，统计数据显示，到2020年，我们的数字足迹将增加50倍）。新的传感器和技术将会出现，并将嵌入到日常物品中，创造所谓的物联网革命。这种数据洪流将需要人机交互方面的重大创新，特别是在人与机器以及机器与环境之间的界限变得模糊的情况下。为了实现这一雄心勃勃的目标，拟议的研究计划旨在开发技术，通过使机器不仅意识到周围环境，而且意识到用户的情感/认知状态，从而增强机器智能。这种情境感知的情感hmi将彻底改变加拿大经济的几个关键部门，包括医疗保健、娱乐、教育和电信。为实现这一长期目标，已根据三个主题确定了六个短期目标。首先，我们建议通过以下方式将现有的上下文感知解决方案提升到一个新的水平：i)启用分布式环境感知，ii)通过将身体作为输入模式来改变人机边界，以及iii)通过创新的上下文质量度量为机器提供额外的智能。然而，声音和神经生理输入模式分别对噪音（例如，房间声学）和人工制品（例如，由于运动）极为敏感。为了克服这一严重的可用性因素，第二个主题旨在为语音和神经生理信号开发下一代上下文感知增强算法。最后，我们建议构建一个功能齐全的上下文感知原型，使用语音、手势和身体作为HMI输入控制信号。机器人控制应用程序中的验证实验将被实施，从而为长期应用打开大门，例如远程医疗机器人可以意识到患者的精神状态，例如，可以检测抑郁症的发作并采取相应的措施。***拟议的研究计划将重点放在高素质人才（HQP）的优秀培训上。HQP将接触到信号处理、认知工程、机器学习和动手原型的跨学科混合，以及具有最先进的可穿戴技术和环境监测传感器的研究设施。此外，通过与行业的密切合作，HQP将具备在当今竞争激烈的就业市场中高需求的技能和专业知识。**

Until recently, the ubiquitous way of interacting with digital information was via a keyboard and mouse. As mobile devices gained popularity, however, touch became the primary input modality for human-machine interfaces (HMIs), followed closely by voice. The next decade, however, will witness tremendous growth in data generation (e.g., statistics suggest a 50-fold increase in our digital footprint by 2020). New sensors and technologies will emerge and will be embedded in everyday objects, creating the so-called Internet-of-Things revolution. This data deluge will require drastic innovations in human-machine interaction, particularly as the boundaries between humans and machines, as well as machines and the environment become blurred. To achieve this ambitious goal, the proposed research program aims at developing technologies that will enhance machine intelligence by making them not only aware of their surroundings, but also of their users' affective/cognitive states. Such context-aware affective HMIs will revolutionize several key sectors of the Canadian economy, including healthcare, entertainment, education, and telecommunications.***To achieve this long-term goal, six short-term objectives have been defined, grouped under three main themes. First, we propose to take existing context-aware solutions to the next level by i) enabling distributed environment awareness, ii) shifting the human-machine boundary by making the body as an input modality, and iii) giving the machine additional intelligence via an innovative Quality-of-Context metric. Voice and neurophysiological input modalities, however, are extremely sensitive to noise (e.g., room acoustics) and artefacts (e.g., due to movement), respectively. To overcome this severe usability factor, the second proposed theme aims at developing next-generation context-aware enhancement algorithms for both iv) speech and v) neurophysiological signals. Lastly, we propose to vi) build a fully-functional context-aware prototype that uses voice, gesture, and body as HMI input control signals. Validation experiments within a robot control application will be implemented, thus opening doors for longer-term applications, such as telemedicine robots that are aware of the patient's mental state and could, for example, detect the onset of depression and take necessary measures accordingly. ***The proposed research program will place emphasis on excellent training of highly qualified personnel (HQP). HQP will be exposed to an interdisciplinary melange of signal processing, cognitive engineering, machine learning, and hands-on prototyping, as well as to research facilities that feature state-of-the-art wearable technologies and environment monitoring sensors. In addition, via close collaboration with industry, HQP will be equipped with skills and expertise that are in high demand in today's competitive job market. **