Towards next-generation context-aware affective human-machine interfaces

直到最近，与数字信息交互的普遍方式是通过键盘和鼠标。然而，随着移动设备的普及，触摸成为人机接口(HMI)的主要输入方式，紧随其后的是语音。然而，下一个十年将见证数据生成的巨大增长(例如，统计数据显示，到2020年，我们的数字足迹将增加50倍)。新的传感器和技术将出现并嵌入日常物品中，创造所谓的物联网革命。这种数据洪流将要求在人机交互方面进行彻底的创新，特别是在人与机器以及机器与环境之间的边界变得模糊的情况下。为了实现这一雄心勃勃的目标，拟议的研究计划旨在开发能够增强机器智能的技术，使它们不仅能意识到周围环境，还能意识到用户的情感/认知状态。这种情景感知的情感HMI将彻底改变加拿大经济的几个关键部门，包括医疗保健、娱乐、教育和电信。 为了实现这一长期目标，已经确定了六个短期目标，并将其归类为三个主题。首先，我们建议通过i)启用分布式环境感知，ii)通过将身体作为输入通道来改变人机边界，以及iii)通过创新的上下文质量度量赋予机器额外的智能，从而将现有的情境感知解决方案提升到下一层次。然而，语音和神经生理输入方式分别对噪声(例如，房间声学)和伪影(例如，由于运动)非常敏感。为了克服这一严重的可用性因素，第二个提出的主题旨在为iv)语音和v)神经生理信号开发下一代上下文感知增强算法。最后，我们建议vi)构建一个使用语音、手势和身体作为HMI输入控制信号的全功能上下文感知原型。将实施机器人控制应用程序中的验证实验，从而为长期应用打开大门，例如远程医疗机器人，这些机器人可以感知患者的精神状态，例如可以检测抑郁症的发作并采取相应的必要措施。 拟议的研究计划将重点放在高素质人才(HQP)的优秀培训上。HQP将接触到信号处理、认知工程、机器学习和实践原型的跨学科混合，以及以最先进的可穿戴技术和环境监测传感器为特色的研究设施。此外，通过与业界的密切合作，HQP将配备当今竞争激烈的就业市场所需的技能和专业知识。

Until recently, the ubiquitous way of interacting with digital information was via a keyboard and mouse. As mobile devices gained popularity, however, touch became the primary input modality for human-machine interfaces (HMIs), followed closely by voice. The next decade, however, will witness tremendous growth in data generation (e.g., statistics suggest a 50-fold increase in our digital footprint by 2020). New sensors and technologies will emerge and will be embedded in everyday objects, creating the so-called Internet-of-Things revolution. This data deluge will require drastic innovations in human-machine interaction, particularly as the boundaries between humans and machines, as well as machines and the environment become blurred. To achieve this ambitious goal, the proposed research program aims at developing technologies that will enhance machine intelligence by making them not only aware of their surroundings, but also of their users’ affective/cognitive states. Such context-aware affective HMIs will revolutionize several key sectors of the Canadian economy, including healthcare, entertainment, education, and telecommunications. To achieve this long-term goal, six short-term objectives have been defined, grouped under three main themes. First, we propose to take existing context-aware solutions to the next level by i) enabling distributed environment awareness, ii) shifting the human-machine boundary by making the body as an input modality, and iii) giving the machine additional intelligence via an innovative Quality-of-Context metric. Voice and neurophysiological input modalities, however, are extremely sensitive to noise (e.g., room acoustics) and artefacts (e.g., due to movement), respectively. To overcome this severe usability factor, the second proposed theme aims at developing next-generation context-aware enhancement algorithms for both iv) speech and v) neurophysiological signals. Lastly, we propose to vi) build a fully-functional context-aware prototype that uses voice, gesture, and body as HMI input control signals. Validation experiments within a robot control application will be implemented, thus opening doors for longer-term applications, such as telemedicine robots that are aware of the patient’s mental state and could, for example, detect the onset of depression and take necessary measures accordingly. The proposed research program will place emphasis on excellent training of highly qualified personnel (HQP). HQP will be exposed to an interdisciplinary melange of signal processing, cognitive engineering, machine learning, and hands-on prototyping, as well as to research facilities that feature state-of-the-art wearable technologies and environment monitoring sensors. In addition, via close collaboration with industry, HQP will be equipped with skills and expertise that are in high demand in today’s competitive job market.