Off body affective and cognitive monitoring for adaptive applications

点击翻译按钮获取中文摘要

This Engage Project wishes to contribute to the development of innovative applications that can monitoremotional and cognitive states based on vocal prosody. Such applications will be beneficial since it will revealinformation about the interaction of two individuals, i.e. satisfaction or frustration, which can require anadjustment in strategy and stress detection or high workload, which can require reallocation of tasks anddemands. Cortex medias' application already includes verbal content recognition. Adding the online monitoringof emotional and cognitive states based on vocal prosody (the way something is said as opposed to the actuallinguistic meanings of the spoken words) will greatly enhance the validity of their application. Our approachcombines experimental psychology and cognitive ergonomics to integrate the automatic calculation and onlineassessment of vocal prosodic features for the recognition of emotions and cognitive traits. First, the applicationwill be further developed by integrating algorithms for the automatic calculation and online monitoring ofemotional and cognitive states during interactions between two individuals. The two dimensions used for theassessment will be satisfaction and workload. Second, the project will allow the refinement and validation ofthe implemented algorithms by corroborating evaluation from the prosodic features using physiologicalmeasures and a facial expression system. Results of this proposed project will define guidelines andrecommendations for designing applications that include recognition of users' emotional and cognitive statesbased on prosodic features. The outcomes of the project will positively impact the Canadian technological,financial and education sectors, as well as other areas such as public security, intelligent tutoring, e-health andthe gaming industry.