Affective Analytics: A Framework for Monitoring Emotions at the Micro, Meso, and Macro Levels

情感趋势可以直接影响我们短期和长期的心理健康。了解这些趋势可以让我们有机会打破有害的习惯或培养有益的习惯。在本提案中，我们提出了一个研究路线图，用于开发基于云的框架，该框架可以持续评估用户的情感状态。该框架将由预先设计的组件组成，允许在各种范围和粒度上分析情感信息。该框架依赖于1)个人计算设备和可穿戴传感器来收集数据，2)运行机器学习模型的云服务器来理解获得的信息。该框架将估计的情绪与可能提供有关情绪趋势根源的有用指示的背景联系起来。因此，用户可以推测出积极或消极情感变化的可能原因。该框架从三个层面监测情绪：-微观监测：指个人层面的情感监测。用户的情绪是通过多种方式来估计的，包括音频、视频和生理渠道。情感数据由包含用户行为模式信息的上下文进行补充。-中观监测：对一群共享可能影响其情绪健康的环境的个人进行情感监测（例如，同一单位的军事人员或消防员）。-宏观监测：指城市、地区或国家层面的监测。研究人员从社交媒体上收集数据，以追踪公众情绪和人群中精神疾病的流行程度。该框架将提供一个通用的可视化模块，显示与上下文数据相关的评估情感信息。该项目的短期目标是：1-使用最先进的深度学习方法开发多模态情感识别方法。2-开发能够分析社交媒体帖子和语音文本情感内容的自然语言处理方法。3 -结合信号处理和深度学习技术开发非接触式生理信号测量方法。开发可视化分析模块，根据上下文数据传达情感模式。5 -开发一个基于云的框架，集成建议的贡献。该框架可用于各种应用程序。

Affective trends can directly influence our short- and long-term mental health. Understanding these trends may give us the opportunity to interrupt detrimental habits or develop beneficial ones. In this proposal, we present a research road map for the development of a cloud-based framework that continuously assesses the emotional status of users. The framework will consist of pre-devised components that permit the analysis of affective information at various scopes and granularities. The framework relies on 1) personal computing devices and wearable sensors to collect data and 2) cloud servers running machine learning models to make sense of the obtained information. The framework associates estimated emotions with a context that may provide useful indications about the roots of emotional trends. Hence, users can surmise the possible causes of positive or negative affective changes. The framework monitors emotions at three levels: -Micro-monitoring: Refers to affective monitoring at the individual level. The user's emotions are estimated from multiple modalities, including audio, video, and physiological channels. The affective data is supplemented by a context that includes information about the user's behavioral patterns. -Meso-monitoring: Corresponds to affective monitoring for a group of individuals that share circumstances that may affect their emotional wellbeing (e.g., military personnel or firefighters in the same unit). -Macro-monitoring: Denotes monitoring at the metropolitan, regional, or national level. Data is scraped from social media to track public sentiment and the prevalence of mental illnesses in a population. The framework will provide a common visualization module that presents the assessed emotional information with respect to contextual data. The short-term objectives of this project are: 1- Developing multi-modal emotion recognition methods using state-of-the-art deep learning approaches. 2- Developing natural language processing methods capable of analyzing the affective content of social media posts and speech transcripts. 3 - Developing non-contact physiological signal measurement methods using a combination of signal processing and deep learning techniques. 4 - Developing visual analytics modules that convey patterns in emotions with respect to contextual data. 5 - Developing a cloud-based framework that integrates the proposed contributions. The framework can be adopted for diverse applications.