Affective Analytics: A Framework for Monitoring Emotions at the Micro, Meso, and Macro Levels

情感倾向会直接影响我们的短期和长期心理健康。了解这些趋势可能会让我们有机会改掉有害的习惯或养成有益的习惯。在这个提案中，我们提出了一个研究路线图，用于开发一个基于云的框架，该框架可以持续评估用户的情绪状态。该框架将由预先设计的组件组成，这些组件允许在不同的范围和粒度分析情感信息。该框架依赖于1)个人计算设备和可穿戴传感器来收集数据，以及2)运行机器学习模型的云服务器来理解所获得的信息。该框架将估计的情绪与可能提供有关情绪趋势根源的有用指示的背景联系起来。因此，用户可以猜测积极或消极情感变化的可能原因。该框架在三个层面上监测情绪：-微观监测：指在个人层面上的情感监测。用户的情绪是从多种形式估计的，包括音频、视频和生理通道。情感数据由包括关于用户行为模式的信息的上下文来补充。-中观监测：相当于对一群有可能影响其情绪健康的共同情况的个人(例如，同一单位的军事人员或消防员)进行情感监测。-宏观监测：指在城市、地区或国家一级进行监测。从社交媒体上收集数据，以跟踪公众情绪和人群中精神疾病的流行情况。该框架将提供一个通用的可视化模块，该模块呈现与上下文数据相关的评估的情绪信息。本项目的短期目标是：1-使用最先进的深度学习方法开发多通道情感识别方法。2-开发能够分析社交媒体帖子和演讲稿的情感内容的自然语言处理方法。开发结合信号处理和深度学习技术的非接触式生理信号测量方法。4-开发视觉分析模块，传达与上下文数据相关的情绪模式。5-开发一个整合了建议贡献的基于云的框架。该框架可以适用于不同的应用。

Affective trends can directly influence our short- and long-term mental health. Understanding these trends may give us the opportunity to interrupt detrimental habits or develop beneficial ones. In this proposal, we present a research road map for the development of a cloud-based framework that continuously assesses the emotional status of users. The framework will consist of pre-devised components that permit the analysis of affective information at various scopes and granularities. The framework relies on 1) personal computing devices and wearable sensors to collect data and 2) cloud servers running machine learning models to make sense of the obtained information. The framework associates estimated emotions with a context that may provide useful indications about the roots of emotional trends. Hence, users can surmise the possible causes of positive or negative affective changes. The framework monitors emotions at three levels: -Micro-monitoring: Refers to affective monitoring at the individual level. The user's emotions are estimated from multiple modalities, including audio, video, and physiological channels. The affective data is supplemented by a context that includes information about the user's behavioral patterns. -Meso-monitoring: Corresponds to affective monitoring for a group of individuals that share circumstances that may affect their emotional wellbeing (e.g., military personnel or firefighters in the same unit). -Macro-monitoring: Denotes monitoring at the metropolitan, regional, or national level. Data is scraped from social media to track public sentiment and the prevalence of mental illnesses in a population. The framework will provide a common visualization module that presents the assessed emotional information with respect to contextual data. The short-term objectives of this project are: 1- Developing multi-modal emotion recognition methods using state-of-the-art deep learning approaches. 2- Developing natural language processing methods capable of analyzing the affective content of social media posts and speech transcripts. 3 - Developing non-contact physiological signal measurement methods using a combination of signal processing and deep learning techniques. 4 - Developing visual analytics modules that convey patterns in emotions with respect to contextual data. 5 - Developing a cloud-based framework that integrates the proposed contributions. The framework can be adopted for diverse applications.