Multimodal machine learning approaches for measuring mental wellbeing using sensor and online data

之前的研究已经证明了身体活动和睡眠对心理健康的好处，还提供了虚拟行为(如社交媒体使用和屏幕时间)与心理健康问题之间的联系。我们认为，一个人的身体或在线行为本身可能不是他们心理健康的可靠指标。因此，这项研究的重点是将个体的身体行为和虚拟行为结合起来，以更好地获得心理健康。这项研究涉及建立机器学习算法，以基于被动感知的行为模式预测心理健康。我们的目标是评估哪些行为特征为预测心理健康提供了最重要的信息。此外，我们还调查了同时包括物理和虚拟行为的机器学习模型是否可以更好地预测心理健康。这项研究使用了通过定制应用程序收集的数据。这款应用程序旨在低调地在个人智能手机的背景下运行。它还通过生态瞬时评估(EMA)为用户提供了一个平台，以注册关于他们的情绪和其他重要事件的信息。这项研究的一个重要方面在于数据收集。数据将使用定制手机应用程序收集，涉及身体(例如，加速计数据、GPS、睡眠数据)和在线(例如，应用程序使用、屏幕时间、社交媒体使用)行为和自我报告的EMA数据(例如，压力、情绪、幸福感)。这项研究的第二个贡献将是开发新的最先进的多模式联合机器学习算法，该算法可以有效地利用收集的数据来预测心理健康(使用EMA作为基本事实)。第三个贡献将是解决用户数据隐私和安全的以隐私为重点的模型。最后一份报告介绍了前面的内容，并将涉及与所有利益攸关方协商，以便为数据收集和机器学习应用程序提供信息。这一研究项目的关键新颖性在于开发了新型的多模式机器学习和隐私驱动模型，可以有效地融合和学习身体行为和在线行为，以便更好地预测心理健康状态。

Previous studies have demonstrated the advantages of physical activity and sleep on mental health and also provided an association between virtual behaviour, such as social media use and screen time, and mental health problems. We believe that a person's physical or online behaviour alone might not be a reliable indicator of their mental health. Therefore, this study focuses on combining physical and virtual behaviour of individuals to better access mental health. This study involves building machine learning algorithms to predict mental wellbeing based on passively sensed behavioural patterns. We aim to assess which behavioural features provide the most important information for predicting mental wellbeing. Additionally, we investigate if machine learning models that include both physical and virtual behaviour can better predict mental health. This study is conducted by using data collected via a custom-made app. This app is designed to run unobtrusively in the background of an individual's smartphone. It also provides a platform via ecological momentary assessment (EMA) for users to register information about their emotions and other important events. A significant aspect of this research lies in data collection. Data will be collected using a custom mobile phone app involving both physical (e.g., accelerometer data, GPS, sleep data) and online (e.g., app usage, screen time, social media use) behaviour and self-reported EMA data (e.g., stress, mood, happiness). The second contribution of this research will be to develop novel state-of-the-art multimodal federated machine learning algorithms that can effectively utilise the collected data to predict mental well-being (using the EMA as ground truth). The third contribution will be to address privacy-focused models for users' data privacy and security. The final contribution informs the previous aspects and will involve consultation with all stakeholders to inform data collection and machine learning applications. The key novelty of this research project lies in the development of novel multimodal machine learning and privacy-driven models that can fuse and learn from physical and online behaviour effectively, in order to better predict mental health status.