Multimodal machine learning approaches for measuring mental wellbeing using sensor and online data

之前的研究已经证明了体育活动和睡眠对心理健康的好处，也提供了虚拟行为（如社交媒体使用和屏幕时间）与心理健康问题之间的联系。我们认为，一个人的身体或上网行为本身可能不是他们心理健康的可靠指标。因此，本研究的重点是将个体的物理和虚拟行为结合起来，以更好地获得心理健康。这项研究涉及建立机器学习算法，根据被动感知的行为模式来预测心理健康。我们的目标是评估哪些行为特征为预测心理健康提供了最重要的信息。此外，我们还研究了包括物理和虚拟行为的机器学习模型是否能更好地预测心理健康。这项研究是通过一个定制的应用程序收集的数据进行的。这个应用程序被设计成在个人智能手机的后台不显眼地运行。它还通过生态瞬间评估（EMA）为用户提供了一个平台，可以记录他们的情绪和其他重要事件的信息。本研究的一个重要方面在于数据收集。数据将使用定制的手机应用程序收集，包括物理（例如，加速度计数据、GPS、睡眠数据）和在线（例如，应用程序使用、屏幕时间、社交媒体使用）行为和自我报告的EMA数据（例如，压力、情绪、幸福）。这项研究的第二个贡献将是开发新的最先进的多模态联合机器学习算法，该算法可以有效地利用收集的数据来预测心理健康（使用EMA作为基础事实）。第三个贡献将是解决以隐私为中心的用户数据隐私和安全模型。最后的贡献通知了前面的方面，并将涉及与所有利益相关者协商，以告知数据收集和机器学习应用。该研究项目的关键新颖之处在于开发新的多模式机器学习和隐私驱动模型，这些模型可以有效地融合和学习物理和在线行为，以便更好地预测心理健康状况。

Previous studies have demonstrated the advantages of physical activity and sleep on mental health and also provided an association between virtual behaviour, such as social media use and screen time, and mental health problems. We believe that a person's physical or online behaviour alone might not be a reliable indicator of their mental health. Therefore, this study focuses on combining physical and virtual behaviour of individuals to better access mental health. This study involves building machine learning algorithms to predict mental wellbeing based on passively sensed behavioural patterns. We aim to assess which behavioural features provide the most important information for predicting mental wellbeing. Additionally, we investigate if machine learning models that include both physical and virtual behaviour can better predict mental health. This study is conducted by using data collected via a custom-made app. This app is designed to run unobtrusively in the background of an individual's smartphone. It also provides a platform via ecological momentary assessment (EMA) for users to register information about their emotions and other important events. A significant aspect of this research lies in data collection. Data will be collected using a custom mobile phone app involving both physical (e.g., accelerometer data, GPS, sleep data) and online (e.g., app usage, screen time, social media use) behaviour and self-reported EMA data (e.g., stress, mood, happiness). The second contribution of this research will be to develop novel state-of-the-art multimodal federated machine learning algorithms that can effectively utilise the collected data to predict mental well-being (using the EMA as ground truth). The third contribution will be to address privacy-focused models for users' data privacy and security. The final contribution informs the previous aspects and will involve consultation with all stakeholders to inform data collection and machine learning applications. The key novelty of this research project lies in the development of novel multimodal machine learning and privacy-driven models that can fuse and learn from physical and online behaviour effectively, in order to better predict mental health status.