Multimodal machine learning approaches for measuring mental wellbeing using sensor and online data

点击翻译按钮获取中文摘要

Previous studies have demonstrated the advantages of physical activity and sleep on mental health and also provided an association between virtual behaviour, such as social media use and screen time, and mental health problems. We believe that a person's physical or online behaviour alone might not be a reliable indicator of their mental health. Therefore, this study focuses on combining physical and virtual behaviour of individuals to better access mental health. This study involves building machine learning algorithms to predict mental wellbeing based on passively sensed behavioural patterns. We aim to assess which behavioural features provide the most important information for predicting mental wellbeing. Additionally, we investigate if machine learning models that include both physical and virtual behaviour can better predict mental health. This study is conducted by using data collected via a custom-made app. This app is designed to run unobtrusively in the background of an individual's smartphone. It also provides a platform via ecological momentary assessment (EMA) for users to register information about their emotions and other important events. A significant aspect of this research lies in data collection. Data will be collected using a custom mobile phone app involving both physical (e.g., accelerometer data, GPS, sleep data) and online (e.g., app usage, screen time, social media use) behaviour and self-reported EMA data (e.g., stress, mood, happiness). The second contribution of this research will be to develop novel state-of-the-art multimodal federated machine learning algorithms that can effectively utilise the collected data to predict mental well-being (using the EMA as ground truth). The third contribution will be to address privacy-focused models for users' data privacy and security. The final contribution informs the previous aspects and will involve consultation with all stakeholders to inform data collection and machine learning applications. The key novelty of this research project lies in the development of novel multimodal machine learning and privacy-driven models that can fuse and learn from physical and online behaviour effectively, in order to better predict mental health status.