Fatigue INtelligent Discovery (FIND) - A Novel Machine Learning and Multi-Modal Fatigue Detection Framework

大约五分之一的人是轮班工人，他们中的大多数人经常疲劳，导致工作表现下降，严重的身心健康风险，给英国经济和NHS带来了巨大的成本。**迫切需要通过及时发现轮班工人的疲劳来减轻经济和NHS的负担，以帮助减轻这种负担。**然而，早期疲劳检测已被证明是困难的。这既是由于普遍缺乏对标准工具的共识，也是由于设计负担得起、非侵入性和易于接受的工具的挑战。机器学习提供了一种自动检测疲劳的解决方案，然而，这种方法的通用性有限。例如，机器学习模型要么是使用非常小或不具代表性的样本开发的，要么是使用来自单个时间点的数据，要么是试图基于单一行为类型（例如，通过观察面部张开的嘴来检测打哈欠作为困倦的迹象）来检测疲劳，而不是整合多种行为类型（例如，声音特征），这将允许更准确的结果。**该项目的目的是利用机器学习模型的力量，随着时间的推移，使用不同的行为类型（语音和面部信息）训练数据，开发出一种首创的、可靠的疲劳筛查工具，该工具在有限的时间和侵入性条件下有效。**为此，我们将通过在线招聘平台收集1000名轮班工人在两周内的几分钟活动时间。活动包括要求个人产生语音和面部表情的简短任务，通过参与者的智能设备麦克风和摄像头捕获（我们之前成功使用并发表的方法为Fara等人，2022）。我们将通过使用文献中的一系列其他任务来验证数据集中的疲劳。最终，该项目将通过早期疲劳检测影响轮班工人和其他安全关键行业，并允许及时干预，如适当的轮班安排、休息和小睡。这将改善轮班工人的健康状况，最大限度地提高英国经济的生产力和回报，并间接缓解NHS的需求。鉴于疲劳在许多情况下都是一种常见的精神疾病症状，该项目将为验证我们的疲劳跟踪工具用于临床目的（例如抑郁症）**奠定基础。

Around 1 in 5 people are shift workers and a majority of them suffer from regular fatigue leading to decreased performance and serious physical and mental health risks, with huge costs for the UK economy and the NHS. **There is an urgent need to relieve this burden on the economy and the NHS, by timely detection of fatigue in shift workers in order to help mitigate it.**Early fatigue detection, however, has proven difficult. This is both due to a general lack of a consensus on a standard instrument as well as to the challenge of designing affordable, non-intrusive and easily acceptable tools. Machine learning offers a solution to automatically detect fatigue, however, this approach has limited generalisability. For example, machine learning models have either been developed using very small or unrepresentative samples, have used data from a single time point or have tried to detect fatigue based on a single behaviour type (e.g. looking at mouth opening in the face to detect yawning as a sign of sleepiness) instead of integrating multiple behaviour types (e.g. voice characteristics), which would allow for more accurate results.**The aim of this project is to leverage the power of machine learning models trained with data over time and using different behaviour types (speech and facial information) to develop a first-of-its-kind, reliable fatigue screening tool that is effective under conditions of limited time and intrusivity.**To this end, we will collect several minutes' worth of activities for two weeks from 1000 shift workers around their shift schedule via an online recruitment platform. Activities include short tasks that require individuals to produce speech and facial expressions, captured via participants' smart device microphone and camera (an approach we have previously successfully used and published as Fara et al., 2022). We will validate fatigue in the dataset by using a range of other tasks from the literature.Ultimately, the project will impact **shift workers and other safety-critical industries** via early fatigue detection, allowing timely interventions such as appropriate shift scheduling, rest breaks and naps. This will improve shift worker health, maximise productivity and **returns for the UK economy and indirectly relieve NHS demand**. Given that fatigue is a common mental ill-health symptom across many conditions, this project will set the stage for the **validation of our fatigue tracking tool for clinical purposes (e.g. depression)**.