A Multi-modal Machine Learning Approach and Toolkit to Automate Recognition of Early Stages of Dementia among British Sign Language Users
A Multi-modal Machine Learning Approach and Toolkit to Automate Recognition of Early Stages of Dementia among British Sign Language Users
复制标题
一种多模式机器学习方法和工具包，可自动识别英国手语用户的痴呆症早期阶段
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Tyron Woolfe
中科院分区：
文献类型：
--
作者：
Xing Liang;A. Angelopoulou;E. Kapetanios;B. Woll;Reda Al;Tyron Woolfe
人口老龄化趋势与痴呆症等后天认知障碍的发病率增加有关。虽然痴呆症无法治愈，但及时诊断有助于获得必要的支持和适当的药物治疗。研究人员正在紧急开发有效的技术工具，帮助医生进行认知障碍的早期识别。特别是，筛查老年痴呆症的英国手语（BSL）聋人签名带来了额外的挑战，因为诊断过程与口译员的质量和可用性以及适当的问卷调查和认知测试等条件有关。另一方面，基于深度学习的图像和视频分析和理解方法很有前途，特别是卷积神经网络（CNN）的采用，这需要大量的训练数据。然而，在本文中，我们以以下方式展示了新奇：a）基于多模态机器学习的自动识别工具包，用于BSL用户中痴呆症的早期阶段，其中来自身体的几个部位的特征有助于符号包络，例如，手臂运动和面部表情，被组合，B）通用性，因为它是独立于语言的，所以可以将我们的技术应用于任何手语的用户，c）给定机器学习（ML）预测模型的复杂性和准确性之间的权衡以及可用的有限量的训练和测试数据，我们表明，我们的方法是不是过度拟合，并有潜力扩大规模。
The ageing population trend is correlated with an increased prevalence of acquired cognitive impairments such as dementia. Although there is no cure for dementia, a timely diagnosis helps in obtaining necessary support and appropriate medication. Researchers are working urgently to develop effective technological tools that can help doctors undertake early identification of cognitive disorder. In particular, screening for dementia in ageing Deaf signers of British Sign Language (BSL) poses additional challenges as the diagnostic process is bound up with conditions such as quality and availability of interpreters, as well as appropriate questionnaires and cognitive tests. On the other hand, deep learning based approaches for image and video analysis and understanding are promising, particularly the adoption of Convolutional Neural Network (CNN), which require large amounts of training data. In this paper, however, we demonstrate novelty in the following way: a) a multi-modal machine learning based automatic recognition toolkit for early stages of dementia among BSL users in that features from several parts of the body contributing to the sign envelope, e.g., hand-arm movements and facial expressions, are combined, b) universality in that it is possible to apply our technique to users of any sign language, since it is language independent, c) given the trade-off between complexity and accuracy of machine learning (ML) prediction models as well as the limited amount of training and testing data being available, we show that our approach is not over-fitted and has the potential to scale up.