Unsupervised behaviour analysis and magnification (uBAM) using deep learning
Unsupervised behaviour analysis and magnification (uBAM) using deep learning
复制标题
DOI：
10.1038/s42256-021-00326-x
复制
发表时间：
2021-04-05
影响因子：
23.8
通讯作者：
Ommer, Bjorn
中科院分区：
文献类型：
--
作者：
Brattoli, Biagio;Buchler, Uta;Ommer, Bjorn
运动行为分析对生物医学研究和临床诊断至关重要，因为它提供了一种非侵入性的策略来识别运动损伤及其由干预引起的变化。最先进的仪器运动分析是时间和成本密集型的，因为它需要放置物理或虚拟标记。除了需要为训练或微调检测器标记关键点或注释所需的工作外，用户还需要事先了解有趣的行为，以提供有意义的关键点。在这里，我们引入了无监督行为分析和放大（uBAM），这是一种通过发现和放大偏差来分析行为的自动深度学习算法。一个中心方面是姿势和行为表征的无监督学习，以实现运动的客观比较。除了发现和量化行为偏差外，我们还提出了一种生成模型，可以直接在视频中视觉放大微妙的行为差异，而无需通过关键点或注释绕路。即使在不同的个体之间，这种偏差放大的关键是外表和行为的分离。对患有神经系统疾病的啮齿动物和人类患者的评估表明，我们的方法具有广泛的适用性。此外，将光遗传刺激与我们的无监督行为分析相结合，表明它是一种将功能与大脑可塑性相关的非侵入性诊断工具。能够精确地分析行为对于研究健康和疾病中的运动行为至关重要，但通常需要耗费大量时间和人力。Brattoli等人开发了一种基于深度学习的自动方法，用于分析运动行为，并在不同物种和不同运动功能上进行评估。
Motor behaviour analysis is essential to biomedical research and clinical diagnostics as it provides a non-invasive strategy for identifying motor impairment and its change caused by interventions. State-of-the-art instrumented movement analysis is time- and cost-intensive, because it requires the placement of physical or virtual markers. As well as the effort required for marking the keypoints or annotations necessary for training or fine-tuning a detector, users need to know the interesting behaviour beforehand to provide meaningful keypoints. Here, we introduce unsupervised behaviour analysis and magnification (uBAM), an automatic deep learning algorithm for analysing behaviour by discovering and magnifying deviations. A central aspect is unsupervised learning of posture and behaviour representations to enable an objective comparison of movement. Besides discovering and quantifying deviations in behaviour, we also propose a generative model for visually magnifying subtle behaviour differences directly in a video without requiring a detour via keypoints or annotations. Essential for this magnification of deviations, even across different individuals, is a disentangling of appearance and behaviour. Evaluations on rodents and human patients with neurological diseases demonstrate the wide applicability of our approach. Moreover, combining optogenetic stimulation with our unsupervised behaviour analysis shows its suitability as a non-invasive diagnostic tool correlating function to brain plasticity.Being able to precisely analyse behaviour is essential for the study of motor behaviour in health and disease, but is often time- and labour-intensive. Brattoli et al. develop an automatic approach based on deep learning for analysing motor behaviour and evaluate it on different species and diverse motor functions.