DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels.
DeepEthogram, a machine learning pipeline for supervised behavior classification from raw pixels.
复制标题
DOI：
10.7554/elife.63377
复制
发表时间：
2021-09-02
期刊：
影响因子：
7.7
通讯作者：
Harvey CD
中科院分区：
文献类型：
--
作者：
Bohnslav JP;Wimalasena NK;Clausing KJ;Dai YY;Yarmolinsky DA;Cruz T;Kashlan AD;Chiappe ME;Orefice LL;Woolf CJ;Harvey CD
动物行为的视频用于量化研究人员定义的感兴趣的行为，以研究神经功能，基因突变和药物治疗。感兴趣的行为通常是手动评分的，这是耗时的，仅限于少数行为，并且在研究人员之间是可变的。我们创建了DeepEthogram：使用监督机器学习将原始视频像素转换为行为图的软件，每个视频帧中存在感兴趣的行为。DeepEthogram被设计成通用的，适用于物种、行为和视频记录硬件。它使用卷积神经网络来计算运动，从运动和图像中提取特征，并将特征分类为行为。在老鼠和苍蝇视频的单帧上，行为分类的准确率超过90%，与专家级的人类表现相匹配。DeepEthogram可以准确地预测罕见的行为，几乎不需要训练数据，并且可以在受试者之间进行推广。图形化界面允许在不进行最终用户编程的情况下进行全过程分析。DeepEthogram对研究人员定义的感兴趣行为的快速，自动和可重复的标记可以加速和增强监督行为分析。代码可从以下网址获得：https://github.com/jbohnslav/deepethogram.
Videos of animal behavior are used to quantify researcher-defined behaviors of interest to study neural function, gene mutations, and pharmacological therapies. Behaviors of interest are often scored manually, which is time-consuming, limited to few behaviors, and variable across researchers. We created DeepEthogram: software that uses supervised machine learning to convert raw video pixels into an ethogram, the behaviors of interest present in each video frame. DeepEthogram is designed to be general-purpose and applicable across species, behaviors, and video-recording hardware. It uses convolutional neural networks to compute motion, extract features from motion and images, and classify features into behaviors. Behaviors are classified with above 90% accuracy on single frames in videos of mice and flies, matching expert-level human performance. DeepEthogram accurately predicts rare behaviors, requires little training data, and generalizes across subjects. A graphical interface allows beginning-to-end analysis without end-user programming. DeepEthogram’s rapid, automatic, and reproducible labeling of researcher-defined behaviors of interest may accelerate and enhance supervised behavior analysis. Code is available at: https://github.com/jbohnslav/deepethogram.