Real-time, low-latency closed-loop feedback using markerless posture tracking.
Real-time, low-latency closed-loop feedback using markerless posture tracking.
复制标题
DOI：
10.7554/elife.61909
复制
发表时间：
2020-12-08
期刊：
影响因子：
7.7
通讯作者：
Mathis MW
中科院分区：
文献类型：
--
作者：
Kane GA;Lopes G;Saunders JL;Mathis A;Mathis MW
基于动物行为实时控制行为任务或刺激神经活动的能力是实验神经学家的重要工具。理想情况下，此类工具是非侵入性、低延迟的，并提供基于状态触发外部硬件的接口。具有深度学习的姿势估计的最新进展使研究人员能够训练深度神经网络来准确量化各种动物行为。在这里，我们提供了一个新的DeepLabCut-Live！实现低延迟实时位姿估计(在15毫秒内，>；100 FPS)的套装，以及实现零延迟反馈的附加前向预测模块，以及允许更高推理速度的动态裁剪模式。我们还提供了三个选项来轻松使用该工具：(1)独立的图形用户界面(称为DLC-Live！图形用户界面)，并集成到(2)盆景，和(3)自动驾驶。最后，我们对各种系统的性能进行了基准测试，以便实验人员可以轻松地确定需要哪些硬件来满足他们的需求。
The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here, we provide a new DeepLabCut-Live! package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called DLC-Live! GUI), and integration into (2) Bonsai, and (3) AutoPilot. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.