An easy-to-use software for 3D behavioral tracking from multi-view cameras

项目摘要 这一行政补充的首要目标是增加采用和利用一个令人兴奋的新 3D姿态跟踪技术，DANNCE，我们正在母公司R01（R01GM136972）中开发。舞蹈 使用深度神经网络从行为动物的多视图视频预测姿势和附属物位置， 在各种物种和环境中实现最先进的性能。虽然一些实验室 已经采用这种技术来研究大脑如何产生运动，并探索其机制， 作为各种疾病的基础，它的广泛使用受到孤立的、未集成的软件的阻碍， 最佳软件工程实践。这会给非专业用户带来瓶颈和错误， 限制了我们变革性技术的范围和适用性。即使对于专家用户来说，目前的软件 使用缓慢、难以针对特定应用程序进行优化、维护繁琐。这也阻碍了 开源社区的贡献和功能开发。此外，我们的工具具有很高的计算能力， 这些需求进一步限制了其在实验室和研究机构中的使用， 基础设施演进为了提高我们技术的可扩展性、可用性和健壮性， 传播到更广泛的研究社区，我们将与专业的软件工程师，以实现 最佳软件工程和社区开源开发实践。具体而言，我们将解决 的目标。 1.为了消除系统瓶颈，我们将大幅重构和改进软件， 组件转换为具有社区标准数据格式的通用后端Python库，从而消除了 常见的陷阱，并建立一个模块化和灵活的平台，可以很容易地扩展和增强。 2.为了确保非专家可以轻松地采用我们的系统，并探索大规模的行为数据， 生成，我们将创建一个易于使用的，集中的图形用户界面。 3.最后，为了提高云就绪性和各种硬件加速器的利用率，我们将 重写后端集成，以允许用户在不同的计算上大规模训练和运行我们的系统 平台，包括内部部署的高性能计算集群和商业云提供商。 通过精简和改进我们最先进的方法所使用的软件包，我们将 极大地提高了它的可用性、灵活性和可扩展性，从而加速了许多正在进行的研究项目 并消除了启动许多其他项目的主要障碍。

Project Summary The overarching goal of this administrative supplement is to increase the adoption and utilization an exciting new technology for 3D pose tracking, DANNCE, that we are developing in the parent R01 (R01GM136972). DANNCE uses a deep neural network to predict pose and appendage positions from multi-view video of behaving animals, achieving state-of-the-art performance across a variety of species and environments. While several labs have already adopted this technology to study how the brain generates movement and to probe the mechanisms underlying a variety of disorders, its wider use is hampered by a siloed, unintegrated software that fails to conform to best software engineering practices. This leads to bottlenecks and errors for non-expert users, severely limiting the reach and applicability of our transformative technology. Even for expert users, the current software is slow to use, difficult to optimize for specific applications, and burdensome to maintain. This also impedes contributions and feature development by the open-source community. Furthermore, our tool has high compute demands that further constrain its use to labs and institutes with access to high-performance computing infrastructure. To improve the scalability, useability, and robustness of our technology and to allow for its dissemination to a wider research community, we will work with professional software engineers to implement best software engineering and community open-source development practices. Specifically, we will address the following aims. 1. To eliminate system bottlenecks, we will significantly refactor and improve the software to integrate all components into a common backend Python library with community standard data formats, thus eliminating common pitfalls, and build a modular and flexible platform that can be easily extended and augmented. 2. To ensure that non-experts can easily adopt our system and explore the large-scale behavioral data it generates, we will create an easy-to-use, centralized graphical user interface. 3. Lastly, to enhance cloud readiness and the utilization of a wide range of hardware accelerators, we will rewrite the backend integration to allow users to train and run our system at scale on different computing platforms, both on-premise high-performance computing clusters and commercial cloud providers. By streamlining and improving the software package underlying our state-of the art method, we will dramatically increase its usability, flexibility, and scalability, thus accelerating many ongoing research programs and removing a main barrier for initiating many others.