An easy-to-use software for 3D behavioral tracking from multi-view cameras

点击翻译按钮获取中文摘要

Project Summary The overarching goal of this administrative supplement is to increase the adoption and utilization an exciting new technology for 3D pose tracking, DANNCE, that we are developing in the parent R01 (R01GM136972). DANNCE uses a deep neural network to predict pose and appendage positions from multi-view video of behaving animals, achieving state-of-the-art performance across a variety of species and environments. While several labs have already adopted this technology to study how the brain generates movement and to probe the mechanisms underlying a variety of disorders, its wider use is hampered by a siloed, unintegrated software that fails to conform to best software engineering practices. This leads to bottlenecks and errors for non-expert users, severely limiting the reach and applicability of our transformative technology. Even for expert users, the current software is slow to use, difficult to optimize for specific applications, and burdensome to maintain. This also impedes contributions and feature development by the open-source community. Furthermore, our tool has high compute demands that further constrain its use to labs and institutes with access to high-performance computing infrastructure. To improve the scalability, useability, and robustness of our technology and to allow for its dissemination to a wider research community, we will work with professional software engineers to implement best software engineering and community open-source development practices. Specifically, we will address the following aims. 1. To eliminate system bottlenecks, we will significantly refactor and improve the software to integrate all components into a common backend Python library with community standard data formats, thus eliminating common pitfalls, and build a modular and flexible platform that can be easily extended and augmented. 2. To ensure that non-experts can easily adopt our system and explore the large-scale behavioral data it generates, we will create an easy-to-use, centralized graphical user interface. 3. Lastly, to enhance cloud readiness and the utilization of a wide range of hardware accelerators, we will rewrite the backend integration to allow users to train and run our system at scale on different computing platforms, both on-premise high-performance computing clusters and commercial cloud providers. By streamlining and improving the software package underlying our state-of the art method, we will dramatically increase its usability, flexibility, and scalability, thus accelerating many ongoing research programs and removing a main barrier for initiating many others.