cloudSLEAP: Maximizing accessibility to deep learning-based motion capture

点击翻译按钮获取中文摘要

cloudSLEAP – PROJECT SUMMARY/ABSTRACT Understanding how the brain produces complex behavior is a central goal of neuroscience, but quantifying behavior is technically challenging, particularly in unrestrained and naturalistic settings. Tools that are able to overcome these limitations leverage deep learning to achieve robust markerless motion capture, enabling characterization of behavior through precise positional tracking of body parts from standard videos of behavior. Unfortunately, like most deep learning systems, existing pose tracking software requires technical expertise to manage the complex software dependencies required for deep learning, and investment in expensive computational hardware (GPUs), both of which curtail equitable access to this technology. This project proposes cloudSLEAP, a platform that builds on the widely used multi-animal pose tracking software SLEAP to address these barriers by providing the infrastructure necessary to run the entire pose tracking workflow through cloud-based systems. This platform enables annotation, visualization and sharing pose tracking datasets directly from the browser, eliminating the need for installation and management of desktop-based software. cloudSLEAP will support data formats from all currently existing tools for pose tracking, and will be integrated with existing data standards and repositories such as NeurodataWithoutBorders and DANDI to ensure that all outputs of cloudSLEAP are standardized and FAIR-compliant. Users will be able to use cloudSLEAP to train pose tracking models on their own data through a cloud-based job orchestration system, eliminating the complexity of deep learning library dependencies. Leveraging the highly efficient model configurations provided by SLEAP, cloudSLEAP will provide users with free computational resources on the cloud to train pose models. This capability effectively eliminates the need for investment in local GPU hardware, thereby removing the single biggest barrier to entry for researchers from under-resourced institutions. The entire platform will be developed as open-source software on public repositories from the start, and all data used for testing and development will be freely available. A core goal for this project is to ensure that cloudSLEAP maximizes accessibility to behavior quantification technology to the widest range of practitioners. To this end, the first aim of this proposal will be to establish a broad-based community of beta testers that represents the diversity of institutions in the BRAIN Initiative and wider neuroscience community. Beta testers will be invited to collaborate throughout development via regular virtual Town Hall meetings, in-person events, direct communication channels and open discussion forums. These efforts will additionally produce documentation and didactic materials that will be used for training and education activities. By ensuring that diverse perspectives are included from the very onset of the project, cloudSLEAP will enable truly equitable access and dissemination of a crucial part of the modern neuroscience toolkit.