cloudSLEAP: Maximizing accessibility to deep learning-based motion capture

cloudSLEAP：最大限度地提高基于深度学习的动作捕捉的可访问性

• 批准号: 10643661
• 负责人: Talmo D. Pereira
• 金额: $ 262.32万
• 依托单位: SALK INSTITUTE FOR BIOLOGICAL STUDIES
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10643661
• 关键词: Acceleration; Address; Adoption; Animal Behavior; Animal Model; Animals; BRAIN initiative; Behavior; Behavioral; Body part; Brain; Cloud Computing; Cloud Service; Collaborations; Communication; Communities; Complex; Computer Hardware; Computer software; Data; Data Set; Dedications; Democracy; Dependence; Development; Documentation; Ecosystem; Educational Activities; Ensure; Environment; Equity; Ethology; Event; Future; Goals; Industry; Informatics; Infrastructure; Institution; Internet; Investments; Label; Libraries; Modeling; Modernization; Motion; Neurosciences; Occupations; Online Systems; Outcome; Output; Persons; Posture; Qualifying; Research; Research Personnel; Resolution; Resources; Running; Services; Software Engineering; Standardization; System; Technical Expertise; Technology; Testing; Training; Training and Education; Training and Infrastructure; Visualization; Work; cloud based; computer infrastructure; computerized tools; computing resources; cost; cost effective; data archive; data format; data repository; data standards; deep learning; experience; gigabyte; graphical user interface; learning strategy; light weight; meetings; next generation; open source; public repository; recruit; tool; usability; virtual; virtual machine; web platform

cloudSLEAP – PROJECT SUMMARY/ABSTRACT Understanding how the brain produces complex behavior is a central goal of neuroscience, but quantifying behavior is technically challenging, particularly in unrestrained and naturalistic settings. Tools that are able to overcome these limitations leverage deep learning to achieve robust markerless motion capture, enabling characterization of behavior through precise positional tracking of body parts from standard videos of behavior. Unfortunately, like most deep learning systems, existing pose tracking software requires technical expertise to manage the complex software dependencies required for deep learning, and investment in expensive computational hardware (GPUs), both of which curtail equitable access to this technology. This project proposes cloudSLEAP, a platform that builds on the widely used multi-animal pose tracking software SLEAP to address these barriers by providing the infrastructure necessary to run the entire pose tracking workflow through cloud-based systems. This platform enables annotation, visualization and sharing pose tracking datasets directly from the browser, eliminating the need for installation and management of desktop-based software. cloudSLEAP will support data formats from all currently existing tools for pose tracking, and will be integrated with existing data standards and repositories such as NeurodataWithoutBorders and DANDI to ensure that all outputs of cloudSLEAP are standardized and FAIR-compliant. Users will be able to use cloudSLEAP to train pose tracking models on their own data through a cloud-based job orchestration system, eliminating the complexity of deep learning library dependencies. Leveraging the highly efficient model configurations provided by SLEAP, cloudSLEAP will provide users with free computational resources on the cloud to train pose models. This capability effectively eliminates the need for investment in local GPU hardware, thereby removing the single biggest barrier to entry for researchers from under-resourced institutions. The entire platform will be developed as open-source software on public repositories from the start, and all data used for testing and development will be freely available. A core goal for this project is to ensure that cloudSLEAP maximizes accessibility to behavior quantification technology to the widest range of practitioners. To this end, the first aim of this proposal will be to establish a broad-based community of beta testers that represents the diversity of institutions in the BRAIN Initiative and wider neuroscience community. Beta testers will be invited to collaborate throughout development via regular virtual Town Hall meetings, in-person events, direct communication channels and open discussion forums. These efforts will additionally produce documentation and didactic materials that will be used for training and education activities. By ensuring that diverse perspectives are included from the very onset of the project, cloudSLEAP will enable truly equitable access and dissemination of a crucial part of the modern neuroscience toolkit.

CloudSLEAP-项目摘要/摘要 了解大脑如何产生复杂的行为是神经科学的中心目标，但量化 行为在技术上是具有挑战性的，特别是在无拘无束和自然主义的环境中。能够使用的工具 克服这些限制利用深度学习实现强大的无标记运动捕获，从而实现 通过对标准行为视频中的身体部位进行精确的位置跟踪来表征行为。 不幸的是，像大多数深度学习系统一样，现有的姿势跟踪软件需要技术专业知识来 管理深度学习所需的复杂软件依赖关系以及昂贵的投资 计算硬件(GPU)，这两者都限制了对这项技术的公平访问。这个项目 提出了CloudSLEAP，这是一个建立在广泛使用的多动物姿势跟踪软件Sleap上的平台 通过提供运行整个姿势跟踪工作流所需的基础设施来解决这些障碍 通过基于云的系统。该平台支持注释、可视化和共享姿势跟踪 直接从浏览器获取数据集，无需安装和管理基于桌面的 软件。CloudSLEAP将支持当前所有现有姿势跟踪工具的数据格式，并将 与现有数据标准和存储库(如NeurodataWithout Borders和Dandi)集成以 确保CloudSLEAP的所有输出都是标准化的和公平兼容的。用户将能够使用 CloudSLEAP通过基于云的作业编排系统，根据自己的数据训练姿势跟踪模型， 消除深度学习库依赖的复杂性。利用高效的模式 由SLEAP提供的配置，CloudSLEAP将在 云来训练姿势模型。此功能有效地消除了对本地GPU的投资需求 硬件，从而消除了资源不足的研究人员进入的最大障碍 机构。整个平台将从一开始就作为公共存储库的开源软件开发， 所有用于测试和开发的数据都将免费提供。该项目的一个核心目标是确保 CloudSLEAP将行为量化技术的可访问性最大化到最大范围 从业者。为此，该提案的第一个目标将是建立一个基础广泛的测试版社区 代表大脑计划和更广泛的神经科学界机构多样性的测试者。 Beta测试员将被邀请通过定期的虚拟市政厅会议在整个开发过程中进行协作， 面对面的活动、直接的交流渠道和开放的论坛。这些努力将进一步 制作将用于培训和教育活动的文件和教学材料。通过 确保从项目一开始就包含不同的视角，CloudSLEAP将真正实现 平等获取和传播现代神经科学工具包的一个重要部分。

项目成果

Fast and efficient root phenotyping via pose estimation.

通过姿态估计快速有效地进行根表型分析。

• DOI: 10.1101/2023.11.20.567949
• 发表时间: 2023
• 期刊: bioRxiv : the preprint server for biology
• 作者: Berrigan,ElizabethM;Wang,Lin;Carrillo,Hannah;Echegoyen,Kimberly;Kappes,Mikayla;Torres,Jorge;Ai-Perreira,Angel;McCoy,Erica;Shane,Emily;Copeland,CharlesD;Ragel,Lauren;Georgousakis,Charidimos;Lee,Sanghwa;Reynolds,Dawn;Talgo,Ave
• 通讯作者: Talgo,Ave