CISE-MSI: DP: SaTC: CyIndiBee - CyberInfrastructure for video analysis of individual bee behavior

CISE-MSI：DP：SaTC：CyIndi​​Bee - 用于单个蜜蜂行为视频分析的网络基础设施

• 批准号: 2318597
• 负责人: Remi Megret
• 金额: $ 59万
• 依托单位: University of Puerto Rico-Rio Piedras
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318597&HistoricalAwards=false
• 关键词: CISE; MSI; DP; SaTC; CyIndiBee

The goal of the CyIndiBee project is to build an innovative cyberinfrastructure showcasing the use of modern Artificial Intelligence (AI) approaches in the study of pollinator behavior in the Caribbean. Pollinators are a key part of our food production system and play a critical role in the life cycle of plants and their ecosystems. Climate change and other anthropogenic activities are endangering pollinators and their habitats, causing behavioral changes, and leading to serious consequences for humans, especially in areas with scarce ecological diversity. A keen understanding of the complex effects that environmental changes, contaminants, and other factors produced on pollinator behavior and their biological mechanisms is needed urgently. This project will develop new computer vision, software and data analysis tools to expand our capacity to measure the individual behavior of a large number of pollinators in order to gather more detailed data over longer periods of time. The contributions will lead to mature and robust tools for automatic video monitoring of insects, which can be readily used and deployed in the field. The project will consolidate the partnership between the Computer Science department and the biology department at UPR Rio Piedras (UPR-RP), a Hispanic Minority Serving Institution (MSI). It will integrate undergraduate and graduate student training into research-intensive activities, thus creating a critical mass of closely interacting professors and students conducting state-of-the-art transdisciplinary research. The project will increase UPR-RP's capacity in the Computer Science field to promote innovation and growth in AI related research. This project is a first step towards transforming UPRRP into the reference research center in the Caribbean on the topic of Artificial Intelligence applied to transdisciplinary research in climate change.This project will develop an integrated cyberinfrastructure for pollinator video monitoring that combines (i) new deep learning models for detection and characterization of bee behavior, phenotype, and identity, (ii) a computational platform to collect monitoring data and perform behavior analysis with graphical interfaces usable by both the biology end-user and AI researchers, (iii) tools for the analysis of long-term individual behavior. The deep learning models leverage the Vision Transformer architecture to provide powerful pre-training using Masked Image Modeling on unannotated video data, thus reducing the need for large annotation efforts when deploying new collection setups. They will also provide flexibility with an extendable system of trainable query tokens to extract various types of information (pose, tag, marking, morphology, presence of pollen…) from the same latent representations. The computational platform will integrate a web application for video visualization and behavior annotation, with interactive dashboards for the analysis of individual behavior and phenotype, from data collected from long-term video monitoring. These tools will be applied to individual analyses to recognize patterns of shift-work and seasonal activity in both marked and unmarked bees. The large-scale analysis enabled by the new cyber-infrastructure will be demonstrated by the creation of two new curated datasets: a large-scale honeybee re-identification dataset, and a foraging behavior dataset combining multiple colonies and foraging trips at the individual and colony level including the presence of pollen and morphology of the bees.This project is jointly funded by the CISE MSI Research Expansion Program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

CyIndi​​Bee 项目的目标是建立一个创新的网络基础设施，展示现代人工智能 (AI) 方法在加勒比地区授粉媒介行为研究中的应用。传粉媒介是我们粮食生产系统的关键组成部分，在植物及其生态系统的生命周期中发挥着关键作用。气候变化和其他人类活动正在危及传粉媒介及其栖息地，导致行为改变，并给人类带来严重后果，特别是在生态多样性稀缺的地区。迫切需要深入了解环境变化、污染物和其他因素对传粉媒介行为及其生物学机制产生的复杂影响。该项目将开发新的计算机视觉、软件和数据分析工具，以扩大我们测量大量授粉媒介个体行为的能力，以便在更长的时间内收集更详细的数据。这些贡献将带来成熟而强大的昆虫自动视频监控工具，可以在现场轻松使用和部署。该项目将巩固拉美裔少数族裔服务机构 (MSI) UPR Rio Piedras (UPR-RP) 计算机科学系和生物学系之间的合作关系。它将把本科生和研究生的培训整合到研究密集型活动中，从而培养出一批密切互动的教授和学生，进行最先进的跨学科研究。该项目将提高 UPR-RP 在计算机科学领域的能力，以促进人工智能相关研究的创新和增长。该项目是将 UPRRP 转变为加勒比地区关于人工智能应用于气候变化跨学科研究主题的参考研究中心的第一步。该项目将开发用于传粉媒介视频监测的综合网络基础设施，该基础设施结合了（i）用于检测和表征蜜蜂行为、表型和身份的新深度学习模型，（ii）收集监测数据并通过图形进行行为分析的计算平台 生物学最终用户和人工智能研究人员都可以使用的界面，(iii) 用于分析长期个人行为的工具。深度学习模型利用 Vision Transformer 架构，使用 Masked Image Modeling 对未注释的视频数据提供强大的预训练，从而减少部署新收集设置时对大量注释工作的需求。它们还将提供可扩展的可训练查询令牌系统的灵活性，以从相同的潜在表示中提取各种类型的信息（姿势、标签、标记、形态、花粉的存在……）。该计算平台将集成用于视频可视化和行为注释的网络应用程序，以及用于根据长期视频监控收集的数据分析个人行为和表型的交互式仪表板。这些工具将应用于个体分析，以识别标记和未标记蜜蜂的轮班工作和季节性活动模式。新的网络基础设施支持的大规模分析将通过创建两个新的精选数据集来证明：一个大规模蜜蜂重新识别数据集，以及一个结合了多个蜂群以及个体和蜂群层面的觅食行程的觅食行为数据集，包括花粉的存在和蜜蜂的形态。该项目由 CISE MSI 研究扩展计划和既定刺激计划共同资助 竞争性研究 (EPSCoR)。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。