Collaborative Research: LightningBug, An Integrated Pipeline to Overcome The Biodiversity Digitization Gap

合作研究：LightningBug，克服生物多样性数字化差距的综合管道

• 批准号: 2104151
• 负责人: Mark Hereld
• 金额: $ 49.4万
• 依托单位: University of Chicago
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2104151&HistoricalAwards=false
• 关键词: Collaborative; Research; LightningBug; Integrated; Pipeline

Insects are the largest and most diverse class of animals on our planet where they play essential roles in ecosystems and the services those provide to society. Entomologists have long been engaged in collecting, preserving and depositing nearly one billion insect specimens at natural history museums around the globe. These collections form the basis for much of our knowledge about insects and provide critical information about the past from which scientists can assess current and future global change impacts. To fully realize the value of these collections, data from insect specimens must first be digitized. However, their small size, delicate structures, and traditional storage and labeling methods creates enormous challenges for large-scale digitization. Consequently, at present, only 5% of specimens have transcribed labels and less than 1% of specimens are imaged. The LightningBug project will break through this digitization bottleneck by establishing a semi-automated workflow involving advancements in robotic multi-view imaging, information extraction and 3D reconstruction. Results from this work will provide researchers with the unprecedented capability to capture specimen metadata representing time, place and taxonomic identity along with accurate three-dimensional surface morphology representing color and shape. These investigators expect LightningBug and related technologies will promote ecomorphological studies at a scale that has not been possible to date.The LightningBug project seeks to create an end-to-end pipeline for high-throughput data acquisition from pinned insects in entomological collections. To accomplish this goal, it will: (1) further develop an existing hardware and software platform to capture multi-view imagery of both labels and specimens; (2) build robust algorithms to automatically process fragmentary views of multiple labels into separate integrated “virtual labels;" (3) connect virtual labels to structured text extraction services; and (4) apply photogrammetric analysis to assemble the 3D shape and structure of specimens. Guided by real-world science use cases that highlight the use of specimen-based multi-view imaging in studies of global change and functional morphology, the entomological collections of the Yale Peabody Museum and the Harvard Museum of Comparative Zoology will be used in rigorous test-case implementations. Results will include robust sets of annotated multi-view images, 3D models of specimens (point clouds, textured meshes), 2D reconstructed “virtual labels” and digitized specimen metadata generated from those labels. These digital specimens will present new challenges for data preservation and access, but they will also catalyze new solutions for large-scale storage and delivery of research imagery. This challenge will be addressed via a partnership with MorphoSource to develop a linked institutional repository model for data access to large digital assets such as those produced by multi-view imaging. Ultimately, the ability to capture multi-view image suites and generate virtual specimens at scale will permit new avenues for remote access to research resources, and enable the application of computer vision and machine learning to trait identification and evolution, species recognition and new species discovery. Label data from pinned insects will give researchers access to critical temporal and geospatial information necessary for relating changes in biodiversity to other biotic and environmental variables. It will also provide collections staff with a complete digital portrait of their holdings, which can enable historical research, streamline collections use and tracking, and improve data quality control. Results from this project will also have applications beyond the natural history collections and research communities, such as computer graphics, product imaging, motion pictures, 3D animation, virtual and augmented realities, and education. More information and results from this project can be found at http://lightningbug.techThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

昆虫是地球上最大、最多样化的动物纲，它们在生态系统和为社会提供的服务中发挥着重要作用。昆虫学家长期以来一直致力于在全球自然历史博物馆收集、保存和存放近十亿昆虫标本。这些藏品构成了我们有关昆虫的大部分知识的基础，并提供了有关过去的重要信息，科学家可以从中评估当前和未来的全球变化影响。为了充分实现这些藏品的价值，必须首先将昆虫标本的数据数字化。然而，它们体积小、结构精致，以及传统的存储和标签方法给大规模数字化带来了巨大的挑战。因此，目前只有 5% 的标本有转录标签，不到 1% 的标本进行了成像。 LightningBug 项目将通过建立半自动化工作流程来突破这一数字化瓶颈，该工作流程涉及机器人多视图成像、信息提取和 3D 重建方面的进步。这项工作的结果将为研究人员提供前所未有的能力，以捕获代表时间、地点和分类身份的标本元数据以及代表颜色和形状的准确三维表面形态。这些研究人员预计 LightningBug 和相关技术将以迄今为止不可能的规模促进生态形态学研究。LightningBug 项目旨在创建一个端到端管道，用于从昆虫学收藏中的固定昆虫中获取高通量数据。为了实现这一目标，它将：（1）进一步开发现有的硬件和软件平台，以捕获标签和标本的多视图图像； (2) 构建强大的算法，自动将多个标签的碎片视图处理成单独的集成“虚拟标签”； (3) 将虚拟标签连接到结构化文本提取服务； (4) 应用摄影测量分析来组装样本的 3D 形状和结构。在现实世界的科学用例的指导下，强调基于标本的多视图成像在全球变化和功能形态学研究中的使用，耶鲁大学皮博迪博物馆和哈佛大学比较动物学博物馆的昆虫学收藏将用于严格的测试用例实施。结果将包括一组强大的带注释的多视图图像、样本的 3D 模型（点云、纹理网格）、2D 重建的“虚拟标签”以及从这些标签生成的数字化样本元数据。这些数字标本将为数据保存和访问带来新的挑战，但它们也将催生大规模存储和研究图像交付的新解决方案。这一挑战将通过与 MorphoSource 合作开发一个链接的机构存储库模型来解决，该模型用于对大型数字资产（例如多视图成像产生的资产）进行数据访问。最终，捕获多视图图像套件和大规模生成虚拟标本的能力将为远程访问研究资源提供新途径，并使计算机视觉和机器学习应用于性状识别和进化、物种识别和新物种发现。来自固定昆虫的标签数据将使研究人员能够获得将生物多样性变化与其他生物和环境变量联系起来所必需的关键时间和地理空间信息。它还将为馆藏工作人员提供其馆藏的完整数字化肖像，从而可以进行历史研究，简化馆藏使用和跟踪，并提高数据质量控制。该项目的成果还将应用于自然历史收藏和研究领域之外的领域，例如计算机图形、产品成像、电影、3D 动画、虚拟和增强现实以及教育。有关该项目的更多信息和结果，请访问 http://lightningbug.tech 该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。