HDR Institute: Imageomics: A New Frontier of Biological Information Powered by Knowledge-Guided Machine Learning

HDR 研究所：图像组学：知识引导机器学习驱动的生物信息新领域

• 批准号: 2118240
• 负责人: Tanya Berger-Wolf
• 金额: $ 1496.91万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Cooperative Agreement
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118240&HistoricalAwards=false
• 关键词: HDR; Institute; Imageomics; New; Frontier

The traits that characterize living organisms, in particular, their morphology, physiology, behavior and genetic make-up, enable them to cope with forces of the physical as well as the biological and social environments that impinge on them. Moreover, since function follows form, traits provide the raw material upon which natural selection operates, thus shaping evolutionary trajectories and the history of life. Interestingly, most living organisms, from microscopic microbes to charismatic megafauna, reveal themselves visually and are routinely captured in copious images taken by humans from all walks of life. The resulting massive amount of image data has the potential to further understanding of how multifaceted traits of organisms shape the behavior of individuals, collectives, populations, and the ecological communities they live in, as well as the evolutionary trajectories of the species they comprise. Images are increasingly the currency for documenting the details of life on the planet, and yet traits of organisms, known or novel, cannot be readily extracted from them. Just like with genomic data two decades ago, our ability to collect data far outstrips our ability to extract biological insight from it. The Institute will establish a new field of Imageomics, in which biologists utilize machine learning (ML) algorithms to analyze vast stores of existing image data—especially publicly funded digital collections from national centers, field stations, museums and individual laboratories—to characterize patterns and gain novel insights on how function follows form in all areas of biology to expand our understanding of the rules of life on Earth and how it evolves. This Institute will introduce structured knowledge from the biological sciences to guide and structure ML algorithms to enable biological trait discovery from images, establishing the field of Imageomics. With images captured and annotated by scientists and the public serving as the basis for the work, the Institute’s convergent approach uses structured biological knowledge to provide scientifically validated inductive biases and rich supervision for ML, and ML will in turn enrich the body of biological knowledge. The resulting ML models and tools will help to make what was hidden visible, so that scientists from a wide range of biological communities can discover and infer the traits of organisms; assess shared similarities and differences between individuals, populations, and species; and come to see the world in new ways. Imageomics will accelerate and transform the biomedical, agricultural, and basic biological sciences as they seek to understand and control genes that relate to specific phenotypes and enable an overarching understanding of how the genome evolved in tandem with the organismal phenome. Because traits are the essential links between genes and the environment, using ML to help characterize them will lead to emergent understandings of how they function. Harnessing the insights that arise from these new visualizations will stimulate the use of new genetic technologies, such as CRISPR gene editing, and more nuanced ecological practices, such as modified land use schemes that emerge from better understanding the connections between individual decision-making within species and their impact on their population dynamics. With the emergence of new and better targeted practices that generate fewer unintended consequences, the new linkages resulting from a better understanding of traits and their consequences will bolster the nation’s bioeconomy. In addition, by leveraging and expanding existing diverse, inclusive and intellectually wide-ranging collaborative networks, the Institute will also educate the next generation of scientists and engage the broader public in scientific inquiry and knowledge discovery so that Imageomics can transform and democratize science for public good.This project is part of the National Science Foundation's Big Idea activities in Harnessing the Data Revolution (HDR). This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Biological Infrastructure within the NSF Directorate for Biological Sciences, and by the Division of Information and Intelligent Systems within the Directorate for Computer and Information Science and Engineering.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

生物体的特征，特别是它们的形态、生理、行为和基因构成，使它们能够应付影响它们的物理以及生物和社会环境的力量。此外，由于功能遵循形式，特征为自然选择提供了原材料，从而塑造了进化轨迹和生命的历史。有趣的是，大多数生物体，从微小的微生物到有魅力的巨型动物，都能在视觉上展示自己，并经常被各行各业的人们拍摄到丰富的图像。由此产生的大量图像数据有可能进一步了解生物体的多方面特征如何塑造个体、集体、种群和它们所生活的生态群落的行为，以及它们所组成的物种的进化轨迹。图像越来越成为记录地球上生命细节的货币，然而，无论是已知的还是新的生物特征，都不能轻易从中提取出来。就像20年前的基因组数据一样，我们收集数据的能力远远超过了我们从中提取生物学见解的能力。该研究所将建立一个新的图像组学领域，其中生物学家利用机器学习（ML）算法来分析大量现有的图像数据-特别是来自国家中心，野外站，博物馆和个人实验室的公共资助的数字收藏-表征模式并获得关于生物学所有领域中功能如何遵循形式的新见解，以扩大我们对地球上生命规则的理解及其演变方式。该研究所将引入生物科学的结构化知识来指导和构建机器学习算法，以实现从图像中发现生物特征，建立图像组学领域。以科学家和公众捕获和注释的图像作为工作的基础，研究所的收敛方法使用结构化的生物知识为ML提供科学验证的归纳偏差和丰富的监督，ML将反过来丰富生物知识体系。由此产生的机器学习模型和工具将有助于使隐藏的东西变得可见，以便来自广泛生物群落的科学家可以发现和推断生物体的特征；评估个体、种群和物种之间共有的相似性和差异性；用新的方式看世界。图像组学将加速和改变生物医学、农业和基础生物科学，因为它们寻求理解和控制与特定表型相关的基因，并使人们能够全面了解基因组如何与生物体表型一起进化。因为性状是基因和环境之间的重要联系，使用ML来帮助表征它们将导致对它们如何运作的新理解。利用从这些新的可视化中产生的见解将刺激使用新的基因技术，例如CRISPR基因编辑，以及更细致的生态实践，例如通过更好地理解物种内部个体决策及其对种群动态的影响之间的联系而产生的修改土地使用计划。随着新的和更好的有针对性的做法的出现，产生更少的意外后果，更好地理解性状及其后果所产生的新联系将支持国家的生物经济。此外，通过利用和扩大现有的多样化、包容性和智力广泛的合作网络，该研究所还将教育下一代科学家，并让更广泛的公众参与科学探究和知识发现，以便Imageomics能够为公共利益转变和民主化科学。该项目是美国国家科学基金会“利用数据革命”（HDR）大创意活动的一部分。该奖项由先进网络基础设施办公室颁发，由美国国家科学基金会生物科学理事会生物基础设施部和计算机与信息科学与工程理事会信息与智能系统部联合支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones

• DOI: 10.1109/cvpr52688.2022.01504
• 发表时间: 2022-02
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Chan Hee Song;Jihyung Kil;Tai-Yu Pan;Brian M. Sadler;Wei-Lun Chao;Yu Su

Learning with Free Object Segments for Long-Tailed Instance Segmentation

• DOI: 10.1007/978-3-031-20080-9_38
• 发表时间: 2022-02
• 作者: Cheng Zhang;Tai-Yu Pan;Tianle Chen;Jike Zhong;Wen-juan Fu;Wei-Lun Chao

Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish Species

生物多样性图像质量元数据增强了鱼类物种的卷积神经网络分类

• 发表时间: 2021
• 期刊: Metadata and Semantic Research. MTSR 2020. Communications in Computer and Information Science
• 作者: Leipzig, J.;Bakis, Y;Wang, X;Elhamod, M.;Diamond, K.;Dahdul, W;Karpante, A;Maga, M;Mabee, P;Bart, H
• 通讯作者: Bart, H

Learning to Detect Mobile Objects from LiDAR Scans Without Labels

• DOI: 10.1109/cvpr52688.2022.00120
• 发表时间: 2022-03
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Yurong You;Katie Luo;Cheng Perng Phoo;Wei-Lun Chao;Wen Sun;Bharath Hariharan;M. Campbell;Kilian Q. Weinberger