CRII: III: Harnessing Deep-Learning to Simplify Biological Inference from Complex Imaging Data

高分辨率的成像数据正在以前所未有的规模产生。计算机视觉的进步需要满足理解和提取这些图像中有意义的信息的挑战，以便准确地告知科学家和医生。为了检测每个图像中的内容，发现信息模式发生的位置，并确定这些模式中的差异可能意味着什么，需要部署深度学习方法。这些方法需要训练数据集，这些数据集将告知网络图像中有意义的模式，从而能够进行适当的分类和量化。一个未解决的挑战是开发合适的训练数据来表示在现实世界中观察到的变化，并实现适当的计算方法来处理这种变化。该奖项有两个目标：(1)探索高度可变数据集上模型性能的变化；(2)将生物数据收集过程与开发深度学习模型相统一，以分析这些数据。前两个目标开发了一个深度学习框架，该框架在高度可变的数据（来自从野外收集的非模式生物）上进行训练，并在不同放大倍数、染色程序和解剖平面的受控变化水平下进行成像。第三个目标还涉及教育背景下数据变化的概念，它开发了一个主动学习研讨会，其中两种不同的数据类型（图像与声音文件）表示相同的数据源（蝙蝠物种），教学生如何提取不同的特征来表示相同的分类方案。这三个概念将数据收集联系在一起；也就是说，研究者和学生如何收集样本并对其进行成像，然后部署深度学习架构来分析这些复杂的数据。在生物医学和组织学成像的深度学习中，一个突出的需求是准确的模型预测，尽管由生物学和方法学来源驱动的数据存在显着差异。该项目探讨了卷积神经网络和预训练网络（例如U-Net）在预测蝙蝠肠道组织学数据分类结果方面的结合性能。它还产生了新的深度学习图像分割模型，以量化通常手动执行且目前不可扩展的组织学数据。该网络将利用调查员整个职业生涯中获得的大量训练数据集，并将这些训练数据与预训练模型相结合，以提高准确性。该合同的产品将实现应用程序编程接口（API），用户可以在其中上传自己的图像，这将进一步有助于训练数据集，并允许加速高通量图像处理。该项目将生物学和计算机科学中的多个不同学科联系起来，利用神经网络算法在医学组织学数据方面的最新进展来解决比较形态学、解剖科学和进化中出现的复杂分析问题。该项目的最终目标是为研究生开发一个研讨会课程模块，以建立深度学习神经网络，从声音呼叫库中识别蝙蝠物种，将生态数据收集与计算生物学和机器学习结合起来。具体来说，学生将根据记录的声音叫声生成神经网络，对不同种类的蝙蝠进行分类。这门课程是独一无二的，因为它让学生接触到数据信息学的整个管道，包括通过收集野外蝙蝠声学数据来获取数据，以不同格式处理数据，生成网络，并分析准确性。了解引入的变化和噪声的来源以及如何克服这种变化是一个关键的学习目标。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Imaging data at high resolutions is being produced at unprecedented scales. Advances in computer vision are required to meet the challenge of understanding and extracting meaningful information in these images to inform scientists and physicians accurately and precisely. To detect what is in each image, discover where the informative patterns occur, and determine what differences in these patterns may mean requires the deployment of deep learning methods. These methods require training data sets that will inform the network of meaningful patterns in the image that will enable proper classification and quantification. An unmet challenge is developing the proper training data that represents variation observed in the real world and implementing the appropriate computational methods to cope with this variation. This award has two goals: (1) the exploration of variation in model performance on highly variable data sets; and, (2) the unification of the processes of biological data collection with the development of deep learning models to analyze this data. The first two objectives develop a deep-learning framework that is trained on highly variable data (from non-model organisms collected from the wild) and imaged under controlled levels of variation at varying magnifications, staining procedures, and anatomical planes. The third objective also addresses the concept of data variation in an educational context, which develops an active learning seminar where two different data types (images versus sound files) that represent the same data source (bat species), teaching students how different features can be extracted to represent the same classification scheme. All three concepts thread together data collection; that is, how samples are collected and imaged by the investigator and students and followed by the deployment of deep-learning architectures to analyze this complex data.An outstanding need in deep learning with biomedical and histological imaging is accurate model prediction despite notable variation in data driven by both biological and methodological sources. The project explores the performance of the combination of convolutional neural networks and pre-trained networks (e.g., U-Net) in the prediction of categorical outcomes of bat histological data of the intestine. It also produces novel models of deep-learning image segmentation to quantify histological data that is often performed manually and currently not scalable. The networks will make use of numerous training data sets obtained throughout the career of the investigator and combine this training data with pre-trained models to improve accuracy. The products of the award will be the implementation of an application programming interface (API) in which users can upload their own images, which will further contribute to training data sets and allow for expedited high-throughput image processing. This project bridges multiple disparate disciplines in biology and computer science, leveraging the properties of recent advances in neural network algorithms for histological data in medicine to solve complex analytical issues that arise in comparative morphology, anatomical sciences, and evolution. A final objective of the project is to also develop a seminar course module for graduate students to build deep-learning neural networks to identify bat species from acoustic call libraries, uniting ecological data collection with computational biology and machine learning. Specifically, students will generate neural networks to classify different bat species based on acoustic calls that they record. The course is unique, as it exposes students to the entire pipeline of data informatics, including data acquisition via collecting bat acoustic data in the field, to processing the data in different formats, to generating the networks, and analyze the accuracies. Understanding sources of variation and noise that is introduced and how to overcome this variation is a key learning objective.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.