CRII: III: Harnessing Deep-Learning to Simplify Biological Inference from Complex Imaging Data

正在以前所未有的规模制作高分辨率成像数据。需要计算机视觉的进步来应对理解和提取这些图像中有意义的信息的挑战，以准确地告知科学家和医生。为了检测每个图像中的内容，发现信息模式出现的位置，并确定这些模式中的差异可能意味着什么，需要部署深度学习方法。这些方法需要训练数据集，这些数据集将向网络通知图像中有意义的模式，从而实现正确的分类和量化。一个尚未解决的挑战是开发适当的训练数据，代表在真实的世界中观察到的变化，并实现适当的计算方法来科普这种变化。该奖项有两个目标：（1）探索高度可变数据集上模型性能的变化;（2）将生物数据收集过程与深度学习模型的开发相统一，以分析这些数据。前两个目标开发了一个深度学习框架，该框架在高度可变的数据（来自从野外收集的非模型生物）上进行训练，并在不同放大率、染色程序和解剖平面的受控变化水平下成像。第三个目标还涉及在教育背景下的数据变化的概念，它开发了一个主动学习研讨会，其中两种不同的数据类型（图像与声音文件），代表相同的数据源（蝙蝠物种），教学生如何提取不同的功能，以代表相同的分类方案。这三个概念将数据收集联系在一起，即研究人员和学生如何收集样本并对其进行成像，然后部署深度学习架构来分析这些复杂的数据。尽管生物学和方法学来源的数据存在显著差异，但生物医学和组织学成像深度学习的一个突出需求是准确的模型预测。该项目探索了卷积神经网络和预训练网络（例如，U-Net）在肠的蝙蝠组织学数据的分类结果的预测中的应用。它还产生了深度学习图像分割的新模型，以量化通常手动执行且目前不可扩展的组织学数据。该网络将利用调查人员在整个职业生涯中获得的大量训练数据集，并将这些训练数据与预先训练的模型联合收割机结合起来，以提高准确性。该奖项的产品将是应用程序编程接口（API）的实施，用户可以上传自己的图像，这将进一步有助于训练数据集，并允许加快高通量图像处理。该项目将生物学和计算机科学中的多个不同学科联系起来，利用神经网络算法的最新进展来解决医学中组织学数据的特性，以解决比较形态学，解剖学和进化中出现的复杂分析问题。该项目的最终目标是为研究生开发一个研讨会课程模块，以建立深度学习神经网络，从声学呼叫库中识别蝙蝠物种，将生态数据收集与计算生物学和机器学习结合起来。 具体来说，学生将生成神经网络，根据他们记录的声音呼叫对不同的蝙蝠物种进行分类。该课程是独一无二的，因为它使学生接触到数据信息学的整个管道，包括通过在现场收集蝙蝠声学数据进行数据采集，以不同格式处理数据，生成网络并分析准确性。了解所引入的变化和噪声的来源以及如何克服这种变化是一个关键的学习目标。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Imaging data at high resolutions is being produced at unprecedented scales. Advances in computer vision are required to meet the challenge of understanding and extracting meaningful information in these images to inform scientists and physicians accurately and precisely. To detect what is in each image, discover where the informative patterns occur, and determine what differences in these patterns may mean requires the deployment of deep learning methods. These methods require training data sets that will inform the network of meaningful patterns in the image that will enable proper classification and quantification. An unmet challenge is developing the proper training data that represents variation observed in the real world and implementing the appropriate computational methods to cope with this variation. This award has two goals: (1) the exploration of variation in model performance on highly variable data sets; and, (2) the unification of the processes of biological data collection with the development of deep learning models to analyze this data. The first two objectives develop a deep-learning framework that is trained on highly variable data (from non-model organisms collected from the wild) and imaged under controlled levels of variation at varying magnifications, staining procedures, and anatomical planes. The third objective also addresses the concept of data variation in an educational context, which develops an active learning seminar where two different data types (images versus sound files) that represent the same data source (bat species), teaching students how different features can be extracted to represent the same classification scheme. All three concepts thread together data collection; that is, how samples are collected and imaged by the investigator and students and followed by the deployment of deep-learning architectures to analyze this complex data.An outstanding need in deep learning with biomedical and histological imaging is accurate model prediction despite notable variation in data driven by both biological and methodological sources. The project explores the performance of the combination of convolutional neural networks and pre-trained networks (e.g., U-Net) in the prediction of categorical outcomes of bat histological data of the intestine. It also produces novel models of deep-learning image segmentation to quantify histological data that is often performed manually and currently not scalable. The networks will make use of numerous training data sets obtained throughout the career of the investigator and combine this training data with pre-trained models to improve accuracy. The products of the award will be the implementation of an application programming interface (API) in which users can upload their own images, which will further contribute to training data sets and allow for expedited high-throughput image processing. This project bridges multiple disparate disciplines in biology and computer science, leveraging the properties of recent advances in neural network algorithms for histological data in medicine to solve complex analytical issues that arise in comparative morphology, anatomical sciences, and evolution. A final objective of the project is to also develop a seminar course module for graduate students to build deep-learning neural networks to identify bat species from acoustic call libraries, uniting ecological data collection with computational biology and machine learning. Specifically, students will generate neural networks to classify different bat species based on acoustic calls that they record. The course is unique, as it exposes students to the entire pipeline of data informatics, including data acquisition via collecting bat acoustic data in the field, to processing the data in different formats, to generating the networks, and analyze the accuracies. Understanding sources of variation and noise that is introduced and how to overcome this variation is a key learning objective.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.