Enabling Next Generation Machine Learning for Large Scale Image Analysis

项目总结/摘要 深度学习通过提供具有临床意义的结果来改变医学图像分析， 比如前列腺癌检测和肺癌筛查。FDA批准全切片数字病理学 用于初步诊断的WSIs进一步增加了人工智能的兴趣、采用和投资， 人工智能（AI）技术用于病理学。使用患者级标签（PLL）从大型医学图像中学习， 成为活跃的计算病理学研究领域。病理诊断或临床结局等PLL 是通过医疗保健操作产生的，通常很容易获得。与学习范式相反， 其取决于图像的专家注释（例如，描绘肿瘤区域）并且因此是时间密集的 并且仅限于较小的队列，使用PLL直接从WSI进行培训将允许开发现实的 训练数据集包含成千上万的主题，可以产生具有临床意义的模型， 副牧师GPU加速器在推进计算深度学习方法方面发挥了重要作用。 病理学工具机器学习框架（MLF），例如，Pytorch和TensorFlow为研究人员提供了 抽象以快速开发利用GPU的模型。GPU和MLF的发展是由以下因素推动的： 小图像的分析，因此将这些工具直接应用于WSI或其他大型医学图像， 体积磁共振或计算机断层摄影是具有挑战性的。使医学成像问题适应 小图像范例导致许多折衷，导致次优性能、增加的简单性， 管理工作，以及增加的软件/设计复杂性（例如，基于修补程序的技术或多实例 学习）。因此，通过直接处理WSI图像，从PLL开发可扩展的ML模型 通过深度学习管道在GPU上是不可行的。最近的努力，使用统一的GPU内存或 流方法来克服GPU内存限制，并尝试在WSI规模上执行端到端训练 已经证明了比注释或MIL具有上级性能。然而，这些方法要么很慢（由于 次优的数据移动策略）、适应/使用复杂，或高度特定于给定的网络架构 （限制了开发和探索新架构的能力）。更加通用、高效和用户友好 需要框架来允许开发WSI规模的深度学习。 本项目将开发一个强大的软件框架，以促进无缝开发和使用可扩展的 ML模型，对处理图像的大小没有任何限制，不受有限内存的限制 GPU的能力。SSTEP（Seamless Scalable Tensor-Expression Execution via Partitioning，无缝可扩展张量表达式执行） ware框架将允许可扩展和可移植的神经网络模型，直接处理全高分辨率 在任何（多）GPU平台上，可以生成任意大小的图像，用于训练或推理。SSTEP将允许开发 新的深度学习范式，专门为医疗应用而构建，并将使开发人员能够 使用熟悉的MLF（PyTorch或TensorFlow）快速创建和评估这些工具。

Project Summary/Abstract Deep learning has transformed medical image analysis by delivering clinically meaningful results on challenging problems like prostate cancer detection and lung cancer screening. FDA approval of whole-slide digital pathology imaging (WSIs) for primary diagnosis is further increasing interest, adoption, and investment in artificial intelli- gence (AI) technology for pathology. Learning from large medical images using patient-level labels (PLLs) has become an active computational pathology research area. PLLs such as pathology diagnosis or clinical outcomes are generated through healthcare operations and are often readily available. In contrast to learning paradigms that depend on the expert annotation of images (e.g., delineating tumor regions) and are therefore time-intensive and limited to smaller cohorts, training directly from WSIs using PLLs will allow the development of realistic training datasets containing tens-of-thousands of subjects that can produce models with clinically-meaningful ac- curacy. GPU accelerators have played a significant role in advancing deep learning methods for computational pathology tools. Machine Learning Frameworks (MLFs), e.g., Pytorch and TensorFlow, provide researchers with abstractions to quickly develop models that utilize GPUs. The evolution of GPUs and MLFs has been driven by the analysis of small images, and so applying these tools directly to WSIs or other large medical images like volumetric magnetic resonance or computed tomography is challenging. Adapting medical imaging problems to the small image paradigm leads to many compromises resulting in suboptimal performance, increased imple- mentation effort, and increased software/design complexity (e.g., patch based techniques or multiple instance learning). As a result, the development of scalable ML models from PLLs by directly processing WSI images through a deep learning pipeline is infeasible today on GPUs. Recent efforts that use unified GPU memory or streaming approaches to overcome GPU memory limits and attempt to perform end-to-end training at WSI scale have demonstrated superior performance to annotation or MIL. However, these approaches are either slow (due to suboptimal data movement strategies), complex to adapt/use, or highly specific to a given network architecture (limiting the ability to develop and explore new architectures). More general-purpose, efficient, and user-friendly frameworks are needed to allow the development of WSI scale deep learning. This project will develop a robust software framework to facilitate seamless development and use of scalable ML models, without the imposition of any limits on the sizes of handled images, unhindered by the limited memory capacity in GPUs. The proposed SSTEP (Seamless Scalable Tensor-Expression Execution via Partitioning) soft- ware framework will allow scalable and portable neural network models that directly process full high-resolution images of arbitrary size for training or inference, on any (multi) GPU platform. SSTEP will allow the development of novel deep learning paradigms that are purpose-built for medical applications, and will enable developers to rapidly create and evaluate these tools using familiar MLFs - PyTorch or TensorFlow.