Extracting Knowledge from 100 years of Microstructural Images: Using Machine Vision and Machine Learning to Address the Microstructural Big Data Challenge

材料研究部;土木，机械和制造创新部;以及高级网络基础设施部为该奖项提供资金。它支持研究和教育，以收集，分析和比较来自大量来源的材料数据。所有的固体物体--从机翼到煎锅--都有肉眼通常看不到的微观结构。这种结构决定了材料的整体性质-例如，它是强还是弱。在过去的世纪里，材料科学家们通过使用显微镜拍摄照片（称为显微照片）来研究这些结构。然后，他们测量显微图像中的重要特征，并将这些测量结果与材料的属性相关联。就像个人摄影一样，数码相机使材料科学家能够拍摄更多照片，并比以往任何时候都更好地利用它们。此外，较旧的显微照片已被扫描到数字档案中。材料科学家现在面临着一组图像，这些图像太大，太多样化，无法手动分析。幸运的是，计算机科学家们已经开发出了“机器视觉”计算机程序，通过在某种意义上模仿人类如何看待物体来识别大量图像中的相似性。该项目将从多个来源收集显微图像，并将其归档，然后使用机器视觉程序自动搜索、排序和分类，无需人工干预。通过以前所未有的规模合成显微图像数据，该项目为发现微观结构和材料性能之间的新联系奠定了基础。研究结果将有助于改进现有材料，甚至开发新材料。材料研究部、土木、机械和制造创新部以及高级网络基础设施部为该奖项提供资金。它支持研究和教育，以收集，分析和比较来自大量来源的材料数据。在过去的100年里，材料科学家在获取、分析和比较微观结构图像方面取得了很大进展。这些努力的大部分是针对深入了解特定的材料系统或微观结构的类别。当已知可能的微观结构特征的目录时，成像技术可以利用明确定义的特征特性来高精度地分析微观结构。然而，当感兴趣的特征不是先验已知的时，这些方法变得棘手、不准确或完全失败。因此，通常，它们仅应用于由人类专家选择的预先选择的一组显微照片。相比之下，这项工作的目标是开发一种通用的方法来找到有用的显微照片之间的关系，没有假设什么功能可能存在。这种方法可以利用过去二十年数字数据的爆炸式增长，有效地调查可用微结构的广度，而无需大量的人为干预。该项目利用计算机科学的最新进展，应用数据科学概念的子集-包括数据采集，机器视觉和机器学习-来推进微结构科学。其结果将是一个框架，用于寻找材料系统内部和跨材料系统的微观结构图像之间的联系，这将支持从计算工具到发现科学的成果，包括：-新的开源工具，用于从各种数字档案中提取显微照片和相关元数据，包括互联网，PDF文档和本地存储介质。一个全面的公开显微照片数据库，具有传统的基于文本的搜索和新颖的基于图像的搜索功能。- 优化的高通量自动机器视觉技术，用于识别对图像分析和微结构科学至关重要的微结构特征。自动和客观的机器学习系统，发现微观结构之间的关系，以发现新的结构-性能和结构-性能的连接。微观结构科学的目标是了解微观结构特征和材料性能之间的联系。通过开发一个开放访问、自动和客观的机器学习系统来寻找微观结构图像之间的关系，该项目为发现可能激发更深入理解或预测能力的新联系奠定了基础。

The Division of Materials Research; the Civil, Mechanical, and Manufacturing Innovation Division; and the Division of Advanced Cyberinfrastructure contribute funds to this award. It supports research and education to collect, analyze, and compare data on materials from vast sources. All solid objects - from an airplane wing to a frying pan - have a microscopic structure that is usually not visible to the naked eye. This structure determines the properties of the material as a whole - whether it is strong or weak, for example. For the past century, materials scientists have studied these structures by using microscopes to take pictures (called micographs) of them. They then measure the important features seen in the microscopic images and relate those measurements to the properties of the material.Just as in personal photography, digital cameras have enabled materials scientists to take more pictures and do more with them than ever before. Moreover, older micrographs have been scanned in to digital archives. Materials scientists are now confronted with a set of images that is too large and too diverse to analyze manually. Fortunately, computer scientists have developed "machine vision" computer programs that identify similarities in large sets of images by in a sense mimicking how humans see objects. This project will gather micrographs from many sources into an open archive and use machine vision programs to search, sort, and classify them automatically without significant human intervention.By synthesizing microscopic image data at a previously impossible scale, this project creates a foundation for discovering new connections between microscopic structures and materials properties. The results will help improve current materials and even develop new ones. The data will be made available to the broader community.The Division of Materials Research; the Civil, Mechanical, and Manufacturing Innovation Division; and the Division of Advanced Cyberinfrastructure contribute funds to this award. It supports research and education to collect, analyze, and compare data on materials from vast sources. Over the past 100 years, materials scientists have made great progress in acquiring, analyzing, and comparing microstructural images. Much of this effort has been directed toward deep understanding of particular materials systems or classes of microstructures. When the catalog of possible microstructural features is known, imaging techniques can take advantage of well-defined feature characteristics to analyze microstructures with high precision. However, when the features of interest are not known a priori, these methods become intractable, inaccurate, or fail completely. Thus, typically, they are applied only to a pre-selected set of micrographs, chosen by a human expert. In contrast, the goal of this effort is to develop a general method to find useful relationships between micrographs without assumptions about what features may be present. Such an approach can leverage the explosion in digital data over the past two decades to survey the breadth of available microstructures efficiently and without significant human intervention.Capitalizing on recent advances in computer science, this project applies a subset of data science concepts - including data harvesting, machine vision, and machine learning - to advance the science of microstructure. The result will be a framework for finding connections between microstructural images within and across material systems, which will support outcomes ranging from computational tools to discovery science, including:- New open source tools for extracting micrographs and associated metadata from various digital archives, including the internet, PDF documents, and local storage media.- A comprehensive database of publicly available micrographs with traditional text-based search and novel image-based search functions. - Optimized, high throughput, automatic machine vision techniques to identify microstructural features that are salient to image analysis and microstructural science.- Automatic and objective machine learning systems that find relationships between microstructures in order to discover new structure-property and structure-performance connections.The goal of microstructural science is to understand the connection between microstructural features and materials properties. By developing an open-access, automatic, and objective machine learning system for finding relationships between microstructural images, this project creates a foundation for discovering new connections that may inspire deeper understanding or predictive capabilities.