CyberSEES: Type 1: Collaborative Research: High-Performance Image Classification and Search Supporting Large-Scale Seafloor Biodiversity and Habitat Surveys

Cyber​​SEES：类型 1：协作研究：支持大规模海底生物多样性和栖息地调查的高性能图像分类和搜索

• 批准号: 1539368
• 负责人: Malcolm Stokes
• 金额: $ 4.82万
• 依托单位: University of California-San Diego Scripps Inst of Oceanography
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1539368&HistoricalAwards=false
• 关键词: CyberSEES; Type; Collaborative; Research; Performance

Seafloor ecosystems are complex environments populated by a great diversity of organisms. Unfortunately, these ecosystems are increasingly threatened by direct and indirect human activities, including changes in land-use practices, coastal runoff, energy and mineral extraction, and fishing pressure. Developing effective sustainability policies to deal with these ecosystem threats requires that we first understand seafloor communities as they are today, and then track how they change over time as human activities shift and sustainability policies are modified. Recent advances in high-resolution underwater imaging offer new ways to do this. Survey ships can zigzag back and forth above a threatened region, towing a submerged camera system that repeatedly snaps pictures of the seafloor. This produces an enormous and valuable image set that captures the current state of a seafloor ecosystem. Surveys like this have been done for many threatened regions, and more are in progress. Substantial challenges remain to process these image sets. A useful characterization of a seafloor habitat requires knowing which specific types of corals, sponges, starfish, and so forth are present, how many there are, and how they are distributed throughout a region. But with each survey image set containing hundreds of thousands or millions of images, manual processing is impractical. Instead of an army of experts examining these images, computer software can scan each image and automatically recognize the color and texture of different seafloor species. Experimental classification software like this exists today in research laboratories, but the software is slow. To be useful for huge image sets, this software must be revised and optimized to run on the latest high-performance supercomputers. This is the focus of the project, which will yield new optimized classification software that can quickly sweep through enormous image sets to classify and count the species present and provide essential information about the health and biodiversity of threatened seafloor ecosystems, or any other ecosystem with a suitable image set. Then, when surveys are repeated for the same region every few years, this processing can reveal important trends that document the health of a region and the impact of new sustainability policies that aim to mitigate continuing threats to these communities.This project leverages prior work prototyping seafloor image classification algorithms. These algorithms divide survey images into small tiles, then characterize each tile with a high-dimensionality feature vector that includes metrics on the colors and textures present in the tile, along with water temperature, salinity, and depth data collected by the survey apparatus at the moment the image was captured. Colors in the feature vector are chosen based upon a quantized hue histogram of the tile, while textures are characterized by luminance Discrete-Cosine-Transform (DCT) coefficients. A tile's feature vector is then compared against stored feature vectors for known species within a large classification library. A probability-based selection using a set of nearest-neighbor matches from the library yields a best guess for the species depicted in the image tile. This process is repeated tile after tile, image after image throughout an image survey. Classification performance is strongly a function of the classification library size and the dimensionality of feature vectors used for image tiles and library entries. This project's approach to improve classification performance uses a customized k-d-tree search data structure for the classification library, along with domain knowledge to guide and tune the classification process. The project begins with new methods to cull the tree, prior to classification, by using broad survey characteristics, such as the geographic region covered, water temperature and salinity, the sea bottom type from acoustic data, and so forth. Additional techniques optimize the construction and matching of feature vectors by using survey and library metrics to cull and weigh vector components (such as contextual color gamut and texture detail reduction, principal component analysis to combine and weigh features), reduce the nearest-neighbor set size by using k-d tree metrics on library diversity, restructure the k-d tree to improve common case search and cache performance, and parallelize the search for efficient classification across multiple threads, cores, processors, and nodes in a large compute cluster. Together these new methods are expected to substantially increase classification performance and enable efficient processing for the latest large survey image sets.

海底生态系统是一个复杂的环境，有多种多样的生物。不幸的是，这些生态系统日益受到直接和间接的人类活动的威胁，包括土地使用做法的变化、沿海径流、能源和矿物开采以及捕鱼压力。制定有效的可持续发展政策来应对这些生态系统威胁，需要我们首先了解海底群落的现状，然后跟踪它们随着人类活动的变化和可持续发展政策的修改而发生的变化。高分辨率水下成像的最新进展提供了新的方法来做到这一点。勘测船可以在受威胁的区域来回曲折，拖着一个水下摄像系统，反复拍摄海底的照片。这产生了一个巨大的和有价值的图像集，捕捉海底生态系统的当前状态。在许多受威胁的地区已经进行了这样的调查，更多的调查正在进行中。处理这些图像集仍然存在重大挑战。海底栖息地的一个有用的特征需要知道存在哪些特定类型的珊瑚，海绵，海星等，有多少，以及它们如何在整个区域分布。但是，由于每个调查图像集包含数十万或数百万图像，手动处理是不切实际的。计算机软件可以扫描每一张图像，并自动识别不同海底物种的颜色和纹理，而不是让大批专家来检查这些图像。像这样的实验性分类软件今天在研究实验室中存在，但软件速度很慢。为了对大型图像集有用，该软件必须进行修改和优化，以便在最新的高性能超级计算机上运行。这是该项目的重点，该项目将产生新的优化分类软件，可以快速扫描大量图像集，对现有物种进行分类和计数，并提供有关受威胁海底生态系统或任何其他生态系统的健康和生物多样性的重要信息。然后，当每隔几年对同一地区进行重复调查时，这种处理可以揭示重要的趋势，这些趋势记录了一个地区的健康状况以及旨在减轻对这些社区的持续威胁的新的可持续发展政策的影响。这些算法将勘测图像划分为小块，然后用高维特征向量来表征每个块，该特征向量包括块中存在的颜色和纹理的指标，以及勘测设备在图像捕获时收集的水温、盐度和深度数据，沿着。特征向量中的颜色是基于瓦片的量化色调直方图来选择的，而纹理的特征在于亮度离散余弦变换（DCT）系数。然后将图块的特征向量与大型分类库中已知物种的存储特征向量进行比较。使用来自库的一组最近邻匹配的基于概率的选择产生图像图块中描绘的物种的最佳猜测。在整个图像调查中，该过程在一个图像之后的图像之后重复。分类性能是分类库大小和用于图像块和库条目的特征向量的维数的函数。这个项目的方法，以提高分类性能使用定制的k-d-tree搜索数据结构的分类库，沿着与领域知识，以指导和调整分类过程。该项目首先采用新的方法，在分类之前挑选树木，利用广泛的调查特征，如覆盖的地理区域、水温和盐度、声学数据中的海底类型等。其他技术通过使用调查和库度量来挑选和加权矢量分量来优化特征矢量的构造和匹配（诸如上下文色域和纹理细节减少、用于联合收割机和加权特征的主成分分析），通过使用关于库多样性的k-D树度量来减少最近邻集合大小，重构k-D树以改进公共情况搜索和高速缓存性能，并且并行化搜索以跨大型计算集群中的多个线程、核心、处理器和节点进行有效分类。这些新方法有望大大提高分类性能，并能够有效处理最新的大型调查图像集。