CyberSEES: Type 1: Collaborative Research: High-Performance Image Classification and Search Supporting Large-Scale Seafloor Biodiversity and Habitat Surveys

海底生态系统是由多种生物组成的复杂环境。不幸的是，这些生态系统日益受到直接和间接人类活动的威胁，包括土地利用方式、沿海径流、能源和矿物开采以及捕鱼压力的变化。制定有效的可持续性政策来应对这些生态系统威胁，要求我们首先了解目前的海底群落，然后随着人类活动的变化和可持续性政策的修改，跟踪它们如何随着时间的推移而变化。高分辨率水下成像的最新进展为实现这一目标提供了新的方法。勘测船可以在受到威胁的区域上空来回曲折前进，拖着一个水下相机系统，反复拍摄海底照片。这产生了一组巨大而有价值的图像，捕捉到了海底生态系统的当前状态。类似的调查已经在许多受威胁的地区进行，更多的调查正在进行中。处理这些图像集仍然面临着巨大的挑战。对海底栖息地的有用描述要求了解存在哪些特定类型的珊瑚、海绵、海星等，数量有多少，以及它们在整个地区的分布情况。但由于每个测量图像集包含数十万或数百万张图像，手动处理是不切实际的。计算机软件可以扫描每一张图像，自动识别不同海底物种的颜色和纹理，而不是一大群专家对这些图像进行检查。像这样的实验性分类软件今天在研究实验室中存在，但软件速度很慢。为了对巨大的图像集有用，该软件必须进行修改和优化，以便在最新的高性能超级计算机上运行。这是该项目的重点，该项目将产生新的优化分类软件，该软件可以快速扫描巨大的图像集，对存在的物种进行分类和计数，并提供有关受威胁海底生态系统或具有合适图像集的任何其他生态系统的健康和生物多样性的基本信息。然后，当每隔几年对同一区域重复进行调查时，这一过程可以揭示重要的趋势，这些趋势记录了一个区域的健康状况以及旨在减轻这些社区持续威胁的新的可持续性政策的影响。该项目利用了先前的海底图像分类算法原型工作。这些算法将测量图像分成小块，然后用高维特征向量表征每个块，该高维特征向量包括关于瓷砖中存在的颜色和纹理的度量，以及在捕获图像的时刻由测量设备收集的水温、盐度和深度数据。特征向量中的颜色是基于块的量化色调直方图来选择的，而纹理是通过亮度离散余弦变换(DCT)系数来表征的。然后，将瓷砖的特征向量与大型分类库中存储的已知物种的特征向量进行比较。使用来自图书馆的一组最近邻匹配的基于概率的选择产生了对图像瓷砖中描述的物种的最佳猜测。这个过程在整个图像调查中一块一块地重复，一幅又一幅图像。分类性能强烈地依赖于分类库的大小以及用于图像块和库条目的特征向量的维度。该项目改进分类性能的方法使用针对分类库的定制k-d树搜索数据结构，以及指导和调整分类过程的领域知识。该项目以新的方法开始，在分类之前，通过使用广泛的调查特征，如覆盖的地理区域、水温和盐度、根据声学数据确定海底类型等，来剔除树木。其他技术通过使用调查和库度量来筛选和加权向量分量(例如上下文色域和纹理细节减少、主成分分析来组合和加权特征)来优化特征向量的构造和匹配，通过使用关于库多样性的k-d树度量来减小最近邻集合大小，重构k-d树以改进常见情况搜索和高速缓存性能，以及并行搜索以跨大型计算集群中的多个线程、核、处理器和节点进行有效分类。这些新方法预计将大大提高分类性能，并实现对最新的大型调查图像集的有效处理。

Seafloor ecosystems are complex environments populated by a great diversity of organisms. Unfortunately, these ecosystems are increasingly threatened by direct and indirect human activities, including changes in land-use practices, coastal runoff, energy and mineral extraction, and fishing pressure. Developing effective sustainability policies to deal with these ecosystem threats requires that we first understand seafloor communities as they are today, and then track how they change over time as human activities shift and sustainability policies are modified. Recent advances in high-resolution underwater imaging offer new ways to do this. Survey ships can zigzag back and forth above a threatened region, towing a submerged camera system that repeatedly snaps pictures of the seafloor. This produces an enormous and valuable image set that captures the current state of a seafloor ecosystem. Surveys like this have been done for many threatened regions, and more are in progress. Substantial challenges remain to process these image sets. A useful characterization of a seafloor habitat requires knowing which specific types of corals, sponges, starfish, and so forth are present, how many there are, and how they are distributed throughout a region. But with each survey image set containing hundreds of thousands or millions of images, manual processing is impractical. Instead of an army of experts examining these images, computer software can scan each image and automatically recognize the color and texture of different seafloor species. Experimental classification software like this exists today in research laboratories, but the software is slow. To be useful for huge image sets, this software must be revised and optimized to run on the latest high-performance supercomputers. This is the focus of the project, which will yield new optimized classification software that can quickly sweep through enormous image sets to classify and count the species present and provide essential information about the health and biodiversity of threatened seafloor ecosystems, or any other ecosystem with a suitable image set. Then, when surveys are repeated for the same region every few years, this processing can reveal important trends that document the health of a region and the impact of new sustainability policies that aim to mitigate continuing threats to these communities.This project leverages prior work prototyping seafloor image classification algorithms. These algorithms divide survey images into small tiles, then characterize each tile with a high-dimensionality feature vector that includes metrics on the colors and textures present in the tile, along with water temperature, salinity, and depth data collected by the survey apparatus at the moment the image was captured. Colors in the feature vector are chosen based upon a quantized hue histogram of the tile, while textures are characterized by luminance Discrete-Cosine-Transform (DCT) coefficients. A tile's feature vector is then compared against stored feature vectors for known species within a large classification library. A probability-based selection using a set of nearest-neighbor matches from the library yields a best guess for the species depicted in the image tile. This process is repeated tile after tile, image after image throughout an image survey. Classification performance is strongly a function of the classification library size and the dimensionality of feature vectors used for image tiles and library entries. This project's approach to improve classification performance uses a customized k-d-tree search data structure for the classification library, along with domain knowledge to guide and tune the classification process. The project begins with new methods to cull the tree, prior to classification, by using broad survey characteristics, such as the geographic region covered, water temperature and salinity, the sea bottom type from acoustic data, and so forth. Additional techniques optimize the construction and matching of feature vectors by using survey and library metrics to cull and weigh vector components (such as contextual color gamut and texture detail reduction, principal component analysis to combine and weigh features), reduce the nearest-neighbor set size by using k-d tree metrics on library diversity, restructure the k-d tree to improve common case search and cache performance, and parallelize the search for efficient classification across multiple threads, cores, processors, and nodes in a large compute cluster. Together these new methods are expected to substantially increase classification performance and enable efficient processing for the latest large survey image sets.