BIGDATA: Collaborative Research: IA: Quantifying Plankton Diversity with Taxonomy and Attribute Based Classifiers of Underwater Microscope Images

浮游生物在全球生态系统中发挥着至关重要的作用，形成海洋食物网的基础，连接大气与深海，并调节无数生态和气候的重要过程。然而，尽管浮游生物很重要，但评估浮游生物丰度和分布的技术仍然有限。单个物种丰度的变化尤其难以解决；这包括对许多沿海系统产生深远经济、社会和生态系统影响的有害藻华。网和瓶子等传统工具可能会在采样过程中破坏脆弱的生物体。另一方面，水下显微镜可以在自然环境中不受干扰地观察生物体。新型水下显微镜每天都会生成数千张单个浮游生物的高分辨率图像。在将这些图像用于科学分析之前，必须对成像的生物体进行识别和分类。然而，此类显微镜生成的大量图像导致了严重的瓶颈：图像的识别和分类需要个人花费极长的时间才能完成。幸运的是，计算机视觉科学的进步在准确执行此类分类任务方面显示出了巨大的希望。该奖项的主要目标是探索和开发浮游生物图像分类的计算机视觉方法。由仪器专家、海洋生态学家和计算机科学家组成的团队（包括两名研究生和一名博士后）将制定、实施和测试方法，以推进高效、准确的自动浮游生物图像分类的目标。该奖项所取得的进展将改进计算机科学中的分类算法，并为浮游生物生态学提供大量新数据流。浮游生物构成海洋食物网的基础，将大气与深海连接起来，并调节全球生物地球化学循环。浮游生物通常通过批量测量或通过对单个类群进行手动计数来研究。斯克里普斯浮游生物相机系统 (SPCS) 等新型水下显微镜系统每天会生成数万张浮游生物个体图像。然而，如果没有对图像的准确注释，潜在的科学意义是有限的。该项目将探索使用多层、深度卷积神经网络（CNN）作为自动计算机识别方法；这些技术有望对全球各个研究小组收集的近万亿水下显微镜图像进行分类。图像的主要来源将是斯克里普斯研究所的一对已经运行了两年的显微镜。海洋学码头，产生 2 亿个兴趣区域。该项目将使用一种新颖的方法建立一个大型训练集数据库：一种台式成像系统，能够快速生成数千张带注释的图像，显​​示与现场相同的所有方向和配置的生物体。基于这些自动收集的训练集以及专家对原位图像的手动注释，深层（多层）CNN 将嵌入分类和属性约束，并将用于对成像的生物体进行分类。如果成功的话，这个庞大的、不断增长的分类数据集将能够在数小时到数十年的时间尺度上对浮游生态系统的动态进行前所未有的、变革性的、特定于分类单元的探索。

Plankton play an essential role in the global ecosystem, forming the base of marine food webs, linking the atmosphere to the deep ocean, and regulating a myriad of ecologically and climatologically important processes. Despite their importance, however, the technology to assess abundances and distributions of plankton has been limited. Changes in abundances of individual species are particularly poorly resolved; this includes the harmful algal blooms that have profound economic, societal, and ecosystem effects in many coastal systems. Traditional tools such as nets and bottles can destroy fragile organisms during sampling. Underwater microscopes, on the other hand, allow observation of the organisms undisturbed, and in their natural setting. New underwater microscopes are generating many thousands of high-resolution images of individual plankton each day. Before these images can be used for scientific analyses, the imaged organisms must be identified and classified. However, the vast number of images generated by such microscopes has led to a serious bottleneck: identification and classification of the images takes an impossibly long time for individuals to accomplish. Fortunately, advances in computer vision science have shown great promise in accurately performing such classification tasks. The main goal of this award is to explore and develop computer vision approaches for plankton image classification. A team of instrumentation specialists, an ocean ecologist, and a computer scientist, including two graduate students and one post doctoral student, will formulate, implement, and test methods to advance the goal of efficient and accurate automated plankton image classification. The advances made in this award will enable both improved classification algorithms in computer science, and vast new data streams for plankton ecology.Plankton form the base of marine food webs, link the atmosphere to the deep ocean, and regulate global biogeochemical cycles. Plankton are often studied either through bulk measures, or by manual enumeration of individual taxa. Novel underwater microscope systems such as the Scripps Plankton Camera System (SPCS) are generating tens of thousands of images of individual plankton daily. However, without accurate annotation of the images, the potential science is limited. This project will explore the use of many-layer, deep Convolutional Neural Nets (CNN) as automated computer recognition methods; these techniques hold promise for classifying the nearly one trillion underwater microscope images that have been collected by a variety of research groups around the globe. The primary source of images will be a pair of microscopes that have been operating for 2 years from the Scripps Inst. of Oceanography's pier, yielding 200 million regions of interest. The project will build a large data base of training sets using a novel approach: a bench-top imaging system that is capable of rapidly producing thousands of annotated images showing organisms in all orientations and configurations identical to that in the field. Based on these automatically collected training sets, and hand annotation of in situ images from experts, a deep (many layer) CNN will embed taxonomic and attribute constraints, and will be used to classify the organisms imaged. With success, this massive, growing, taxonomically classified dataset will enable unprecedented, transformative, taxon-specific explorations of the dynamics of the planktonic ecosystem on time scales from hours to decades.