Collaborative Research: FORABOT: An Autonomous and Accessible System for Sorting Foraminifera

有孔虫或“有孔虫”是生活在微观外壳中的海洋原生动物，通常由方解石矿物制成。有孔虫贝壳为古海洋学领域的许多工作提供了支柱，古海洋学是利用海底沉积物研究过去的气候。学生和实验室工作人员经常被要求为每项研究从海洋沉积物中挑选数千个标本。在陡峭的学习曲线之后，采摘因此成为一项重复和低回报的任务，使其非常适合使用机器学习和机器人技术进行自动化。该项目旨在开发一个有孔虫自动分类系统，科学界可以使用（就可用性和成本而言）。该系统将与现有的现成显微镜兼容，它将利用微流体技术（或替代性的微操作），以促进样品从容器运输到其分类容器，并将利用机器学习进行识别。研究人员开发的工具和数据集将提供给整个科学界，目的是将制造成本控制在3000美元以下。在研究人员之前的工作基础上，他们开发了一个视觉识别系统，使用不同照明方向下的图像识别六种有孔虫，该项目将：（1）通过使用微流体使成像和分选过程自动化（或者将开发的另一种显微操作技术）;（2）通过整合多个实验室进行成像，将识别范围扩大到35种古海洋学家广泛使用的浮游有孔虫，以及用于数据采集和标记的众包的云基础设施;（3）扩展现有的机器学习技术，以实现有孔虫的稳健联合形态表征和识别;以及（4）提供人类和自主性能之间的详细比较。为了训练所需的模型，研究人员将考虑一些技术，包括迁移学习和数据增强。将利用使用不同成像方式从其他有孔虫数据集学习的深层特征。在这个项目中获得的数据集将通过创建有孔虫的合成图像来增强。为了确保鲁棒性，将采用惩罚条款，强制执行分割的拓扑持久性和识别图像扰动的鲁棒性。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Foraminifera or "forams" are marine protozoa that live in microscopic shells, most often made of the mineral calcite. Foraminifera shells provide the backbone for much work in the field of paleoceanography, which is the study of past climates using seafloor sediments. Students and lab employees are often required to pick several thousands of specimens from ocean sediments for each study. After a steep learning curve, picking therefore becomes a repetitive and low-reward task, making it well-suited for automation using machine learning and robotics. The project aims to develop an autonomous sorting system for foraminifera, which is accessible (in terms of usability and cost) to the scientific community. This system will be compatible with existing off-the-shelf microscopes, it will make use of microfluidics (or alternatively micromanipulation) in order to facilitate the transport of the samples from a container to their sorted receptacles, and will utilize machine learning for recognition. The tools and datasets developed by the researchers will be made available to the entire scientific community, and the aim is to keep the fabrication cost under three thousand dollars.Building on prior work from the researchers, in which they developed a visual identification system for six species of forams using images under varying lighting directions, this project will: (1) automate the imaging and sorting process by using microfluidics (or alternatively another micromanipulation technique that will be developed); (2) scale up the recognition to thirty five species of planktonic foraminifera that are widely used by paleoceanographers by incorporating multiple laboratories for imaging, and a cloud infrastructure for crowd-sourcing of the data capture and labeling; (3) expand on the existing machine learning techniques to enable robust joint morphological characterization and recognition of forams; and (4) provide a detailed comparison between human and autonomous performance. In order to train the required models, the researchers will consider a number of techniques including transfer learning and data augmentation. Deep features learned from other datasets of forams using different imaging modalities will be exploited. The dataset obtained in this project will be augmented by creating synthetic images of forams. In order to ensure robustness, penalty terms that enforce topological persistence for segmentation and robustness to image perturbations for recognition will be employed.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.