Collaborative Research: FORABOT: An Autonomous and Accessible System for Sorting Foraminifera

有孔虫或“有孔虫”是生活在微小贝壳中的海洋原生动物，最常见的是由矿物方解石组成。有孔虫贝壳为古海洋学领域的许多工作提供了支柱，古海洋学是利用海底沉积物研究过去气候的学科。学生和实验室工作人员经常被要求在每项研究中从海洋沉积物中挑选数千个样本。在经历了陡峭的学习曲线后，挑选因此成为一项重复性和低回报的任务，使其非常适合使用机器学习和机器人技术进行自动化。该项目旨在开发一个对有孔虫进行自主分类的系统，科学界可以(在可用性和成本方面)使用该系统。该系统将与现有的现成显微镜兼容，它将利用微流体(或替代微操作)，以便于将样品从一个容器运送到其分类的容器，并将利用机器学习进行识别。研究人员开发的工具和数据集将提供给整个科学界，目标是将制造成本保持在3000美元以下。在研究人员之前的工作基础上，他们开发了一个视觉识别系统，使用不同光照方向下的图像对六种有孔虫进行识别，该项目将：(1)通过使用微流体(或另一种将开发的微操作技术)实现成像和分类过程的自动化；(2)通过整合多个成像实验室和云基础设施，扩大对35种浮游有孔虫的识别，这些浮游有孔虫被古海洋学家广泛使用。(3)扩展现有的机器学习技术，以实现稳健的关节形态表征和孔洞识别；以及(4)提供人类和自主性能之间的详细比较。为了训练所需的模型，研究人员将考虑一些技术，包括转移学习和数据增强。将利用从使用不同成像模式的其他有孔虫数据集学习的深层特征。在这个项目中获得的数据集将通过创建有孔虫的合成图像来扩大。为了确保稳健性，将采用惩罚条款，强制执行用于分割的拓扑持久性和用于识别的对图像扰动的稳健性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Foraminifera or "forams" are marine protozoa that live in microscopic shells, most often made of the mineral calcite. Foraminifera shells provide the backbone for much work in the field of paleoceanography, which is the study of past climates using seafloor sediments. Students and lab employees are often required to pick several thousands of specimens from ocean sediments for each study. After a steep learning curve, picking therefore becomes a repetitive and low-reward task, making it well-suited for automation using machine learning and robotics. The project aims to develop an autonomous sorting system for foraminifera, which is accessible (in terms of usability and cost) to the scientific community. This system will be compatible with existing off-the-shelf microscopes, it will make use of microfluidics (or alternatively micromanipulation) in order to facilitate the transport of the samples from a container to their sorted receptacles, and will utilize machine learning for recognition. The tools and datasets developed by the researchers will be made available to the entire scientific community, and the aim is to keep the fabrication cost under three thousand dollars.Building on prior work from the researchers, in which they developed a visual identification system for six species of forams using images under varying lighting directions, this project will: (1) automate the imaging and sorting process by using microfluidics (or alternatively another micromanipulation technique that will be developed); (2) scale up the recognition to thirty five species of planktonic foraminifera that are widely used by paleoceanographers by incorporating multiple laboratories for imaging, and a cloud infrastructure for crowd-sourcing of the data capture and labeling; (3) expand on the existing machine learning techniques to enable robust joint morphological characterization and recognition of forams; and (4) provide a detailed comparison between human and autonomous performance. In order to train the required models, the researchers will consider a number of techniques including transfer learning and data augmentation. Deep features learned from other datasets of forams using different imaging modalities will be exploited. The dataset obtained in this project will be augmented by creating synthetic images of forams. In order to ensure robustness, penalty terms that enforce topological persistence for segmentation and robustness to image perturbations for recognition will be employed.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.