Multimodal image and spectral feature learning for efficient analysis of water-suspended particles.
Multimodal image and spectral feature learning for efficient analysis of water-suspended particles.
复制标题
DOI：
10.1364/oe.470878
复制
发表时间：
2023-01
期刊：
影响因子：
3.8
通讯作者：
Tomoko Takahashi;Zonghua Liu;T. Thevar;N. Burns;D. Lindsay;John Watson;Sumeet Mahajan;Satoru Yukioka;Shuhei Tanaka;Yukiko Nagai;B. Thornton
中科院分区：
文献类型：
--
作者：
Tomoko Takahashi;Zonghua Liu;T. Thevar;N. Burns;D. Lindsay;John Watson;Sumeet Mahajan;Satoru Yukioka;Shuhei Tanaka;Yukiko Nagai;B. Thornton
我们已经开发出一种方法，联合收割机形态和化学信息的准确识别不同的颗粒类型，使用光学测量技术，不需要样品制备。一个组合的全息成像和拉曼光谱设置用于收集数据，从六种不同类型的海洋颗粒悬浮在大量的海水。使用卷积和单层自动编码器对图像和光谱数据进行无监督特征学习。学习的功能相结合，其中我们证明，非线性降维的组合多模态功能可以实现高聚类宏F1得分为0.88，相比之下，最大值为0.61时，只有图像或光谱特征被使用。该方法可用于长期监测海洋中的颗粒物，而无需样品采集。此外，它可以应用于不同类型的传感器测量数据，而无需进行重大修改。
We have developed a method to combine morphological and chemical information for the accurate identification of different particle types using optical measurement techniques that require no sample preparation. A combined holographic imaging and Raman spectroscopy setup is used to gather data from six different types of marine particles suspended in a large volume of seawater. Unsupervised feature learning is performed on the images and the spectral data using convolutional and single-layer autoencoders. The learned features are combined, where we demonstrate that non-linear dimensional reduction of the combined multimodal features can achieve a high clustering macro F1 score of 0.88, compared to a maximum of 0.61 when only image or spectral features are used. The method can be applied to long-term monitoring of particles in the ocean without the need for sample collection. In addition, it can be applied to data from different types of sensor measurements without significant modifications.