AI platform for microscopy image restoration and virtual staining

显微图像恢复与虚拟染色AI平台 项目摘要： 荧光显微镜使生物医学科学的许多重大发现。尽管 光学、激光、探头、照相机和新技术的快速发展，空间 时间分辨率、曝光量、信噪比、穿透深度和探测光谱 来限制可能的实验类型。深度学习（DL）算法非常适合于 基于图像的问题，如SNR/超分辨率恢复和虚拟染色，具有很大的 使显微镜实验成为可能。以前不可能的实验可以实现， 实现高信噪比和/或时空分辨率而没有光漂白/光毒性; 同时观察许多图像通道，而不干扰本机进程等。 为基于显微镜的发现的量子飞跃铺平了道路， 功能和机制的障碍，并使新的诊断和治疗人类疾病。 然而，这些新方法还没有被广泛地转化为新的显微镜实验。的 延迟是由于若干实际障碍和挑战，如所需的专门知识、计算和信任。在 为了加速DL在显微镜中的采用，需要为生物学家量身定制的新型AI平台 用于训练、应用和验证DL模型和输出。 本项目旨在开发一个用于显微图像恢复和虚拟的人工智能平台。 AI for Restoring and Staining（AIRS）是一个新的染色平台。与我们的合作者哈里·史洛夫博士 （国家生物医学成像和生物工程研究所）我们已经成功地创建了DL模型， 针对各种成像条件的SNR恢复、超分辨率恢复和虚拟染色， 细胞器在我们的初步研究。AIRS平台旨在（1）提供一套全面的 用于显微镜修复和虚拟染色应用（包括SNR）的经过验证的DL模型 恢复，超分辨率恢复，空间反褶积，光谱解混，三维预测 2d图像，细胞器虚拟染色和分析;（2）为普通显微镜提供即插即用 实验;（3）提供半自动更新训练，以定制DL模型，以匹配高级 显微镜实验;（4）为新的DL模型培训提供用户友好的支持， 显微镜实验;（5）提供置信度评分，以评估DL模型的输出结果，（6） 提供DL模型，避免图像伪影（幻觉），并允许持续学习和进化; (7)并能够访问所需的计算基础设施和数据库连接。

AI Platform for Microscopy Image Restoration and Virtual Staining Project Summary: Fluorescence microscopy has enabled many major discoveries in biomedical sciences. Despite the rapid advancements in optics, lasers, probes, cameras and novel techniques, major factors such as spatial and temporal resolution, light exposure, signal-to-noise, depth of penetration and probe spectra continue to limit the types of experiments that are possible. Deep learning (DL) algorithms are well suited for image-based problems like SNR/super-resolution restoration and virtual staining, which have great enabling potentials for microscopy experiments. Previously impossible experiments could be realized such as achieving high signal-to-noise and/or spatial-temporal resolution without photobleaching/phototoxicity; simultaneously observing many image channels without interfering with native processes, etc. This could pave the way for a quantum leap forward in microscopy-based discoveries that elucidate biological functions and the mechanisms of disorders, and enable new diagnostics and therapies for human diseases. However, these new methods have not been widely translated to new microscopy experiments. The delay is due to several practical hurdles and challenges such as required expertise, computing and trust. In order to accelerate the adoption of DL in microscopy, novel AI platform tailored for biologists are needed for training, applying and validating DL models and outputs. The present project aims to develop an AI platform for microscopy image restoration and virtual staining called AI for Restoring and Staining (AIRS) platform. With our collaborator, Dr. Hari Shroff (National Institute of Biomedical Imaging and Bioengineering) we have successfully created DL models for SNR restoration, super-resolution restoration and virtual staining for a variety of imaging conditions and organelles in our preliminary studies. The AIRS platform intends to (1)provide a comprehensive suite of validated DL models for microscopy restoration and virtual staining applications including SNR restoration, super-resolution restoration, spatial deconvolution, spectral unmixing, prediction of 3d from 2d images, organelle virtual staining and analysis; (2)provide plug and play for common microscopy experiments; (3)provide semi-automatic update training to tailor DL models to match advanced microscopy experiments; (4)provide user friendly support for new DL model training for pioneering microscopy experiments; (5)provide confidence scores to assess the output results by a DL model, (6) provide DL models that avoid image artifact (hallucination) and allow continuous learning and evolution; (7) and be able to access the required computing infrastructure and database connection.