基于Mask R-CNN/PointRend的舌象形质智能诊断与可视化方法研究

基于图像的智能化诊断是舌诊技术现代化发展的主要方向。舌象形质特征量化是舌象分析难点，以往图像处理方法对舌体形质老嫩、齿痕等，舌苔形质润燥、腐腻等定性、定量分析效果不佳。深度学习被广泛应用于医学图像识别，Faster R-CNN等卷积神经网络在医学图像诊断应用中具有良好效果。本研究基于团队已有大样本舌象标准图像库，针对舌体形质（胖瘦、老嫩、齿痕、裂纹、点刺、瘀点、瘀斑），舌苔形质（厚薄、润燥、腐腻、剥落），运用神经网络Mask R-CNN与PointRend探索复杂多样性前提下的图像分类核心特征，研究建立舌象形质定性分类与定量分析相结合的智能化诊断新方法，运用模糊集成梯度方法（Blur IG）实现智能诊断的可视化，并运用上述方法以肺癌、糖尿病病证为对象，分析舌象诊断特征关联规则，实现舌象形质智能诊断及可视化。研究提出的智能诊断新方法将提升舌诊图像诊断准确性，对中医互联网医疗发展具有重要意义。

Image-based intelligent diagnosis is the main direction of modern development of tongue diagnosis technology. The quantification of tongue image features is a difficult point in tongue image analysis. The previous image processing methods are not effective in qualitative and quantitative analysis of tongue texture and shape, such as tooth marks, moist, dry, and greasy, etc. Deep learning has been widely used in medical image recognition and classification research, and convolutional neural networks such as Faster R-CNN have good performance in medical image diagnosis applications. This study is based on the team’s existing large-sample standard image library of tongue images, focusing on the shape and texture of the tongue body (swelling- thin, old-tender, tooth marks, cracks, pricks, petechiae, ecchymosis), the shape and texture of tongue coating (thick-thin, moist-dry, sticky-granular, rot, greasy, peeled), using Mask R-CNN and PointRend network models to explore the core features of image classification under the premise of complex diversity, research and establish a new intelligent diagnosis method combining qualitative classification and quantitative analysis of tongue image quality, using Blur Integrated Gradients method realizes the visualization of intelligent diagnosis, and uses the above methods to analyze the association rules of tongue diagnosis characteristics with lung cancer and diabetes syndromes, and realizes the intelligent diagnosis and visualization of tongue shape and texture. The new intelligent diagnosis method proposed by the research will improve the accuracy of tongue diagnosis, which is of great significance to the development of TCM Internet medical treatment.