CC-Glasses: Color Communication Support for People with Color Vision Deficiency Using Augmented Reality and Deep Learning
CC-Glasses: Color Communication Support for People with Color Vision Deficiency Using Augmented Reality and Deep Learning
复制标题
DOI：
10.1145/3582700.3582707
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Zhenyang Zhu;Jiyi Li;Ying Tang;K. Go;M. Toyoura;K. Kashiwagi;I. Fujishiro;Xiaoyang Mao
中科院分区：
文献类型：
--
作者：
Zhenyang Zhu;Jiyi Li;Ying Tang;K. Go;M. Toyoura;K. Kashiwagi;I. Fujishiro;Xiaoyang Mao
患有色觉缺陷（CVD）的人在与他人交流时可能会遇到困难，因为他们无法识别由颜色名称所指代的目标物体。而现有的大多数CVD补偿研究都集中在颜色对比度损失的问题上。虽然有一些方法可以为用户提供颜色名称的线索，但这些技术要么需要培训，要么不能保护用户的隐私，即，有CVD的事实。在本文中，基于增强现实（AR）和深度学习技术，我们提出了一种新的系统，为受CVD影响的用户提供支持信息，以帮助进行色彩交流。采用最先进的参考分割（RS）深度神经网络（DNN）模型来生成支持信息，并使用AR眼镜进行信息呈现。为了进一步提高该系统的性能，一个新的数据集构造的基础上一个新的概念称为颜色对象名词对。评估实验结果表明，新的数据集可以提高所采用的DNN模型的性能，所提出的系统可以帮助受CVD影响的用户成功地识别目标对象的颜色名称。
People who suffer from color vision deficiency (CVD) can face difficulties when communicating with others by failing to identify target objects referred by their color names. While most existing studies on CVD compensation have focused on the issue of color contrast loss. Although there are approaches can provide clues of color name to users, these techniques either require training, or cannot protect users’ privacy, i.e., the fact of having CVD. In this paper, based on augmented reality (AR) and deep learning technologies, we propose a novel system to provide supporting information to users affected by CVD for color communication assistance. The state-of-the-art deep neural network (DNN) model for referring segmentation (RS) is adopted to generate supporting information, and AR glasses are utilized for information presentation. To improve the performance of the proposed system further, a new dataset is constructed based on a novel concept called Color–Object Noun Pair. The results of evaluation experiments show that the new dataset can enhance the performance of the adopted DNN model, and the proposed system can help users affected by CVD successfully identify target objects by their color names.