Nonverbal Communication Cue Recognition: A Pathway to More Accessible Communication
Nonverbal Communication Cue Recognition: A Pathway to More Accessible Communication
复制标题
DOI：
10.1109/cvprw59228.2023.00600
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Z. Shafique;Haiyan Wang;Yingli Tian
中科院分区：
文献类型：
--
作者：
Z. Shafique;Haiyan Wang;Yingli Tian
非语言交流，如肢体语言、面部表情和手势，对人类交流至关重要，因为它比口头语言传达了更多关于情感和态度的信息。然而，盲人或低视力（BLV）的人可能无法使用这种交流方式，导致对话中的不对称。为BLV社区开发识别非语言交际线索（NVCs）的系统将加强双方的沟通和理解。本文的重点是开发一个多模态计算机视觉系统来识别和检测NVCs。为了实现我们的目标，我们正在收集一个非语言交流线索的数据集。在这里，我们提出了一个识别nvc的基线模型，并在Aff-Wild2数据集上给出了初步结果。我们的基线模型在af - wild2验证集上实现了68%的准确率和64%的F1-Score，使其与以前的最先进的结果相媲美。此外，我们还讨论了与NVC识别相关的各种挑战以及我们当前工作的局限性。
Nonverbal communication, such as body language, facial expressions, and hand gestures, is crucial to human communication as it conveys more information about emotions and attitudes than spoken words. However, individuals who are blind or have low-vision (BLV) may not have access to this method of communication, leading to asymmetry in conversations. Developing systems to recognize nonverbal communication cues (NVCs) for the BLV community would enhance communication and understanding for both parties. This paper focuses on developing a multimodal computer vision system to recognize and detect NVCs. To accomplish our objective, we are collecting a dataset focused on nonverbal communication cues. Here, we propose a baseline model for recognizing NVCs and present initial results on the Aff-Wild2 dataset. Our baseline model achieved an accuracy of 68% and a F1-Score of 64% on the Aff-Wild2 validation set, making it comparable with previous state of the art results. Furthermore, we discuss the various challenges associated with NVC recognition as well as the limitations of our current work.