Research on an effective method of video communication of sign language in consideration of cognitive characteristics

本研究的目的是利用信息通信技术为听障人士提供沟通支持。提供的结果是以下项目。(a)我们关注的是语言认知，语义加工过程的N400的基础上分析;事件相关电位测量进行了检查，以了解日本手语的理解机制。结果表明，由于N400在语义一致句中被观察到的手语和字符，在字符刺激中的潜力很大，并通过呈现方式显示出加工过程的差异。(b)讨论了手语数字编码电影的质量评价技术。这一结果保证了意见测试可以取代可懂度测试。进一步的研究表明，高帧率的运动图像比高精度的运动图像更清晰。另一个结果表明，屏幕尺寸的标志图片不 ...更多信息 不影响可理解性。(c)本文研究的是手语运动图像的低比特率数字编码。根据手语的特点，提出了一种适合手语的编码方法。结果，与现有方法相比，使用我们提出的方法对图像数据进行编码更可能以低比特率进行。此外，我们的方法在相同的参数条件下已被证实，它是获得一个更清晰的图像相比，现有的方法。(d)我们还研究了心理负荷的问题，即手语使用者在互动交流过程中所承受的心理压力。我们发现NASA-TLX能够有效地测量脑力负荷的程度。（5）在研究过程中，我们通过视频图像和电子数据收集了一个语料库，通过三维运动对手语进行语言学分析。电子数据通过BoT 3D引擎以手语动画的形式显示在Web上。我们的研究结果已经发表在IEICE交易之一和Human Interface Society之一等期刊上。我们感到有必要开展这项研究。在此研究成果的基础上，我们继续从认知科学的角度对手语进行会话分析。少

This research has aimed at the communication support using ICT for the hearing impaired people. Provided result is the following items.(a) We are concerned with language cognition, semantic processing process analysis based on N400 ; event related potential measurement was examined for the understanding the Japanese sign language comprehension mechanisms. As a result, because it is understood that N400 is observed in semantic congruity sentence by sign language and the character, and the potential is large in the character stimulation, and was shown the difference of the processing process by the presentation modality.(b) We discussed evaluation techniques for the quality of digitally encoded motion picture of sign language. This result guarantees that opinion test can be substituted for intelligibility test. Further investigation shows that motion pictures with high frame rate are more legible than those with high precision. Another result shows that screen size of sign picture doesn' … More t affect comprehensibility.(c) We are concerned with low-bit rate digital encoding of moving picture of sign language. A new encoding method suitable for sign language is presented based on characteristics of sign language. As a result, an encoding image data using our proposed method is more possible at a low bit rate in comparison with an existing method. Moreover, our method in the same parameter condition has been confirmed that it was obtained a clearer image in comparison with an existing method.(d) We have also worked on a problem of mental workload, which is mental stress that sign language users undergo during interactive communication. We have found that NASA-TLX is effective enough for measuring the degree of mental workload(e) In the research process, we collected a corpus by video image and electronic data by three-dimensional movement for linguistic analysis of sign language. The electronic data showed it on Web as sign language animation by the BoT3D engine.Our findings have been appeared in journals such as the one of the transactions on IEICE, and the one of Human Interface Society.We feel necessity to develop this research. On the basis of result of this research, we continue analysis work from the side of cognitive science through conversation analysis of sign language. Less