Frame rate as a QoS parameter and its influence on speech perception
Frame rate as a QoS parameter and its influence on speech perception
复制标题
帧速率作为 QoS 参数及其对语音感知的影响
DOI：
10.1007/s005300050099
复制
发表时间：
1998
影响因子：
3.9
通讯作者：
Kaoru Nakazono
中科院分区：
文献类型：
--
作者：
Kaoru Nakazono
摘要。在数据网络中保持多媒体业务的QoS是一个难题。本文主要研究视频帧率对语音感知的影响。当声音和图像不一致时（例如，声音“ba”与视觉“ga”相结合），受试者感知到不同的声音（例如“da”）。这种现象被称为麦格克效应。本文研究了视频帧率下降对语音感知的影响。研究表明，当帧率降低时，不同刺激下的正确听力得到改善，而一致刺激（声音和图像相同）下的正确听力则下降。此外，我们还研究了通过采样时间和嘴唇位置的同步来捕获嘴唇闭合的情况。在这种情况下，帧率对一致性刺激的误听影响不大。对于差异刺激，误听随着帧率的降低而减少。这些结果表明，低帧率导致的嘴唇僵硬运动不能为语音感知提供足够的唇部信息。此外，还研究了延迟图像对低帧率校正的影响。然而，由于与声音和图像同步相关的复合效应，结果并不像预期的那样确定。最后，我们对正常日语语音的静止图像进行了检查，并从辅助听力的角度确定了帧率的下限。
Abstract. The preservation of QoS for multimedia traffic through a data network is a difficult problem. We focus our attention on video frame rate and study its influence on speech perception.When sound and picture are discrepant (e.g., acoustic ‘ba’ combined with visual ‘ga’), subjects perceive a different sound (such as ‘da’). This phenomenon is known as the McGurk effect. In this paper, the influence of degraded video frame rate on speech perception is studied.It is shown that, when frame rate decreases, correct hearing is improved for discrepant stimuli and is degraded for congruent (voice and picture are the same) stimuli. Furthermore, we studied the case where lip closure was always captured by the synchronization of sampling time and lip position. In this case, frame rate has little effect on mishearing for congruent stimuli. For discrepant stimuli, mishearing is decreased with degraded frame rate. These results indicate that the stiff motion of lips resulting from low frame rate cannot give enough labial information for speech perception. In addition, the effect of delaying the picture to correct for low frame rate was studied. The results, however, were not as definitive as expected, because of compound effects related to the synchronization of sound and picture. Finally, we inspected the still pictures of normal Japanese speech and determined a lower limit of frame rate from the view point of assisting hearing.