Teaching English to the healthy and speech training to the intellectually challenged : an integrative analysis of speech and body movements

为了将人体运动和语音关联起来，设计了一种语音同步的二维运动捕捉系统。这使得测量与连接语音相关的运动成为可能。在软件开发的第一阶段，检测到语音和视频图像之间的滞后时间为200ms到300ms，不在可接受的误差范围内。通过使用语音同步的二维运动捕获系统，检测到关于手部运动的观察及其与语音的关系的三种主要倾向：1)手运动的各种轨迹和速度模式在智力障碍者中显示出病理特征的模式，但在正常受试者中不显示；2)在没有任何语音清晰度的情况下，手部运动的向下和/或向上速度的故意增加影响了基频和音调模式。3)正常人的手部运动基频与最大运动速度之间存在最小的时差，而智力障碍者则不存在。

A speech synchronized two-dimensional motion capture system was designed in order to correlate body movements and speech. This has made the measurement of movement in relation to connected speech possible. At the firs stage of software development, 200 ms to 300 ms lag time between speech waves and video images was detected, which was not within an acceptable error range. Almost a year and a half was consumed to calibrate the lag within less than 33.3 ms.Three predominant tendencies are detected concerning observations on hand movements and their relation to speech by utilising a speech synchronized two-dimensional motion capture system 1) various patterns of trajectory ＆ velocity of hand movements display patterns of pathological features among the intellectually challenged but not among normal subjects 2) the intentional increase of downward and/or upward velocity of hand movements without any reference to the articulation of speech influences the fundamental frequency and the pitch pattern. 3) there exists minimum time lag between the onset of fundamental frequency and the maximum velocity of hand movements among the normal people but not the intellectually challenged.