Media Conversion for User-friendly Human-Machine Interface

点击翻译按钮获取中文摘要

In 1992, I proposed media conversion system from both text and voice to facial image. Lip motion is controlled by the analysis results of text and voice. At first, segment boundaries of vowel positions are extracted using spectrum and power transition. Strict vowel and consonant positions are decided by DP matching of segmentation result and text information. By this algorithm, accurate phoneme segmentation using text information is realized. This system can be applied to speech synthesis, too. The keyframes of mouth animation are located on the segment boundary decided by segmentation result. Standard mouth shapes and durations are decided by the text analysis. The parameters between keyframes are decided by 3-D Spline interpolation, so the motion becomes smooth and natural.In 1993, I constructed Scenario Making System to describe facial animation easily. Expression is realized by the modification of 3D wire frame model. Operator only has to decide the keyframes position by locating the Iconized facial expression on the time axis. Using this system, I realized proto-type e-mail interface system with facial expression. User of transmitter side only gives text, voice and expression positions. Only few parameters are transmitted and after a few delay, the facial motion image synchronized with the voice appears at the terminal of receiver. Ultra low bit rate image transmittion can be realized by our system. The facial expression recognition is our future subject.