Person Recognition by Multi-modal Information

1. 提出了一种基于语音和面部图像的双峰信息的人脸识别新方法。提出的方法利用隐马尔可夫模型（HMM）对口语单词的唇运动图像序列进行处理。我们研究了强度和位置归一化算法，对双峰数据库Tulips1（12人，4位英文单词）的识别精度达到95%左右。我们还提出了一种新的归一化算法，并表明它比之前提出的算法减少了计算量。我们还将该方法应用于比Tulips1大的双峰数据库M2VTS，该数据库由37人的10位单词组成。在此基础上，研究了基于隐马尔可夫模型的人脸图像归一化和唇部定位跟踪算法。我们只使用唇读信息进行了口语单词识别和说话人识别实验。实验结果表明，使用强度和位置归一化是非常有效的。我们获得了一个单词“0”的说话人识别率为81.0%，对37个人的10个数字的说话人识别率为74.2%。针对基于语音的说话人识别问题，提出了一种利用二阶全通扭曲函数相位特征的频谱参数估计方法。该方法可以改变任意区域内语音频谱的频率分辨率。利用该方法进行了基于区别特征提取（DFE）的说话人识别实验，该方法优化了频谱的扭曲函数，实现了说话人识别。分别用该方法和传统方法进行了说话人识别实验。实验结果表明，该方法比传统方法更有效，2kHz左右的频谱对说话人识别非常重要。

1. We proposed a new technique for person recognition using bimodal information comprising of speech and facial image. The proposed method utilizes a Hidden Markov Model(HMM) for a image sequence of lip movement of a spoken word. We studied intensity and location normalization algorithms and obtained a recognition accuracy of about 95% for a bimodal database Tulips1(12 persons, 4 digit word in English). We also proposed a new normalization algorithm and showed that it reduces the calculation amount less than the one we proposed before.2. We also applied the proposed method to a bimodal database M2VTS bigger than Tulips1, which consists of 10 digit words of 37 persons. Furthermore, some algorithms based on HMM for normalization of facial image and tracking of lip location were studied. We carried out spoken word recognition and speaker identification experiments using only lip reading information. The experimental results have shown that an use of intensity and location normalization is very effective. We obtained a speaker identification rate of 81.0% using one word "0" and a word recognition rate of 74.2% for 10 digits for 37 persons, respectively.3. For speaker identification using speech, we proposed a new spectral parameter estimation method which utilizes a phase characteristics of a second-order all-pass warping function. This method can change the frequency resolution of speech spectrum in an arbitrary region. Using the proposed method we carried out speaker recognition experiments based on a discriminative feature extraction (DFE), which optimizes the warping function of spectrum for speaker recognition. We carried out speaker identification experiments by the proposed method and conventional ones. Experimental results have shown that this method is more effective than conventional methods and spectrum around 2kHz is very important for speaker identification.