A research for automatic structure extraction and information retrieval from video data using speech and voice

本研究的第一个主题是实现视频的自动分割。为此，我们开发了提取视频信息局部和全局特征以及对视频数据集进行分类的方法。第二个主题是利用分割部分之间的局部和全局相似性和不相似性自动提取分割视频数据集的结构。本研究首先对视频数据集中的局部声学和图像相似性进行了分析，并提取了在音乐片段中重复出现的相似部分。然后，所开发的方法分别使用局部特征和全局特征来区分语音部分和音乐部分。我们证实了所开发的方法对真实的视频数据集效果良好。本研究还对上述方法分割的语音视频片段进行了文本检索和语音查询。在语音检索方面，提出了对查询进行任意词处理的新方法和对多个子词模型进行集成的方法，实验结果表明，该方法比以前的方法具有更好的性能。这些结果在许多国内和国际会议上进行了报道。在未来，我们将开发实际使用的方法。

The first theme of this research is to realize an automatic video segmentation. For this purpose,we developed the methods for extracting a local and global feature of video information and for classifying video data sets. The second theme is automatic extraction of the structure for segmented video data sets using local and global similarity and dissimilarity between the segmented sections. This research first conducted the analysis of local acoustic and image similarity in video data sets,and extracted similar partial sections that are repeated in a music piece. The developed method then discriminated speech sections and music sections respectively using a local feature and a global feature.We confirmed the developed methods worked well for real video data sets. The research also conducted speech retrieval by a text and speech query for speech video sections segmented by the above method. For speech retrieval,new technique of dealing with any words for a query and the integration method for plural subword models were proposed,and the experimental results demonstrated the method showed better performance compared to former methods. These results were reported at the many domestic and international conferences. In the future,we are going to develop the methods for the actual use.