Research on a Prototype System to Integrate and Understand Plural Information

1. 多元信息处理系统的构建我们在已经完成的基础测试的基础上，在工作站（SUN-4）上构建了一个既能处理图像信息又能处理语言信息的系统。本系统由两部分组成；语言处理部分和视觉处理部分。我们对每个部分和综合系统的结果如下所述。(a)语言处理部分我们安排了自然语言的表示；指的是一个形状和关于它的情节。构建了语言处理系统，对自然语言句子进行分析和语义提取；基于我们的工作，意义表示方法适合处理图像和语言信息之间的联系。(b)图像处理部分完成了将视觉信息和语言信息联系起来需要提取的特征的安排。根据对象的模型，构建了提取所需特征的系统；三维单数物体。(c)集成系统集成上述系统，构建了具有自然语言接口的三维对象数据库系统。该数据库保存了三维单一物体及其片段的图像数据。在该系统中，图像处理部分在配准数据时提取物体形状等特征，然后按照自然语言给出的情节进行配准。通过以上方法，我们实现了同时处理图像和语言信息的系统。对今后存在的问题进行总结和整理。在上述研究知识的基础上，我们总结了这一步对多元信息的总体理解，并从语言处理、图像处理和集成系统三个方面安排了需要解决的问题。

1. Construction of Plural Information Processing SystemWe have constructed a system on the workstation (SUN-4), which is able to handle both image and linguistic information, based on basic examination which we also have done. This system consists of two parts ; linguistic processing part and visual processing part. Our results of each part and integrated system are described below.(a) Linguistic Processing PartWe have arranged representations of natural language ; which refer to a shape and episodes about that. And we constructed the language processing system which analyzes natural language sentences and extracts meanings of those ; based on our work of meaning representation methods fit to handling connection of both image and linguistic information.(b) Image Processing PartThe arrangements of features that we should extract for the connection of visual and linguistic information has done. And we constructed the system which extracts features needed, refering to the model of the object ; 3D singular object.(c) Integrated SystemIntegrating the systems above, we constructed the 3D object data base system which owned natural language interface. This data base holds image data of 3D singular objects and episodes of those. In this system, image processing part extracts features such as that of shape of object at registratering data, then the data is registrated in the same way of the episodes given by natural language does. Through this methods above, we have realized the system which is able to handle both image and linguistic information in the same way.2. Summary and Arrangement of problems in the future.Under the knowledge got through our research described above, we summarized overall comprehension of plural information at this step, and arranged the problems we should solve from the view of both linguistic processing, image processing, and integrated system sides.