Multi-Modal Spatial Querying

本计画探讨使用两种同时进行的沟通管道，图形与语音，来达成人与地理资讯系统（GIS）之间的成功互动。 我们的目标是构建一个多模态的空间查询语言，用户与地理数据库进行交互，通过绘制草图所需的配置，同时谈论的空间对象和空间关系绘制。 这项研究将增加我们对多模态空间交互的理解，并将导致在GIS中智能集成和处理这种多模态空间查询的改进策略。 这种互动的关键是利用互补或冗余的信息存在于图形和文字描述相同的空间场景。 一个多分辨率的空间关系模型被用来捕捉草图及其相应的口头描述的基本方面。 该模型强调拓扑属性，如包容性和邻域，并考虑度量属性，如距离和方向，在必要时进行细化。 该模型能够检索空间查询和地理数据库之间的相似匹配，而不仅仅是精确匹配。 这种多模式空间查询和空间相似性检索的新方法将使专家和新手用户能够进行更容易的空间搜索，最终使新的用户群体能够访问空间数据库。

This project investigates the use of two concurrent communication channels, graphics and speech, to achieve a successful interaction between a person and a geographic information system (GIS). The objective is to construct a multi-modal spatial query language in which users interact with a geographic database by drawing sketches of the desired configuration, while simultaneously talking about the spatial objects and the spatial relations drawn. This study will increase our understanding of multi-modal spatial interactions, and will lead to improved strategies for intelligent integration and processing of such multi-modal spatial queries in a GIS. The key to this interaction is the exploitation of complementary or redundant information present in both graphical and verbal descriptions of the same spatial scenes. A multiple-resolution model of spatial relations is used to capture the essential aspects of a sketch and its corresponding verbal description. The model stresses topological properties, such as containment and neighborhood, and considers metrical properties, such as distance and directions, as refinements where necessary. This model enables the retrieval of similar, not only exact, matches between a spatial query and a geographic database. Such new methods of multi-modal spatial querying and spatial similarity retrieval will empower experts as well as novice users to perform easier spatial searches, ultimately providing new user communities access to spatial databases.