Multi-Modal Spatial Querying

该项目研究如何使用图形和语音这两个并发通信渠道来实现人与地理信息系统(GIS)之间的成功交互。其目的是构建一种多模式空间查询语言，其中用户通过绘制所需配置的草图来与地理数据库交互，同时谈论空间对象和所绘制的空间关系。这项研究将增加我们对多模式空间交互的理解，并将导致改进的策略，在地理信息系统中智能地集成和处理这种多模式空间查询。这种相互作用的关键是利用相同空间场景的图形和语言描述中存在的互补或冗余信息。空间关系的多分辨率模型被用来捕捉草图及其相应的语言描述的本质方面。该模型强调拓扑属性，如包容和邻域，并考虑度量属性，如距离和方向，作为必要的细化。该模型能够检索空间查询和地理数据库之间的相似匹配，而不仅仅是精确匹配。这种多模式空间查询和空间相似性检索的新方法将使专家和新手用户能够更容易地进行空间搜索，最终为新的用户群体提供对空间数据库的访问。

This project investigates the use of two concurrent communication channels, graphics and speech, to achieve a successful interaction between a person and a geographic information system (GIS). The objective is to construct a multi-modal spatial query language in which users interact with a geographic database by drawing sketches of the desired configuration, while simultaneously talking about the spatial objects and the spatial relations drawn. This study will increase our understanding of multi-modal spatial interactions, and will lead to improved strategies for intelligent integration and processing of such multi-modal spatial queries in a GIS. The key to this interaction is the exploitation of complementary or redundant information present in both graphical and verbal descriptions of the same spatial scenes. A multiple-resolution model of spatial relations is used to capture the essential aspects of a sketch and its corresponding verbal description. The model stresses topological properties, such as containment and neighborhood, and considers metrical properties, such as distance and directions, as refinements where necessary. This model enables the retrieval of similar, not only exact, matches between a spatial query and a geographic database. Such new methods of multi-modal spatial querying and spatial similarity retrieval will empower experts as well as novice users to perform easier spatial searches, ultimately providing new user communities access to spatial databases.