SpeakQL: Towards Speech-driven Multimodal Querying of Structured Data
SpeakQL: Towards Speech-driven Multimodal Querying of Structured Data
复制标题
DOI：
10.1145/3318464.3389777
复制
发表时间：
2020-05
期刊：
影响因子：
--
通讯作者：
Vraj Shah;Side Li;Arun Kumar;L. Saul
中科院分区：
文献类型：
--
作者：
Vraj Shah;Side Li;Arun Kumar;L. Saul
语音驱动查询在智能手机、平板电脑甚至会话助手等新设备环境中变得越来越流行。然而，这种查询在很大程度上仅限于自然语言。类型化SQL仍然是复杂结构化查询的黄金标准，尽管它在许多环境中是痛苦的，它限制了用户何时以及如何使用他们的数据。在这项工作中，我们建议通过设计一个语音驱动的查询系统和结构化数据的接口来弥合这一差距，我们称之为SpeakQL。我们支持一个实际有用的常规SQL子集，并允许用户使用新颖的基于触摸/语音的人机循环校正机制在任何领域进行查询。自动语音识别（ASR）在转录中引入了无数形式的错误，给我们带来了技术挑战。我们利用对SQL属性、语法和查询数据库的观察来构建模块化体系结构。我们提供了第一个语音SQL查询数据集，以及为任意模式生成它们的通用方法。我们的实验表明，SpeakQL可以自动纠正ASR转录中的大部分错误。用户研究表明，与在平板设备上打字相比，SpeakQL可以帮助用户更快地指定SQL查询，平均速度提高2.7倍，最高可达6.7倍。与原始输入相比，SpeakQL还将用户指定查询的工作量平均减少了10倍，最多减少了60倍。
Speech-driven querying is becoming popular in new device environments such as smartphones, tablets, and even conversational assistants. However, such querying is largely restricted to natural language. Typed SQL remains the gold standard for sophisticated structured querying although it is painful in many environments, which restricts when and how users consume their data. In this work, we propose to bridge this gap by designing a speech-driven querying system and interface for structured data we call SpeakQL. We support a practically useful subset of regular SQL and allow users to query in any domain with novel touch/speech based human-in-the-loop correction mechanisms. Automatic speech recognition (ASR) introduces myriad forms of errors in transcriptions, presenting us with a technical challenge. We exploit our observations of SQL's properties, its grammar, and the queried database to build a modular architecture. We present the first dataset of spoken SQL queries and a generic approach to generate them for any arbitrary schema. Our experiments show that SpeakQL can automatically correct a large fraction of errors in ASR transcriptions. User studies show that SpeakQL can help users specify SQL queries significantly faster with a speedup of average 2.7x and up to 6.7x compared to typing on a tablet device. SpeakQL also reduces the user effort in specifying queries by a factor of average 10x and up to 60x compared to raw typing effort.