MaDrIgAL: MultiDimensional Interaction management and Adaptive Learning

随着谷歌、Facebook、苹果和微软等科技巨头继续投资语音技术，全球语音识别市场预计到2017年将达到1330亿美元的价值（companiesandmarkets.com，2015）。特别是支持语音的交互系统，如苹果的Siri和微软的Cortana，开始显示出重大的经济影响，虚拟个人助理（VPA）市场估计将从2012年的3.52亿美元增长到2020年的30多亿美元。（Grand View Research，2014）。尽管这样的商业系统允许消费者使用他们的语音与他们的设备和服务进行交互，由于缺乏对话的自然性和VPA有限的社交智能，用户体验仍然有限。此外，这些用户界面的质量依赖于大型的、精心制作的规则集，这使得开发工作量很大，并且不能扩展到新的应用程序域。随着物联网和智能家居中语音控制的出现，人们对跨任务领域的自然会话界面的可扩展开发需求巨大。MaDrIgAL将开发一种全新的方法，通过利用自然语言会话的多维性质来构建交互式口语界面：除了执行基本任务或活动外，对话的参与者还同时处理沟通的其他几个方面，如提供和征求反馈意见以及遵守社会习俗。与牧歌中的歌声类似，每个维度的同步过程和谐地运作，产生多功能的自然话语。考虑以下示例中的两个备选响应S2a和S2b：U1：Hello，I would like to book a flight to London. S2a：Which date did you have in mind？好的，请问您哪天飞往伦敦？S2a只要求下一条信息来预订航班（一维），而S2b也给出了关于到达城市的反馈，允许用户纠正任何识别错误（多维）。我们的目标是开发一个原则性的多维建模和学习框架，涵盖广泛的不同现象，包括S2b中的隐式确认。这种多维方法不仅使我们能够构建支持与用户进行更自然和有效交互的系统，而且还可以通过学习可转移的会话技能（例如，选择域独立维度中的动作）。因此，我们将通过在家庭自动化场景中为多个应用领域构建交互式口语界面来展示我们的方法，允许用户与例如智能电视或加热控制系统进行交互。我们将与行业合作伙伴SemVox密切合作，探索这一场景。该项目将汇集统计机器学习方法的专业知识，以最先进的口语对话系统和自然语言生成，以及多维对话建模的语言学理论（特别是与学术合作伙伴Bunt教授合作）。MaDrIgAL将开发与健康技术和辅助生活相关的下一代交互技术，并解决物联网和大数据的通用用户界面问题。

As tech giants like Google, Facebook, Apple and Microsoft continue to invest in speech technology, the global voice recognition market is projected to reach a value of $133 billion by 2017 (companiesandmarkets.com, 2015). Speech-enabled interactive systems in particular, such as Apple's Siri and Microsoft's Cortana, are starting to show significant economic impact, with the virtual personal assistant (VPA) market estimated to grow from $352 million in 2012 to over $3 billion in 2020 (Grand View Research, 2014).Although such commercial systems allow consumers to use their voice in interacting with their devices and services, the user experience is still limited due to the lack of naturalness of the conversations and limited social intelligence of the VPA. Moreover, the quality of these user interfaces relies on large, carefully crafted rule sets, making development labour-intensive and not scalable to new application domains. With the emergence of the Internet of Things and voice control in the smart home, there is a huge demand for scalable development of natural conversational interfaces across task domains.MaDrIgAL will develop a radically new approach to building interactive spoken language interfaces by exploiting the multi-dimensional nature of natural language conversation: in addition to carrying out the underlying task or activity, participants in a dialogue simultaneously address several other aspects of communication, such as giving and eliciting feedback and adhering to social conventions. In analogy to the singing voices in a madrigal, simultaneous processes for each dimension operate in harmony to produce multifunctional, natural utterances. Consider the two alternative responses S2a and S2b in the following example:U1: Hello, I would like to book a flight to London.S2a: Which date did you have in mind?S2b: Ok, flying to London on what date?Whereas S2a only asks for the next piece of information to book the flight (uni-dimensional), S2b also gives feedback about the arrival city, allowing the user to correct any recognition errors (multi-dimensional). We aim to develop a principled multidimensional modelling and learning framework that covers a wide range of different phenomena, including the implicit confirmation in S2b.This multi-dimensional approach will not only allow us to build systems that support more natural and effective interactions with users, but also enables cost-effective development of such interfaces for a variety of domains by learning transferable conversational skills (e.g., selecting actions in domain independent dimensions). We will therefore demonstrate our approach by building interactive spoken language interfaces for multiple application domains in a home automation scenario, allowing users to interact with for example their Smart TV or heating control system. We will closely collaborate with the industrial partner SemVox to explore this scenario.The project will bring together expertise in statistical machine learning approaches to state-of-the-art spoken dialogue systems and natural language generation, as well as linguistic theories of multi-dimensional dialogue modelling (collaborating in particular with academic partner Prof. Bunt). MaDrIgAL will develop Next Generation Interaction Technologies relevant to Health Technology and Assisted Living, as well as tackle the question of a common user interface to the Internet of Things and Big Data.