Unmute: Opening Spoken Language Interaction to the Currently Unheard

每天，数以百万计的用户与机器交谈，无论是手机上的语音助手，还是家庭和工作场所的独立设备（如Alexa或谷歌Home）。通过在智能用户界面中通过语音媒介使用自然语言处理进行输入和输出，人们能够访问大量的内容和服务。这些系统依赖于几个因素：i)首先使用自动语音识别（ASR），它可以将语音信号转换为可用的语言符号；Ii)系统中是否有满足用户需要的知识和资源；iii)适合最终用户的上下文和能力的智能用户界面交互设计。虽然最先进的系统开始满足“传统”用户（即那些说一种广泛的口头和书面语言的人；具有相对较高的识字程度、接触数字互动和其他资源的人）的需求，但全球仍有数亿人被排除在外。矛盾的是，这些有资源限制的用户（如数字和文本素养低）可能是口语NLP系统进步的最大受益者，这些系统开辟了目前尚未实现的经济、社会和教育可能性。该项目解决了当今开放智能接口的方法的局限性，以实现当前数字“闻所未闻”。我们将应对三方面的挑战。首先，也是至关重要的一点，有必要探索高度创新的ASR技术，以应对文本资源有限甚至没有文本资源的语言。传统上，ASR系统依赖于大量的转录语音来开发和训练模型。我们的重点是那些数据很少的语言，以及那些没有既定书面形式的语言。为了使系统对我们的目标社区有用，它们当然必须提供这些用户使用的语言的相关内容；因此，第二个挑战是建立基础设施和用户参与，以提供生成此类内容的方法。在此过程中，我们希望生成一个蓝图和工具包，可供许多其他低资源或零资源语言社区使用。最后，我们将与之合作开发这些新方法的用户群体对“传统”用户有不同的看法，第三个挑战是在与内容和服务的智能界面交互时，将需求和价值呈现出来，以便底层算法、交互设备和样式是适当和有效的。我们团队之前的工作表明，当这些系统暴露在印度和南非城镇的非正式定居点时，在传统环境中部署的语音助手的假设就会失效。通过采用这种方法，我们期望创新和颠覆目前在智能语音和语言界面中使用的界面风格和交互设备，不仅满足当前被排除在外的用户的需求，还为世界其他地区提供新的可能性。这项工作将两个世界领先的团队聚集在一起进行新的合作——爱丁堡的CSTR，在语音技术创新方面有着悠久的历史；还有，斯旺西计算铸造厂的FIT实验室，十多年来，该实验室为新兴用户开创了互动创新，并在农村、城郊和城市发展中国家的社区中发展和倡导负责任的创新。这些团体与非政府组织、衍生机构和国际学术利益相关者的现有和新的合作伙伴一起加入，他们将帮助塑造工作并确保其具有直接和可持续的影响。

Everyday, millions of users talk to machines, be they voice assistants on their mobile phones or standalone devices - such as Alexa or Google Home - in the homes and workplaces. By using Natural Language Processing for input and output through the medium of speech in an intelligent user interface, people are able to access a plethora of content and services. These systems rely on a several factors: i) firstly the use of automatic speech recognition (ASR) that can convert the speech signal into usable language tokens; ii) the availability of knowledge and resources in the system to address the user need; and, iii) an intelligent user interface interaction design that fits the contexts and capabilities of the end user. While state-of-the-art systems are beginning to address the needs of "conventional" users (i.e. those who speak a widely spoken and written language; and who have relatively high degrees of literacy, exposure to digital interactions and other resources), there are many hundreds millions of people who are being excluded globally. Paradoxically, these users who have resource constraints (such as low digital and textual literacy) could be the ones to most benefit from advances in spoken NLP systems opening up economic, social and educational possibilities currently unmet. This project addresses the limitations of today's approaches to open up intelligent interfaces to the currently digitally 'unheard'. The challenges we will address are threefold. Firstly, and crucially, there is a need to explore highly innovative ASR techniques that can cope with languages that have limited or even no textual resources. Conventionally, ASR systems rely on vast amounts of transcribed speech to develop and train models. Our focus is on languages where there is little of this data and indeed on languages where there is no established written form. For the systems to be useful to the sorts of community we target, they have to of course make available relevant content that is in the language these users use; so the second challenge is establish infrastructures and user involvement to provide ways to generate such content. In doing so we hope to produce a blueprint and toolkit that can be used by many other low or zero-resource language communities. Finally, the user communities we will work with to develop these new approaches have a different perspective to "conventional" users and the third challenge is to surface the needs and values when interacting with an intelligent interface for content and services so that the underlying algorithms and the interaction devices and styles are appropriate and effective. Prior work by our team has shown that assumptions of what works in speech assistants that are deployed in conventional settings break down when these systems are exposed to groups in informal settlements in India and townships in South Africa. By taking this approach, we expect to innovate on and disrupt the interface styles and interaction devices currently used in intelligent speech and language interfaces, addressing the need of not just currently excluded users but offering up new possibilities for the rest of the world too. The work brings together two world leading groups in a new collaboration - Edinburgh's CSTR with its long track record on speech technology innovation; and, the FIT Lab at Swansea's computational foundry that has pioneered interaction innovation for and with emergent users for over a decade, developing and advocating responsible innovation with communities in rural, peri-urban and urban developing world contexts. These groups are joined by both existing and new collaborations with NGO, spin-out and international academic stakeholders who will help shape the work and ensure it has direct and sustainable impact.