Unmute: Opening Spoken Language Interaction to the Currently Unheard

每天都有数百万用户与机器对话，无论是移动的手机上的语音助手还是家庭和工作场所中的独立设备（如Alexa或Google Home）。通过在智能用户界面中使用自然语言处理通过语音媒介进行输入和输出，人们能够访问大量内容和服务。这些系统依赖于几个因素：i）首先，使用自动语音识别（ASR），可以将语音信号转换为可用的语言标记; ii）系统中的知识和资源的可用性，以满足用户需求;以及iii）智能用户界面交互设计，适合最终用户的上下文和能力。虽然最先进的系统正开始满足“传统”用户的需求（即说一种广泛使用的口头和书面语言的人;识字程度相对较高、接触数字互动和其他资源的人），但全球仍有数亿人被排除在外。具有讽刺意味的是，这些具有资源限制（例如数字和文本素养低）的用户可能是从口语NLP系统的进步中受益最多的用户，这些系统开辟了目前尚未满足的经济，社会和教育可能性。这个项目解决了当今方法的局限性，为目前数字化的“闻所未闻”打开了智能界面。我们将应对三方面的挑战。首先，至关重要的是，需要探索高度创新的ASR技术，以科普具有有限甚至没有文本资源的语言。传统上，ASR系统依赖于大量的转录语音来开发和训练模型。我们的重点是语言，很少有这种数据，实际上是对语言，没有建立书面形式。为了使系统对我们的目标社区有用，他们当然必须以这些用户使用的语言提供相关内容;因此，第二个挑战是建立基础设施和用户参与，以提供生成此类内容的方法。在这样做的过程中，我们希望产生一个蓝图和工具包，可供许多其他低资源或零资源的语言社区使用。最后，我们将与之合作开发这些新方法的用户社区与“传统”用户有着不同的视角，第三个挑战是在与内容和服务的智能界面交互时，要使需求和价值浮出水面，以便底层算法、交互设备和风格是适当和有效的。我们团队之前的工作表明，当这些系统暴露在印度非正式定居点和南非乡镇的群体中时，在传统环境中部署的语音助理中起作用的假设就会崩溃。通过采用这种方法，我们希望创新和颠覆目前在智能语音和语言界面中使用的界面风格和交互设备，不仅满足目前被排除在外的用户的需求，还为世界其他地区提供新的可能性。这项工作汇集了两个世界领先的团体在一个新的合作-爱丁堡的CSTR与语音技术创新的长期跟踪记录;和，在斯旺西的计算铸造厂的FIT实验室，已经率先互动创新和与新兴用户超过十年，发展和倡导负责任的创新与农村社区，城郊和城市发展中国家的背景。这些团体与非政府组织，衍生机构和国际学术利益相关者的现有和新合作都将有助于塑造工作并确保其具有直接和可持续的影响。

Everyday, millions of users talk to machines, be they voice assistants on their mobile phones or standalone devices - such as Alexa or Google Home - in the homes and workplaces. By using Natural Language Processing for input and output through the medium of speech in an intelligent user interface, people are able to access a plethora of content and services. These systems rely on a several factors: i) firstly the use of automatic speech recognition (ASR) that can convert the speech signal into usable language tokens; ii) the availability of knowledge and resources in the system to address the user need; and, iii) an intelligent user interface interaction design that fits the contexts and capabilities of the end user. While state-of-the-art systems are beginning to address the needs of "conventional" users (i.e. those who speak a widely spoken and written language; and who have relatively high degrees of literacy, exposure to digital interactions and other resources), there are many hundreds millions of people who are being excluded globally. Paradoxically, these users who have resource constraints (such as low digital and textual literacy) could be the ones to most benefit from advances in spoken NLP systems opening up economic, social and educational possibilities currently unmet. This project addresses the limitations of today's approaches to open up intelligent interfaces to the currently digitally 'unheard'. The challenges we will address are threefold. Firstly, and crucially, there is a need to explore highly innovative ASR techniques that can cope with languages that have limited or even no textual resources. Conventionally, ASR systems rely on vast amounts of transcribed speech to develop and train models. Our focus is on languages where there is little of this data and indeed on languages where there is no established written form. For the systems to be useful to the sorts of community we target, they have to of course make available relevant content that is in the language these users use; so the second challenge is establish infrastructures and user involvement to provide ways to generate such content. In doing so we hope to produce a blueprint and toolkit that can be used by many other low or zero-resource language communities. Finally, the user communities we will work with to develop these new approaches have a different perspective to "conventional" users and the third challenge is to surface the needs and values when interacting with an intelligent interface for content and services so that the underlying algorithms and the interaction devices and styles are appropriate and effective. Prior work by our team has shown that assumptions of what works in speech assistants that are deployed in conventional settings break down when these systems are exposed to groups in informal settlements in India and townships in South Africa. By taking this approach, we expect to innovate on and disrupt the interface styles and interaction devices currently used in intelligent speech and language interfaces, addressing the need of not just currently excluded users but offering up new possibilities for the rest of the world too. The work brings together two world leading groups in a new collaboration - Edinburgh's CSTR with its long track record on speech technology innovation; and, the FIT Lab at Swansea's computational foundry that has pioneered interaction innovation for and with emergent users for over a decade, developing and advocating responsible innovation with communities in rural, peri-urban and urban developing world contexts. These groups are joined by both existing and new collaborations with NGO, spin-out and international academic stakeholders who will help shape the work and ensure it has direct and sustainable impact.