Small Scalable Natural Language Models using Explicit Memory

深度神经网络在自然语言处理方面取得了巨大的成功，在家庭和汽车以及包括金融、医药和法律在内的许多有价值的行业中，作为自动辅助设备的一部分得到了广泛的部署。推动这一成功的是使用越来越大的模型，训练资源呈指数级增长，伴随着硬件和能源需求。该项目旨在开发更紧凑的模型，基于显式可搜索存储器的合并，这将大大减少模型大小，硬件要求和能源使用。这将使现代自然语言处理更容易获得，同时也提供了更大的灵活性，允许更适应和便携的技术。

Deep neural networks have had spectacular success in natural language processing, seeing wide-spread deployment as part of automatic assistant devices in homes and cars, and across many valuable industries including finance, medicine and law. Fueling this success is the use of ever larger models, with exponentially increasing training resources, accompanying hardware and energy demands. This project aims to develop more compact models, based on the incorporation of an explicit searchable memory, which will dramatically reduce model size, hardware requirements and energy usage. This will make modern natural language processing more accessible, while also providing greater flexibility, allowing for more adaptable and portable technologies.