Small Scalable Natural Language Models using Explicit Memory

使用显式记忆的小型可扩展自然语言模型

• 批准号: DP230102775
• 负责人: Prof Tom Drummond
• 金额: $ 33.82万
• 依托单位: The University of Melbourne
• 依托单位国家: 澳大利亚
• 项目类别: Discovery Projects
• 财政年份: 2023
• 资助国家: 澳大利亚
• 起止时间: 2023-01-01 至 2025-12-31
• 项目状态: 未结题
• 来源: https://dataportal.arc.gov.au/RGS/Web/Grants/DP230102775
• 关键词: Small; Scalable; Natural; Language; Models

Deep neural networks have had spectacular success in natural language processing, seeing wide-spread deployment as part of automatic assistant devices in homes and cars, and across many valuable industries including finance, medicine and law. Fueling this success is the use of ever larger models, with exponentially increasing training resources, accompanying hardware and energy demands. This project aims to develop more compact models, based on the incorporation of an explicit searchable memory, which will dramatically reduce model size, hardware requirements and energy usage. This will make modern natural language processing more accessible, while also providing greater flexibility, allowing for more adaptable and portable technologies.

深度神经网络在自然语言处理方面取得了惊人的成功，广泛部署在家庭和汽车以及金融、医药和法律等许多有价值的行业的自动辅助设备中。推动这一成功的是越来越大的型号的使用，培训资源呈指数级增加，伴随而来的是硬件和能源需求。该项目旨在开发更紧凑的模型，基于结合显式可搜索存储器，这将显著减少模型大小、硬件要求和能源使用。这将使现代自然语言处理更容易获得，同时也提供更大的灵活性，允许更适应和可移植的技术。