Efficient and Fair Language Modelling for Natural Language Processing, investigating lightweight language modelling approaches and aiming at fairness

自然语言处理的高效公平语言建模，研究轻量级语言建模方法并以公平为目标

• 批准号: 2894795
• 金额: --
• 依托单位: University of Sheffield
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2894795
• 关键词: Efficient; Fair; Language; Modelling; Natural

In natural language processing (NLP), pre-trained language models, such as BERT, and GPT-3 provide the de facto units for processing languages, but have expensive training costs and hardware requirements like Graphics Processing Units (GPUs), and there are questions about their interpretability and fairness. This research will investigate lightweight language modelling approaches and methods to account for fairness in the learning process. The goal is to enable the training of language model in an efficient manner, contributing to environmentally-friendly NLP research and widening access to research communities with limited computational resources. These goals are particularly timely in light of recent legislation and regulations regarding the use of artificial intelligence (AI)-based systems around the world.

在自然语言处理（NLP）中，预先训练的语言模型（如BERT和GPT-3）提供了处理语言的事实单位，但具有昂贵的训练成本和硬件要求，如图形处理单元（GPU），并且存在关于其可解释性和公平性的问题。本研究将调查轻量级语言建模方法和方法，以考虑学习过程中的公平性。目标是以有效的方式训练语言模型，为环境友好的NLP研究做出贡献，并扩大对计算资源有限的研究社区的访问。考虑到最近关于在世界各地使用基于人工智能（AI）的系统的立法和法规，这些目标特别及时。