Efficient and Fair Language Modelling for Natural Language Processing, investigating lightweight language modelling approaches and aiming at fairness

在自然语言处理(NLP)中，预先训练的语言模型，如BERT和GPT-3，为处理语言提供了事实上的单位，但像图形处理单元(GPU)一样，训练成本和硬件要求都很高，而且它们的可解释性和公平性也存在问题。这项研究将调查轻量级语言建模的途径和方法，以说明学习过程中的公平性。其目标是以有效的方式培训语言模型，促进对环境友好的自然语言规划研究，并以有限的计算资源扩大进入研究界的机会。鉴于世界各地最近关于使用基于人工智能(AI)的系统的立法和法规，这些目标尤其及时。

In natural language processing (NLP), pre-trained language models, such as BERT, and GPT-3 provide the de facto units for processing languages, but have expensive training costs and hardware requirements like Graphics Processing Units (GPUs), and there are questions about their interpretability and fairness. This research will investigate lightweight language modelling approaches and methods to account for fairness in the learning process. The goal is to enable the training of language model in an efficient manner, contributing to environmentally-friendly NLP research and widening access to research communities with limited computational resources. These goals are particularly timely in light of recent legislation and regulations regarding the use of artificial intelligence (AI)-based systems around the world.