Efficient and Fair Language Modelling for Natural Language Processing, investigating lightweight language modelling approaches and aiming at fairness

点击翻译按钮获取中文摘要

In natural language processing (NLP), pre-trained language models, such as BERT, and GPT-3 provide the de facto units for processing languages, but have expensive training costs and hardware requirements like Graphics Processing Units (GPUs), and there are questions about their interpretability and fairness. This research will investigate lightweight language modelling approaches and methods to account for fairness in the learning process. The goal is to enable the training of language model in an efficient manner, contributing to environmentally-friendly NLP research and widening access to research communities with limited computational resources. These goals are particularly timely in light of recent legislation and regulations regarding the use of artificial intelligence (AI)-based systems around the world.