Security and compilers for machine learning

点击翻译按钮获取中文摘要

Machine learning (ML) is rapidly gaining traction across various industries, promising transformative benefits in diverse fields. However, the increasing reliance on ML systems has brought to light the crucial need for robust security and safety measures. This is due to the inherent vulnerabilities associated with ML models and the potential consequences of their misuse.One of the primary concerns is the susceptibility of ML models to adversarial attacks, where malicious actors manipulate data, model parameters, or model architecture to exploit the system. These attacks can result in biased, inaccurate, and even dangerous decision-making. Additionally, the complexity of ML models makes it challenging to identify and mitigate vulnerabilities, making them difficult to defend against.Another significant issue is AI alignment. Alignment refers to the process of ensuring that artificial intelligence (AI) systems behave in ways that align with human values and objectives. It involves developing techniques to guide AI models towards making decisions and taking actions that are beneficial to humanity, while minimizing potential harms. AI alignment is crucial for the responsible development and deployment of AI systems, as it helps ensure that AI technologies align with human interests and are used ethically and beneficially.This PhD explores various avenues in improving ML security and safety. The initial projects are as follows.One project is to improve the quality of human preference data used to fine-tune ML models. Alignment relies heavily on the quality of human preferences, but much of the existing data is generated by overworked and underpaid workers with no real incentive to provide good data. This project would experimentally research the effect of paying workers bonuses based on whether their preferences successfully improve the performance of the model on existing benchmarks. The research will also determine whether increasing human motivation in this way increases the performance of the model even in metrics which are not rewarded. If so, this could lead to better performance in metrics for which there are no good benchmarks, such as political bias.Another project to improve the security of models against adversarial attack is to investigate whether the new push towards self-rewarding language models creates an opportunity for backdoors to be amplified through the inherent feedback loop in these self-rewarding models. This follows from ideas such as Model Collapse, in which training on LLM-generated data can lead to total performance failure, and the existing body of work on data poisoning to insert backdoors in LLMs.A third project is to investigate various methods for locking machine learning models to specific hardware, such as by using a difficult-to-forge hardware fingerprint (e.g. based on the number of clock cycles required to complete an operation) as an encryption key for the weights of the model, or by optimising models for particular quantisation schemes that only exist on some hardware.This project aligns with the EPSRC research area "Artificial intelligence technologies".