Enhancing Ensemble Diversity in Neural Ensemble Search for Uncertainty Quantification

点击翻译按钮获取中文摘要

This project falls within the EPSRC `Artificial intelligence technologies' research area.It is an investigation into how to enhance uncertainty quantification through ensembles of deep neural networks.Especially for safety-critical applications such as autonomous driving or medical diagnosis, uncertainty quantification is relevant, however, neural networks are often badly calibrated and display either overly high or low confidence in their predictions. A popular non-Bayesian approach to improve calibration in neural networks is deep ensembles which average predictions of neural networks that have been trained from different random initialisations thereby achieving competitive predictive accuracy and calibration.To further enhance performance, methods proposed in the literature for automatically constructing ensembles of neural networks with varying architectures. They show that architectural variation leads to higher ensemble diversity resulting in ensembles with higher uncertainty calibration and robustness compared to the previous state-of-the-art deep ensembles.To automatically choose base learner architectures they suggest random search in combination with a greedy selection algorithm (NES-RS), an approach that is easily parallelisable, and a more sophisticated evolutionary algorithm based on regularized evolution (NES-RE), which exhibits better performance. Concurrent to this, Wenzel et al. proposed hyper-deep ensembles which aggregate ensembles over different random initialisations stratified over multiple random hyperparameters. However, they keep the architecture of their base learners fixed. The authors show that this approach for inducing diversity into ensembles also leads to increased predictive accuracy and calibration compared to deep ensembles.Therefore, the question arises whether a more directed search for hyperparameters can benefit the construction of more accurate, well-calibrated, and robust ensembles of neural networks, and, moreover, whether it is possible to achieve even better results by integrating the two approaches and extend NES-RE to search for optimal combinations of architecture and hyperparameters (e.g., dropout rate and different L2-regularizers) in base learners.This project will explore this novel approach by experimenting with different modifications to NES-RS and NES-RE that enable searching for ensembles with both varying architectures and hyperparameters simultaneously.Aims and Objectives1. Improve upon the current state-of-the-art NES-RE in terms of uncertainty quantification and robustness.2. If this is the case, investigate whether this approach to uncertainty quantification can aid safety in autonomous driving