Collaborative Research: Efficient Bayesian Global Optimization with Applications to Deep Learning and Computer Experiments

点击翻译按钮获取中文摘要

The primary objective of this research is to develop global optimization methods which will dramatically enhance the optimization efficiency in the studies of complex scientific problems. The research findings will significantly accelerate discoveries in numerous scientific disciplines involving artificial intelligence and numerical simulations such as mechanical engineering, energy, automated transportation, aerospace engineering, environmental science, and materials science. Integrated into the research is an education plan that emphasizes interdisciplinary training for a broad body of students and increasing participation from underrepresented groups. The PIs will recruit female students and undergraduate students from underrepresented groups and actively involve them in this research. Research findings will be disseminated at conferences. Furthermore, research findings will also be integrated into PIs’ courses to have optimization and data analysis training for graduate and undergraduate students. This research focuses on Bayesian global optimization which refers to active learning strategies developed by stochastic process priors for the optimization of expensive "black box" functions. Motivated by the challenges emerged from global optimization in deep learning and computer experiments, two innovative Bayesian active learning methods will be developed which are applicable to problems with conditionally dependent inputs and non-Gaussian stochastic outputs. The first method will address an important but unresolved issue arising from the optimization of stochastic outputs in classification problems. The novelty lies in an expected improvement criterion developed based on a generalized Gaussian process which leads to a tractable objective function with an intuitive interpretation. The asymptotic convergence properties will be developed rigorously under the continuum-armed-bandit settings. The second method is based on a new correlation function for a branching and nested structure, which occurs commonly in practice. Sufficient conditions on the validity of the new correlation functions is expected to be derived and a new class of optimal initial designs will be systematically constructed. The innovative idea of automatic-tuning in deep learning by a rigorous Bayesian global optimization will shed light on new methodologies for the optimization of "black box" functions and inspire new research ideas in machine learning, optimization, and spatial statistics. Beyond the applications to computer vision and optimal controls in robotics, the design, modeling, and optimization strategies can open new avenues for studying complex optimization problems with expensive unknown functions and energize both theoretical and applied research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.