Collaborative Research: Efficient Bayesian Global Optimization with Applications to Deep Learning and Computer Experiments

协作研究：高效贝叶斯全局优化及其在深度学习和计算机实验中的应用

基本信息

批准号：
2113407
负责人：
Chih-Li Sung
金额：
$ 14.2万
依托单位：
Michigan State University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-07-01 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113407&HistoricalAwards=false
关键词：
Collaborative Research Efficient Bayesian Global

项目摘要

The primary objective of this research is to develop global optimization methods which will dramatically enhance the optimization efficiency in the studies of complex scientific problems. The research findings will significantly accelerate discoveries in numerous scientific disciplines involving artificial intelligence and numerical simulations such as mechanical engineering, energy, automated transportation, aerospace engineering, environmental science, and materials science. Integrated into the research is an education plan that emphasizes interdisciplinary training for a broad body of students and increasing participation from underrepresented groups. The PIs will recruit female students and undergraduate students from underrepresented groups and actively involve them in this research. Research findings will be disseminated at conferences. Furthermore, research findings will also be integrated into PIs’ courses to have optimization and data analysis training for graduate and undergraduate students. This research focuses on Bayesian global optimization which refers to active learning strategies developed by stochastic process priors for the optimization of expensive "black box" functions. Motivated by the challenges emerged from global optimization in deep learning and computer experiments, two innovative Bayesian active learning methods will be developed which are applicable to problems with conditionally dependent inputs and non-Gaussian stochastic outputs. The first method will address an important but unresolved issue arising from the optimization of stochastic outputs in classification problems. The novelty lies in an expected improvement criterion developed based on a generalized Gaussian process which leads to a tractable objective function with an intuitive interpretation. The asymptotic convergence properties will be developed rigorously under the continuum-armed-bandit settings. The second method is based on a new correlation function for a branching and nested structure, which occurs commonly in practice. Sufficient conditions on the validity of the new correlation functions is expected to be derived and a new class of optimal initial designs will be systematically constructed. The innovative idea of automatic-tuning in deep learning by a rigorous Bayesian global optimization will shed light on new methodologies for the optimization of "black box" functions and inspire new research ideas in machine learning, optimization, and spatial statistics. Beyond the applications to computer vision and optimal controls in robotics, the design, modeling, and optimization strategies can open new avenues for studying complex optimization problems with expensive unknown functions and energize both theoretical and applied research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项研究的主要目的是开发全球优化方法，该方法将大大提高复杂科学问题研究的优化效率。研究发现将在众多科学学科中大大加快涉及人工智能和数值模拟的发现，例如机械工程，能源，自动运输，航空工程，环境科学和材料科学。整合到研究中的是一项教育计划，该计划强调了针对大量学生的跨学科培训，并增加了代表性不足的群体的参与。 PI将招募来自代表性不足的小组的女学生和本科生，并积极参与这项研究。研究结果将在会议上传播。此外，研究结果还将集成到PIS的课程中，以对研究生和本科生进行优化和数据分析培训。这项研究重点是贝叶斯全球优化，这是指由随机过程先验开发的积极学习策略，以优化昂贵的“黑匣子”功能。由全球优化和计算机实验中的全球优化所带来的挑战所激发，将开发两种创新的贝叶斯主动学习方法，这些方法适用于有条件依赖的输入和非高斯随机输出的问题。第一种方法将解决由分类问题中随机输出的优化引起的重要但尚未解决的问题。新颖性在于基于广义高斯过程开发的预期改进标准，该过程导致具有直观解释的可聊天目标函数。不对称的收敛性能将在连续武装的伴随式环境下进行严格开发。第二种方法基于分支和嵌套结构的新相关函数，该函数通常在实践中发生。预计将得出有关新相关函数有效性的足够条件，并系统地构建新的最佳初始设计。严格的贝叶斯全球优化在深度学习中进行自动调整的创新思想将阐明“黑匣子”功能的新方法，并激发机器学习，优化和空间统计的新研究思想。除了对机器人技术中的计算机视觉和最佳控制的应用外，设计，建模和优化策略还可以为研究复杂的优化问题提供新的途径，并具有昂贵的未知功能，并为理论和应用研究提供了充满活力。该奖项反映了NSF的法规使命，并认为通过基金会的知识优点和广泛的criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia criperia均被认为是宝贵的。