权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Collaborative Research: Efficient Bayesian Global Optimization with Applications to Deep Learning and Computer Experiments

协作研究：高效贝叶斯全局优化及其在深度学习和计算机实验中的应用

基本信息

批准号：
2113407
负责人：
Chih-Li Sung
金额：
$ 14.2万
依托单位：
Michigan State University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-07-01 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113407&HistoricalAwards=false
关键词：
Collaborative Research Efficient Bayesian Global

项目摘要

The primary objective of this research is to develop global optimization methods which will dramatically enhance the optimization efficiency in the studies of complex scientific problems. The research findings will significantly accelerate discoveries in numerous scientific disciplines involving artificial intelligence and numerical simulations such as mechanical engineering, energy, automated transportation, aerospace engineering, environmental science, and materials science. Integrated into the research is an education plan that emphasizes interdisciplinary training for a broad body of students and increasing participation from underrepresented groups. The PIs will recruit female students and undergraduate students from underrepresented groups and actively involve them in this research. Research findings will be disseminated at conferences. Furthermore, research findings will also be integrated into PIs’ courses to have optimization and data analysis training for graduate and undergraduate students. This research focuses on Bayesian global optimization which refers to active learning strategies developed by stochastic process priors for the optimization of expensive "black box" functions. Motivated by the challenges emerged from global optimization in deep learning and computer experiments, two innovative Bayesian active learning methods will be developed which are applicable to problems with conditionally dependent inputs and non-Gaussian stochastic outputs. The first method will address an important but unresolved issue arising from the optimization of stochastic outputs in classification problems. The novelty lies in an expected improvement criterion developed based on a generalized Gaussian process which leads to a tractable objective function with an intuitive interpretation. The asymptotic convergence properties will be developed rigorously under the continuum-armed-bandit settings. The second method is based on a new correlation function for a branching and nested structure, which occurs commonly in practice. Sufficient conditions on the validity of the new correlation functions is expected to be derived and a new class of optimal initial designs will be systematically constructed. The innovative idea of automatic-tuning in deep learning by a rigorous Bayesian global optimization will shed light on new methodologies for the optimization of "black box" functions and inspire new research ideas in machine learning, optimization, and spatial statistics. Beyond the applications to computer vision and optimal controls in robotics, the design, modeling, and optimization strategies can open new avenues for studying complex optimization problems with expensive unknown functions and energize both theoretical and applied research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

本研究的主要目标是发展全局优化方法，以显著提高复杂科学问题研究中的优化效率。该研究成果将显著加速机械工程、能源、自动化运输、航空航天工程、环境科学、材料科学等涉及人工智能和数值模拟的众多科学学科的发现。与这项研究相结合的是一项教育计划，该计划强调为广大学生提供跨学科培训，并增加代表性不足群体的参与。pi将从代表性不足的群体中招募女学生和本科生，并积极让她们参与本研究。研究结果将在会议上传播。此外，研究成果也将整合到pi的课程中，对研究生和本科生进行优化和数据分析训练。本研究的重点是贝叶斯全局优化，这是一种由随机过程先验开发的主动学习策略，用于优化昂贵的“黑箱”函数。在深度学习和计算机实验中出现的全局优化挑战的激励下，将开发两种创新的贝叶斯主动学习方法，适用于具有条件依赖输入和非高斯随机输出的问题。第一种方法将解决一个重要但尚未解决的问题，即在分类问题中随机输出的优化。其新颖性在于基于广义高斯过程开发的期望改进准则，该准则可导致易于处理的目标函数并具有直观的解释。在连续武装-强盗设置下，将严格地发展渐近收敛性质。第二种方法是基于分支和嵌套结构的新的关联函数，这种方法在实践中很常见。期望得到新的相关函数有效性的充分条件，并系统地构建一类新的最优初始设计。通过严格的贝叶斯全局优化在深度学习中进行自动调整的创新思想将为优化“黑匣子”函数提供新的方法，并激发机器学习，优化和空间统计方面的新研究思路。除了应用于计算机视觉和机器人的最优控制之外，设计、建模和优化策略可以为研究具有昂贵未知函数的复杂优化问题开辟新的途径，并为理论和应用研究提供活力。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。