Adaptive Design for Fast Machine/Statistical Learning

快速机器/统计学习的自适应设计

• 批准号: RGPIN-2019-05019
• 负责人: Welch, William
• 金额: $ 1.82万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=714924
• 关键词: Adaptive; Design; Fast; Machine; Statistical

The overarching goal of the research program is to extend Gaussian processes (GPs) to enable much more complex applications. First, new methodology will scale up GPs to enable large sample sizes and use adaptive sampling, for accurate statistical modelling of complex relationships arising cross a broad spectrum of scientific and engineering disciplines. Second, efficiencies in adaptive search methods using GPs will allow automatic tuning of computationally intensive machine/statistical learners. GPs have had profound impact on science and engineering, where they are used directly as machine/statistical learners. Complex computer codes of physical systems can be too slow for optimization, calibration of unknowns, sensitivity analysis, etc. GPs trained on limited computer model runs are used for these purposes as computationally fast surrogates for the particular scientific objective. It is well known, however, that the computational time to train a GP increases as the cube of the sample size. Thus, GPs are less attractive for sample sizes of a few thousand or more. Existing methods, mainly based on localized modelling or special fixed experimental designs, will be assessed to determine the domain of problems where they are effective. It is clear in advance, however, that new methods will be required for complex applications: those with moderate to high-dimensional input, nonlinear relationships, and/or high-order interaction effects. Only by adapting the experiment - taking further observations where the target function has special features - can a dense sampling of the input space be obtained where it matters. Divide and conquer methods are especially promising. How to divide high-dimensional space, how to choose sub-regions for data augmentation, and guidance on the number of new runs per iteration will be critical research questions here. GPs are also used indirectly in support of other machine-learning (ML) methods such as deep learning neural networks. Neural networks for image classification, for example, have "tuning" parameters that have to be set by the user, to determine the basic network architecture or regularization, for instance. Users tune these so-called hyperparameters by trying different values and attempting to minimize validation error in various ways. To obtain the validation error requires training the ML method, which is itself computationally very intensive. Hence, systematic methods known as Bayesian optimization train a GP to model the relationship between the hyperparameter settings and validation error, and hence adaptively optimize the error. The research program will continue work in my lab on "automatic ML", to minimize the number of tries of the expensive underlying ML algorithm. Advances here will likely have impact on other computationally challenging optimization problems where the objective is produced by an expensive algorithm.

研究计划的首要目标是扩展高斯过程（GPs），以实现更复杂的应用。首先，新的方法将扩大全球定位系统，以实现大样本量，并使用自适应抽样，以便对广泛的科学和工程学科之间产生的复杂关系进行准确的统计建模。其次，使用GPs的自适应搜索方法的效率将允许自动调整计算密集型机器/统计学习器。