Stochastic gradient boosting
Stochastic gradient boosting
复制标题
DOI：
10.1016/s0167-9473(01)00065-2
复制
发表时间：
2002-02-28
影响因子：
1.8
通讯作者：
Friedman, JH
中科院分区：
文献类型：
--
作者：
Friedman, JH
梯度提升通过在每次迭代时通过最小二乘法将简单的参数化函数（基础学习器）顺序拟合到当前的“伪”残差来构建加性回归模型。伪残差是相对于在当前步骤评估的每个训练数据点处的模型值被最小化的损失函数的梯度。研究表明，通过将随机化纳入梯度提升过程，可以大大提高梯度提升的逼近精度和执行速度。具体地说，在每次迭代中，训练数据的子样本从完整的训练数据集中随机抽取（不替换）。然后使用这个随机选择的子样本代替完整样本来拟合基本学习器并计算当前迭代的模型更新。这种随机化的方法还增加了对基础学习器的能力过剩的鲁棒性。(C)2002爱思唯尔科技有限公司。保留所有权利。
Gradient boosting constructs additive regression models by sequentially fitting a simple parameterized function (base learner) to current "pseudo"-residuals by least squares at each iteration. The pseudo-residuals are the gradient of the loss functional being minimized, with respect to the model values at each training data point evaluated at the current step. It is shown that both the approximation accuracy and execution speed of gradient boosting can be substantially improved by incorporating randomization into the procedure. Specifically, at each iteration a subsample of the training data is drawn at random (without replacement) from the full training data set. This randomly selected subsample is then used in place of the full sample to fit the base learner and compute the model update for the current iteration. This randomized approach also increases robustness against overcapacity of the base learner. (C) 2002 Elsevier Science B.V. All rights reserved.