End-to-End Learning for Prediction and Optimization with Gradient Boosting
End-to-End Learning for Prediction and Optimization with Gradient Boosting
复制标题
DOI：
10.1007/978-3-030-67664-3_12
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Takuya Konishi;Takuro Fukunaga
中科院分区：
文献类型：
--
作者：
Takuya Konishi;Takuro Fukunaga
数学优化是决策的基本工具。然而，由于参数的不确定性，通常很难获得优化问题的精确公式。机器学习框架对于解决这个问题很有吸引力：我们预测不确定的参数，然后根据预测优化问题。最近，预测和优化连续问题的端到端学习方法在优化和机器学习领域都受到了关注。在本文中，我们专注于梯度助推，这是一个强大的集成方法，并开发了端到端的学习算法，直接优化问题的性能最大化。我们的算法通过隐式微分将现有的基于梯度的优化扩展到二阶优化，以有效地学习梯度提升。我们还进行了计算实验，分析如何端到端的方法工作得很好，并显示我们的端到端的方法的有效性。
Mathematical optimization is a fundamental tool in decision making. However, it is often difficult to obtain an accurate formulation of an optimization problem due to uncertain parameters. Machine learning frameworks are attractive to address this issue: we predict the uncertain parameters and then optimize the problem based on the prediction. Recently, end-to-end learning approaches to predict and optimize the successive problems have received attention in the field of both optimization and machine learning. In this paper, we focus on gradient boosting which is known as a powerful ensemble method, and develop the end-to-end learning algorithm with maximizing the performance on the optimization problems directly. Our algorithm extends the existing gradient-based optimization through implicit differentiation to the second-order optimization for efficiently learning gradient boosting. We also conduct computational experiments to analyze how the end-to-end approaches work well and show the effectiveness of our end-to-end approach.