Convex Regression: Theory, Practice, and Applications
Convex Regression: Theory, Practice, and Applications
复制标题
凸回归：理论、实践和应用
DOI：
10.7939/r3t43j98b
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
Gábor Balázs
中科院分区：
文献类型：
--
作者：
Gábor Balázs
本论文探讨了凸（形状约束）回归的理论，计算和实践方面，提供了新的超额风险上界，凸回归技术与理论保证的比较，一种新的启发式训练算法的最大仿射表示，并在凸随机规划的应用。新的超额风险上界的一般经验风险最小化设置没有任何形状的限制，并提供了一个概率保证的情况下，无界的假设类，目标和噪声模型。强度的一般结果证明，通过将其应用到线性回归下的损失平方套索和岭回归，以及凸非参数最小二乘估计，在每种情况下，允许一个获得近极大极小的风险上限。接下来，切割平面和交替方向方法的乘数算法进行了比较训练的最大仿射最小二乘估计，估计，我们提供明确的超额风险界。这些技术也扩展到分区凸配方（这是享受最佳的极大极小率）。我们还提供了一个实证研究的各种算法来解决非凸优化问题的分区凸制定。设计了一种新的最大仿射估计器，该估计器在大样本情况下具有良好的扩展性，并在许多情况下改善了现有技术的推广误差。它的训练时间是成比例的自适应设置的模型大小，使其在计算上有吸引力的估计问题，目标可以有效地近似最大仿射函数。现实的凸回归应用程序的凸随机规划框架，如能源存储优化，使用太阳能与经济7关税定价模型，以及多产品组装问题的啤酒酿造厂的经营。
This thesis explores theoretical, computational, and practical aspects of convex (shape-constrained) regression, providing new excess risk upper bounds, a comparison of convex regression techniques with theoretical guarantee, a novel heuristic training algorithm for max-affine representations, and applications in convex stochastic programming. The new excess risk upper bound is developed for the general empirical risk minimization setting without any shape constraints, and provides a probabilistic guarantee for cases with unbounded hypothesis classes, targets, and noise models. The strength of the general result is demonstrated by applying it to linear regression under the squared loss both for lasso and ridge regression, as well as for convex nonparametric least squares estimation, in each case allowing one to obtain near-minimax upper bounds on the risk. Next, cutting plane and alternating direction method of multipliers algorithms are compared for training the max-affine least squares estimators; estimators for which we provide explicit excess risk bounds. These techniques are also extended for the partitioned convex formulation (which is shown to enjoy optimal minimax rates). We also provide an empirical study of various heuristics for solving the non-convex optimization problem underlying the partitioned convex formulation. A novel max-affine estimator is designed, which scales well for large sample sizes and improves the generalization error of current techniques in many cases. Its training time is proportional to the adaptively set model size, making it computationally attractive for estimation problems where the target can be efficiently approximated by max-affine functions. Realistic convex regression applications are synthetized for the convex stochastic programming framework such as an energy storage optimization using a solar source with an Economy 7 tariff pricing model, as well as a multi-product assembly problem of operating a beer brewery.