Sign-Constrained Regularized Loss Minimization
Sign-Constrained Regularized Loss Minimization
复制标题
DOI：
--
复制
发表时间：
2017-10
期刊：
影响因子：
--
通讯作者：
Tsuyoshi Kato;Misato Kobayashi;Daisuke Sano
中科院分区：
文献类型：
--
作者：
Tsuyoshi Kato;Misato Kobayashi;Daisuke Sano
在实际分析中，通常会积累有关分析目标的领域知识，尽管通常这些知识在统计分析阶段已被丢弃，并且统计工具已作为黑匣子应用。在本文中，我们引入了符号约束，对于一般学习问题的非专家来说，这是一种方便而简单的表示。我们通过简单地将符号校正步骤分别插入到原始 Pegasos 和 SDCA 中，开发了两种用于符号约束正则化损失最小化的新优化算法，称为符号约束 Pegasos (SC-Pega) 和符号约束 SDCA (SC-SDCA)。我们提出的理论分析保证了符号校正步骤的插入不会降低两种算法的收敛速度。提出了两个应用程序，其中符号约束学习是有效的。一是利用有关解释变量和目标变量之间相关性的先验信息。另一个是引入符号约束的SVM-Pairwise方法。实验结果表明，通过在两个应用中引入符号约束，泛化性能得到显着提高。
In practical analysis, domain knowledge about analysis target has often been accumulated, although, typically, such knowledge has been discarded in the statistical analysis stage, and the statistical tool has been applied as a black box. In this paper, we introduce sign constraints that are a handy and simple representation for non-experts in generic learning problems. We have developed two new optimization algorithms for the sign-constrained regularized loss minimization, called the sign-constrained Pegasos (SC-Pega) and the sign-constrained SDCA (SC-SDCA), by simply inserting the sign correction step into the original Pegasos and SDCA, respectively. We present theoretical analyses that guarantee that insertion of the sign correction step does not degrade the convergence rate for both algorithms. Two applications, where the sign-constrained learning is effective, are presented. The one is exploitation of prior information about correlation between explanatory variables and a target variable. The other is introduction of the sign-constrained to SVM-Pairwise method. Experimental results demonstrate significant improvement of generalization performance by introducing sign constraints in both applications.