Moment Distributionally Robust Tree Structured Prediction
Moment Distributionally Robust Tree Structured Prediction
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Yeshu Li;D. Saeed;Xinhua Zhang;Brian D. Ziebart;Kevin Gimpel
中科院分区：
文献类型：
--
作者：
Yeshu Li;D. Saeed;Xinhua Zhang;Brian D. Ziebart;Kevin Gimpel
树形对象的结构化预测在句法依赖分析的名义下得到了大量的研究。基于最大似然法或边际法的现行做法对评估损失是不可知的或不一致的。风险最小化消除了训练和测试目标之间的差异，但通常会导致非凸问题。这些方法采用显式正则化来对抗过拟合，而无需概率解释。我们提出了一种基于矩的分布鲁棒优化方法的树结构预测，其中最坏情况下的预期损失超过一组分布内的有界矩偏离经验分布最小化。我们开发有效的算法树形和其他变种的树。我们推导出我们所提出的方法的Fisher一致性，收敛速度和推广界。我们评估其依赖分析基准的实证有效性。
Structured prediction of tree-shaped objects is heavily studied under the name of syntactic dependency parsing. Current practice based on maximum likelihood or margin is either agnostic to or inconsistent with the evaluation loss. Risk minimization alleviates the discrepancy between training and test objectives but typically induces a non-convex problem. These approaches adopt explicit regularization to combat overfitting without probabilistic interpretation. We propose a moment-based distributionally robust optimization approach for tree structured prediction, where the worst-case expected loss over a set of distributions within bounded moment divergence from the empirical distribution is minimized. We develop efficient algorithms for arborescences and other variants of trees. We derive Fisher consistency, convergence rates and generalization bounds for our proposed method. We evaluate its empirical effectiveness on dependency parsing benchmarks.