RI: Small: Ensemble Methods for Structured Prediction

集成方法是机器学习中的通用技术，用于组合多个假设以创建更准确的预测器。在批量学习设置中，诸如装袋、提升、堆叠、纠错技术、贝叶斯平均或其他平均方案等技术是这些方法的常见实例。这些方法通常在实践中显著提高性能，并且通常受益于有利的学习保证，特别是在训练样本的边际上。然而，集成方法及其理论主要是为常见的二元分类问题或标准回归任务而开发的，其中目标标签是实数，因此没有结构。这些技术不容易应用于结构化预测问题，如发音建模、语音识别、解析、机器翻译或图像处理。本提案的目标是为设计有效的结构化预测技术集成创建理论基础，大规模算法和实用技术。这些算法的好处可能至少与二元分类中的集成技术所带来的好处一样显著。我们的解决方案将对广泛的应用程序至关重要，并将通过开源软件程序被广泛使用。这些软件和开源程序将使广泛的研究人员和工程师社区能够使用我们的学习算法。更广泛地说，我们的技术将通过发现更准确的解决方案来解决各种重要问题，包括语音识别、语音合成和机器翻译，从而造福社会。

Ensemble methods are general techniques in machine learning for combining several hypotheses to create a more accurate predictor. In the batch learning setting, techniques such as bagging, boosting, stacking, error-correction techniques, Bayesian averaging, or other averaging schemes are common instances of these methods. These methods often significantly improve performance in practice and often benefit from favorable learning guarantees, typically in terms of the margins of the training samples. However, ensemble methods and their theory have been developed primarily for the common binary classification problem, or standard regression tasks where the target labels are real numbers and thus have no structure. These techniques do not readily apply to structured prediction problems such as pronunciation modeling, speech recognition, parsing, machine translation, or image processing. The objective of this proposal is to create the theoretical foundation, large-scale algorithms, and practical techniques for devising effective ensembles of structured prediction techniques. The benefits of these algorithms are likely to be at least as significant as those resulting from ensemble techniques in binary classification.Our solutions will be crucial to a broad set of applications and will be made widely accessible through open-source software programs. These software and open-source programs will make the use of our learning algorithms accessible to a broad community of researchers and engineers. More broadly, our techniques will benefit the society through the discovery of significantly more accurate solutions to a variety of important problems including speech recognition, speech synthesis, and machine translation.