Robust and Efficient Structured Prediction

机器学习被广泛应用于科学和技术领域，如电子邮件的自动垃圾邮件分类或数码相机的面部探测器。然而，对于结构化预测的基本问题，即学习进行多个相互关联的预测（例如，预测机器翻译中输入句子的翻译单词顺序），目前的部分解决方案在准确性和可扩展性方面远远落后于二元分类。进步是碎片化的，理论几乎不存在，阻碍了技术在新领域的广泛采用。结构化预测的关键挑战是输出选择的组合爆炸，需要计算和统计的全新结合。******本研究计划的目标是为稳健和高效的结构化预测制定一个通用的理论和算法框架。目标是将结构化预测提升到与现代机器学习中的二元分类相似的成熟度和可用性水平。我计划通过代理损失的统计一致性工具来解决这个问题。我们的理论工作将为建立能够处理弱监督的新的鲁棒结构化预测模型提供基础。通过我们在高级凸优化和组合优化方面的算法工作，将获得更有效的结构化预测机。该框架的适用性将通过计算机视觉、自然语言处理和计算生物学的应用来证明。******更具体地说，我的研究计划将解决以下四个科学挑战：***1)提供统一的结构化预测模型的理论分析。***2)提出新颖的结构化预测模型，具有良好的理论性质，易于处理，并解决该领域的特殊性，如弱监督。***3)设计高效的算法来解决底层大规模的凸或非凸优化问题。***4)演示该框架在多个应用领域的适用性。******结构化预测方面的突破性进展将对统计机器学习研究产生重大影响，特别是为做出鲁棒相关预测的开放问题提供了新的解决方案。此外，所开发的方法将通过广泛采用先进的结构化预测，直接影响科学和技术的许多应用领域。

Machine learning is widely used in science and technology with deployed tools like automatic spam classification for emails or face detectors in digital cameras. Yet today's partial solutions to the fundamental problem of structured prediction, that is, learning to make multiple interrelated predictions (e.g. predicting the sequence of translated words for an input sentence in machine translation), lag far behind those available for binary classification in term of accuracy and scalability. Progress has been fragmented, and the theory is almost nonexistent, preventing the widespread adoption of the technology to new areas. The key challenge in structured prediction is the combinatorial explosion of choices for the output, requiring a radical new marriage of computation and statistics.******The objective of this research program is to elaborate a general theoretical and algorithmic framework for robust and efficient structured prediction. The goal is to bring structured prediction to a level of maturity and usability similar to that of binary classification in modern machine learning. I plan to attack this problem through the tools of statistical consistency of surrogate losses. Our theoretical work will provide the groundwork to build new robust structured prediction models that can handle weak supervision. Radically more efficient structured prediction machines will be obtained through our algorithmic work on advanced convex and combinatorial optimization. The applicability of the framework will be demonstrated through applications in computer vision, natural language processing and computational biology.******More specifically, my research program will address the following four scientific challenges:***1) Provide a unified theoretical analysis of structured prediction models.***2) Propose novel structured prediction models that enjoy good theoretical properties, are tractable, and address the particularities of the field such as weak supervision.***3) Design efficient algorithms that solve the underlying large-scale convex or non-convex optimization problems.***4) Demonstrate the applicability of the framework in several application areas.******Breakthrough progress on structured prediction will have high impact on statistical machine learning research, notably by providing a new solution to the open problem of making robust interrelated predictions. Moreover, the developed methodology will directly impact numerous application areas in science and technology by enabling the widespread adoption of advanced structured prediction.