III: Small: Collaborative Research: Structured Methods for Multi-Task Learning

人类在相关学习任务中学习和转移知识的能力使我们能够仅从几个例子中掌握复杂的概念。例如，一个三岁的孩子能够区分椅子和桌子，而不需要接触数百个不同的例子。相比之下，计算机学习程序通常需要在大量的例子上进行训练，才能达到类似的识别水平。这促使人们研究多任务学习，即同时学习多个相关的任务，从而促进任务间的知识迁移。然而，大多数多任务学习研究仅限于具有明确任务和结构的问题。该项目旨在开发算法和工具（包括开源软件）来解决传统上没有处理的问题，但可以通过多任务学习来重新制定和更有效地解决这些问题。这使得广泛的具有挑战性的机器学习问题受益于多任务学习技术。该项目还开发了一个新的课程，将拟议的研究纳入课堂。此外，该项目将使pi能够继续积极招募来自代表性不足群体的学生并为其提供咨询。为了实现这些目标，本项目侧重于一个创新的、综合的研究和教育计划，该计划包括以下组成部分：(1)为将问题重新表述为多任务学习形式主义提供原则性指导；(2)建立稳健的聚类多任务学习模型，以识别和防止不相关任务之间的错误交互；(3)开发稀疏性诱导多任务学习模型，以捕获结构丰富的任务交互；(4)建立高阶多任务学习模型，从特征之间的相互作用中捕捉任务相关性；(5)研究多任务学习的计算算法和理论性质。该项目的成果包括将各种机器学习问题重新表述为多任务学习框架的能力，并提供全新的方法来解决传统方法无法有效解决的具有挑战性的问题。本项目对多任务学习的系统研究有望产生新颖的重新表述、结构化的数学模型、高效的优化算法和原则性的理论分析，这将导致多任务学习在实践和理论方面取得重大进展。

The ability of human to learn from and transfer knowledge across related learning tasks enables us to grasp complex concepts from only a few examples. For instance, a three-year old child is able to discriminate chairs from tables without having been exposed to hundreds of different examples. In contrast, computer learning programs typically require training on a large number of examples in order to achieve similar levels of recognition. This prompts the study of multi-task learning in which multiple related tasks are learned simultaneously, thereby facilitating inter-task knowledge transfer. However, most multi-task learning studies are restricted to problems with well-defined tasks and structures. This project aims at developing algorithms and tools (including open source software) to attack problems that are not traditionally treated, but can potentially be reformulated and solved more effectively by multi-task learning. This allows a broad class of challenging machine learning problems to benefit from multi-task learning techniques. This project also develops a new curriculum that incorporates the proposed research into classroom. In addition, this project will allow the PIs to continue the ongoing efforts of actively recruiting and advising students from under-represented groups. To achieve these goals, this project focuses on an innovative, integrated research and education plan that includes the following components: (1) providing principled guidelines for reformulating problems into the multi-task learning formalism; (2) developing robust and clustered multi-task learning models to identify and prevent false interactions among unrelated tasks; (3) developing sparsity-inducing multi-task learning models to capture richly structured task interactions; (4) developing high-order multi-task learning models to capture task relatedness from interactions between features; and (5) investigating computational algorithms and theoretical properties of multi-task learning. The outcome of this project includes the capabilities of reformulating diverse machine learning problems into the multi-task learning framework and providing radically new ways to attack challenging problems that cannot be solved effectively by traditional methods. The systematic study of multi-task learning in this project is expected to generate novel reformulations, structured mathematical models, efficient optimization algorithms, and principled theoretical analyses, which will lead to significant practical and theoretical advances in multi-task learning.