III: Small: Collaborative Research: Structured Methods for Multi-Task Learning

人类从相关学习任务中学习和转移知识的能力使我们能够从几个例子中掌握复杂的概念。例如，一个三岁的孩子能够区分椅子和桌子，而不需要接触数百个不同的例子。相比之下，计算机学习程序通常需要在大量示例上进行训练，以达到类似的识别水平。这促使多任务学习的研究，其中多个相关的任务同时学习，从而促进任务间的知识转移。然而，大多数多任务学习研究仅限于明确定义的任务和结构的问题。该项目旨在开发算法和工具（包括开源软件），以解决传统上没有处理的问题，但可以通过多任务学习进行重新制定和更有效地解决。这使得大量具有挑战性的机器学习问题能够从多任务学习技术中受益。该项目还开发了一个新的课程，将拟议的研究纳入课堂。此外，该项目将使PI继续积极招募代表性不足群体的学生并为其提供咨询。为了实现这些目标，本项目着重于一个创新的、综合的研究和教育计划，其中包括以下几个组成部分：（1）为将问题重新表述为多任务学习形式主义提供原则性的指导方针;（2）开发强大的和集群的多任务学习模型，以识别和防止不相关任务之间的错误交互;（3）开发稀疏诱导的多任务学习模型以捕获结构丰富的任务交互：（4）开发高阶多任务学习模型以从特征之间的交互捕获任务相关性;（5）研究多任务学习的计算算法和理论性质。该项目的成果包括将不同的机器学习问题重新定义为多任务学习框架，并提供全新的方法来解决传统方法无法有效解决的挑战性问题。该项目对多任务学习的系统研究有望产生新颖的重新表述、结构化的数学模型、高效的优化算法和原则性的理论分析，这将导致多任务学习的重大实践和理论进展。

The ability of human to learn from and transfer knowledge across related learning tasks enables us to grasp complex concepts from only a few examples. For instance, a three-year old child is able to discriminate chairs from tables without having been exposed to hundreds of different examples. In contrast, computer learning programs typically require training on a large number of examples in order to achieve similar levels of recognition. This prompts the study of multi-task learning in which multiple related tasks are learned simultaneously, thereby facilitating inter-task knowledge transfer. However, most multi-task learning studies are restricted to problems with well-defined tasks and structures. This project aims at developing algorithms and tools (including open source software) to attack problems that are not traditionally treated, but can potentially be reformulated and solved more effectively by multi-task learning. This allows a broad class of challenging machine learning problems to benefit from multi-task learning techniques. This project also develops a new curriculum that incorporates the proposed research into classroom. In addition, this project will allow the PIs to continue the ongoing efforts of actively recruiting and advising students from under-represented groups. To achieve these goals, this project focuses on an innovative, integrated research and education plan that includes the following components: (1) providing principled guidelines for reformulating problems into the multi-task learning formalism; (2) developing robust and clustered multi-task learning models to identify and prevent false interactions among unrelated tasks; (3) developing sparsity-inducing multi-task learning models to capture richly structured task interactions; (4) developing high-order multi-task learning models to capture task relatedness from interactions between features; and (5) investigating computational algorithms and theoretical properties of multi-task learning. The outcome of this project includes the capabilities of reformulating diverse machine learning problems into the multi-task learning framework and providing radically new ways to attack challenging problems that cannot be solved effectively by traditional methods. The systematic study of multi-task learning in this project is expected to generate novel reformulations, structured mathematical models, efficient optimization algorithms, and principled theoretical analyses, which will lead to significant practical and theoretical advances in multi-task learning.