Augmented Multi-Task Learning by Optimal Transport
Augmented Multi-Task Learning by Optimal Transport
复制标题
通过最佳传输增强多任务学习
DOI：
10.1137/1.9781611975673.3
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
and Zhou, J
中科院分区：
文献类型：
--
作者：
Liu, B.;Tan, P.N.;and Zhou, J
多任务学习（MTL）提供了一种有效的方法来改善多个相关预测任务的泛化误差，通过联合学习任务，假设它们的模型参数具有共同的结构。尽管它的成功，共享参数的假设是无效的，当样本量为某些任务太小，从数据中推断出正确的任务关系。为了克服这一限制，我们提出了一种新的框架，通过使用从其他相关任务的训练数据生成的伪标记实例来增加每个任务的有效样本量。从其他任务中提取训练数据对于回归问题来说是一个挑战，因为由于协变量偏移和响应漂移问题，它们的数据分布可能不一致。我们提出的框架通过将多任务回归与一系列最佳传输步骤相结合来解决这一挑战，通过从其他源域中识别相关训练实例并细化伪标签来迭代学习伪标签实例，直到它们与目标域的训练实例一致。在合成数据和真实数据上的实验结果表明，我们的框架始终优于其他最先进的MTL方法。
Multi-task learning (MTL) provides an effective approach to improve generalization error for multiple related prediction tasks by learning the tasks jointly, assuming there is a common structure shared by their model parameters. Despite its successes, the shared parameter assumption is ineffective when the sample sizes for some tasks are too small to infer the task relationships correctly from data. To overcome this limitation, we propose a novel framework for increasing the effective sample size of each task by augmenting it with pseudo-labeled instances generated from the training data of other related tasks. Incorporating training data from other tasks is a challenge for regression problems as their data distributions may not be consistent due to the co-variate shift and response drift problems. Our proposed framework addresses this challenge by coupling multitask regression with a series of optimal transport steps to iteratively learn the pseudo-labeled instances by identifying relevant training instances from other source domains and refining the pseudo-labels until they are consistent with the training instances of the target domain. Experimental results on both synthetic and real-world data showed that our framework consistently outperformed other state-of-the-art MTL methods.