Mediated Uncoupled Learning and Validation with Bregman Divergences: Loss Family with Maximal Generality
Mediated Uncoupled Learning and Validation with Bregman Divergences: Loss Family with Maximal Generality
复制标题
DOI：
--
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
Ikko Yamane;Y. Chevaleyre;Takashi Ishida;F. Yger
中科院分区：
文献类型：
--
作者：
Ikko Yamane;Y. Chevaleyre;Takashi Ishida;F. Yger
在中介分离学习(MU-Learning)中，目标是在给定输入变量X的情况下预测输出变量Y，而训练数据集没有(X，Y)的联合样本，而只有(X，U)和(U，Y)的独立样本，每个样本都带有中介变量U。现有的MU学习方法只能处理平方损失，不能使用其他流行的损失函数，如交叉熵损失。我们提出了一个通用的MU-学习框架，该框架以统一的方式考虑了Bregman分歧的问题，该分歧涵盖了对各种类型的任务有用的广泛的损失函数。这个损失族在条件期望的极小化特征中具有极大的通用性。我们证明了所提出的目标函数是对Oracle损失的更紧密的近似，如果(X，Y)的普通监督样本可用，则可以最小化该损失。我们还提出了仅使用(X，U)-和(U，Y)-数据来估计包含训练模型的预测的预期检验损失的区间的估计量。通过对合成数据的回归实验和基准数据集的低质量图像分类实验，对该方法的超额风险进行了理论分析，并验证了该方法的实用性。
In mediated uncoupled learning (MU-learning), the goal is to predict an output variable Y given an input variable X as in ordinary supervised learning while the training dataset has no joint samples of ( X, Y ) but only independent samples of ( X, U ) and ( U, Y ) each observed with a mediating variable U . The existing MU-learning methods can only handle the squared loss, which prohibited the use of other popular loss functions such as the cross-entropy loss. We propose a general MU-learning framework that allows for the problems with Bregman divergences, which cover a wide range of loss functions useful for various types of tasks, in a unified manner. This loss family has maximal generality among those whose minimizers characterize the conditional expectation. We prove that the proposed objective function is a tighter approximation to the oracle loss that one would minimize if ordinary supervised samples of ( X, Y ) were available. We also propose an estimator of an interval containing the expected test loss of predictions of a trained model only using ( X, U ) - and ( U, Y ) -data. We provide a theoretical analysis on the excess risk for the proposed method and confirm its practical usefulness with regression experiments with synthetic data and low-quality image classification experiments with benchmark datasets.