A Primal-Dual Subgradient Approach for Fair Meta Learning
A Primal-Dual Subgradient Approach for Fair Meta Learning
复制标题
DOI：
10.1109/icdm50108.2020.00091
复制
发表时间：
2020-11
期刊：
影响因子：
--
通讯作者：
Chengli Zhao;Feng Chen;Zhuoyi Wang;L. Khan
中科院分区：
文献类型：
--
作者：
Chengli Zhao;Feng Chen;Zhuoyi Wang;L. Khan
在训练步骤中学习对看不见的类进行概括的问题，也称为少镜头分类，已经引起了相当大的关注。基于模型的方法，例如基于梯度的模型不可知元学习（MAML）[1]，通过“学习微调”来解决少数学习问题。这些方法的目标是学习适当的模型初始化，以便可以从少量的梯度更新步骤的标记的例子中学习新类的分类器。少数元学习以其快速适应能力和对未知任务的准确概括而闻名[2]。公平地学习并获得公正的结果是人类智力的另一个重要标志，这在少量元学习中很少被触及。在这项工作中，我们提出了一个原始-对偶公平元学习框架，即PDFM，它只使用一些基于相关任务数据的示例来学习训练公平机器学习模型。其关键思想是学习一个公平模型的原始和对偶参数的良好初始化，以便它可以通过几个梯度更新步骤来适应新的公平学习任务。PDFM不是通过网格搜索手动调整对偶参数作为超参数，而是通过次梯度原始-对偶方法联合优化原始和对偶参数的初始化，以实现公平的元学习。我们进一步实例化了使用决策边界协方差（DBC）[3]作为每个任务的公平性约束的偏差控制示例，并通过将其应用于各种三个真实数据集的分类来证明我们所提出的方法的多功能性。我们的实验表明，在此设置的最佳先前的工作有很大的改进。我们的代码和数据集可以在https://github.com/charliezhaoyinpeng/PDFM.git上找到。
The problem of learning to generalize on unseen classes during the training step, also known as few-shot classification, has attracted considerable attention. Initialization based methods, such as the gradient-based model agnostic meta-learning (MAML) [1], tackle the few-shot learning problem by “learning to fine-tune”. The goal of these approaches is to learn proper model initialization, so that the classifiers for new classes can be learned from a few labeled examples with a small number of gradient update steps. Few shot meta-learning is well-known with its fast-adapted capability and accuracy generalization onto unseen tasks [2]. Learning fairly with unbiased outcomes is another significant hallmark of human intelligence, which is rarely touched in few-shot meta-learning. In this work, we propose a Primal-Dual Fair Meta-learning framework, namely PDFM, which learns to train fair machine learning models using only a few examples based on data from related tasks. The key idea is to learn a good initialization of a fair model's primal and dual parameters so that it can adapt to a new fair learning task via a few gradient update steps. Instead of manually tuning the dual parameters as hyperparameters via a grid search, PDFM optimizes the initialization of the primal and dual parameters jointly for fair meta-learning via a subgradient primal-dual approach. We further instantiate an example of bias controlling using decision boundary covariance (DBC) [3] as the fairness constraint for each task, and demonstrate the versatility of our proposed approach by applying it to classification on a variety of three realworld datasets. Our experiments show substantial improvements over the best prior work for this setting. Our code and datasets are available at https://github.com/charliezhaoyinpeng/PDFM.git.