Learning reliable representations when proxy objectives fail

表征学习涉及使用一个目标来学习从数据空间到表征空间的映射。当必须学习映射的下游任务未知或成本太高而无法设定为目标时，我们必须依靠代理目标进行学习。在这篇论文中，我专注于图像的表征学习，并解决了三种代理目标无法产生在下游任务上执行良好的映射的情况。当学习从图像空间到离散散列空间的神经网络映射用于基于内容的图像检索时，需要一个代理目标来捕捉相关响应比不相关的响应更接近任何查询的散列的要求。同时，重要的是要确保图像散列在整个散列空间中的均匀分布，以有效地利用信息并提高区分度。当代理目标不满足这些要求时，它们将失败。我建议使用标准分类器来预测类别标签，并将其转换为二进制表示，以便在图像检索任务中实现最先进的性能。我还提出了一种二进制深度决策树层(DDTL)来进一步建模类内差异，并产生近似均匀分布的哈希码。DDTL在学习过程中不需要离散化，并产生哈希码，与以前的方法相比，哈希码能够更好地区分同一类中的数据，同时保持对数据空间中真实世界的扩展的健壮性。在我们需要神经网络将数据划分成与地面事实标签很好地对应的集群的场景中，需要一个代理目标来定义这些集群是如何形成的。一个这样的代理目标涉及最大化由神经网络从多个视点进行的集群分配之间的互信息。在这种情况下，视图是对同一图像的不同放大，而聚类分配是由神经网络计算的表示。我证明了这个代理目标为神经网络产生了次优的参数，因为使用相同的目标和不同的训练方法可以找到更好的参数集。我引入了深度层次对象分组(DHOG)作为一种方法来学习代理目标的解决方案的层次结构(在易于难以排序的意义上，而不是结构)，并展示了这如何提高下游任务的性能。当训练数据中存在更容易计算类别预测的特征(例如，背景颜色)时，当与计算类别预测相对较困难的特征(例如，数字类型)相比时，标准分类目标(例如，交叉熵)不能产生稳健的分类器。问题是，如果一个模型学会依赖“简单”特征，它也会忽略“复杂”特征(在这种情况下，简单与复杂纯粹是相对的)。通过使用矢量量化变分自动编码器将训练数据的底层结构建模为潜在表示，然后使用基于梯度的过程来调整该表示中的特征以混淆受约束的分类器的预测，从而引入潜在对抗性去偏向(LAD)来将容易的特征从类别标签中分离出来。然后对数据的调整表示进行解码，以产生可用于以标准方式进行训练的扩充训练数据集。我在前面提到的场景中展示了代理目标可能会失败，并展示了替代方法可以缓解相关的失败。我建议使用一种分析方法来了解每个用例的代理目标的限制，以便对数据或目标进行调整，并确保下游任务的良好性能。

Representation learning involves using an objective to learn a mapping from data space to a representation space. When the downstream task for which a mapping must be learned is unknown, or is too costly to cast as an objective, we must rely on proxy objectives for learning. In this Thesis I focus on representation learning for images, and address three cases where proxy objectives fail to produce a mapping that performs well on the downstream tasks. When learning neural network mappings from image space to a discrete hash space for content-based image retrieval, a proxy objective is needed which captures the requirement for relevant responses to be nearer to the hash of any query than irrelevant ones. At the same time, it is important to ensure an even distribution of image hashes across the whole hash space for efficient information use and high discrimination. Proxy objectives fail when they do not meet these requirements. I propose using a standard classifier to predict class labels and convert these to a binary representation for state-of-the-art performance on the image retrieval task. I also propose a binary deep decision tree layer (DDTL) to model further intra-class differences and produce approximately evenly distributed hash codes. The DDTL requires no discretisation during learning and produces hash codes that enable better discrimination between data in the same class when compared to previous methods, while remaining robust to real-world augmentations in the data space. In the scenario where we require a neural network to partition the data into clusters that correspond well with ground-truth labels, a proxy objective is needed to define how these clusters are formed. One such proxy objectives involves maximising the mutual information between cluster assignments made by a neural network from multiple views. In this context views are different augmentations of the same image and the cluster assignments are the representations computed by a neural network. I demonstrate that this proxy objective produces parameters for the neural network that are sub-optimal in that a better set of parameters can be found using the same objective and a different training method. I introduce deep hierarchical object grouping (DHOG) as a method to learn a hierarchy (in the sense of easy-to-hard orderings, not structure) of solutions to the proxy objective and show how this improves performance on the downstream task. When there are features in the training data from which it is easier to compute class predictions (e.g., background colour), when compared to features for which it is relatively more difficult to compute class predictions (e.g., digit type), standard classification objectives (e.g., cross-entropy) fail to produce robust classifiers. The problem is that if a model learns to rely on `easy' features it will also ignore `complex' features (easy versus complex are purely relative in this case). I introduce latent adversarial debiasing (LAD) to decouple easy features from the class labels by first modelling the underlying structure of the training data as a latent representation using a vector-quantised variational autoencoder, and then I use a gradient-based procedure to adjust the features in this representation to confuse the predictions of a constrained classifier trained to predict class labels from the same representation. The adjusted representations of the data are then decoded to produce an augmented training dataset that can be used for training in a standard manner. I show in the aforementioned scenarios that proxy objectives can fail and demonstrate that alternative approaches can mitigate against the associated failures. I suggest an analytic approach to understanding the limits of proxy objectives for every use case in order to make the adjustments to the data or the objectives and ensure good performance on downstream tasks.