Semi-supervised learning of deep hierarchical hidden representations

直到20世纪末，大多数计算机程序都是手动执行可以自动化的重复性任务，从而减轻了人力劳动。然而，在本世纪末，机器学习领域出现了，目的是创建可以通过数据和示例自动生成程序的算法。这些方法加上可用数据和计算能力的指数增长，使得训练深度层次模型成为可能。如今，深度层次模型在物体识别、自动翻译、语音识别、自动运输和医疗应用等各种任务上正在实现甚至超越人类的表现。当前最先进的模型的一个主要问题是，它们需要完全注释的数据来解决任何特定的任务。这种类型的问题被称为监督学习任务。由于这个原因，训练这些模型的瓶颈之一是生成好的大型数据集，因为它们需要大量的手工注释。为了解决这个问题，半监督学习领域使用没有注释的数据来帮助监督学习部分。例如，在标签稀缺的问题中，可以使用未标记的数据来学习可用于改进监督模型性能的分层隐藏表示。新方法仍在研究中，这是本博士的主要主题之一。另一个问题是，目前大多数机器学习文献都假设模型训练期间可用的数据与部署期间可用的未来数据遵循相同的分布。然而，这种假设只在少数可控的情况下成立；比如在一个封闭的工厂里。相反，大多数真实的场景会随着新的对象、单词或模式而演变和改变。因此，为机器学习模型提供新模式出现时通知的能力非常重要，从而避免可能的错误。本博士提出用半监督学习技术来解决这个问题。使用新技术，我们希望通过赋予现有模型辨别已知和未知模式的能力来增强它们。以这样一种方式，模型能够表现出对其预测的信心。这是很重要的，以便在熟悉的模式下做出自信的预测，同时能够要求进一步检查。总之，学习数据的深度层次隐藏表示的机器学习模型越来越受欢迎。这些模型被应用于各种各样的问题，其中一些具有重要意义。然而，它们需要大量带注释的数据，并且不能在预测中输出置信度值。因此，本博士的主题是改进使用未标记数据的模型，使其了解新情况，并能够避免不知情的决策。

Until the end of the 20th century, most of the computer programs were manuallyimplemented to perform repetitive tasks that could be automated, thus alleviating humanwork. However, at the end of the century, the field of Machine Learning emerged in order tocreate algorithms that could generate programs automatically by means of data andexamples. These methods together with an exponential growth of available data andcomputational power allowed the training deep hierarchical models. Nowadays, deephierarchical models are achieving and occasionally surpassing human performance on avariety of tasks like object recognition, automatic translation, speech recognition,autonomous transportation and medical applications.One of the main problems of the current state-of-the-art models is that they need fullyannotated data to solve any specific task. This type of problems is known as SupervisedLearning tasks. For this reason, one of the bottlenecks for training these models is thegeneration of good and large datasets, as they require lots of manual annotation.To solve this problem, the field of Semi-Supervised learning uses data that has not beenannotated in order to help the Supervised Learning part. For example, in problems wherelabels are scarce, it is possible to use unlabeled data to learn hierarchical hiddenrepresentations that can be used to improve the performance of Supervised models. Newmethods are still being investigated and this is one of the main topics of this Ph.D. Anotherproblem is that most of the current literature in Machine Learning assumes that dataavailable during the training of the models follows the same distribution as the future dataavailable during the deployment time. However, this assumption is only true in a fewcontrolled scenarios; for example in a closed factory. On the contrary, most of the real casescenarios evolve and change with new objects, words or patterns. For this reason, it isimportant to provide Machine Learning models with the ability to notify when new patternsappear, thus avoiding possible mistakes.This Ph.D. proposes to address this problem by means of Semi-Supervised Learningtechniques. Using new techniques we want to enhance existent models by giving them theability to discern between known and unknown patterns. In such a way that models are ableto manifest the confidence on their predictions. This is important in order to make confidentpredictions given familiar patterns while being able to ask for further inspection otherwise.In conclusion, there is an increasing popularity of machine learning models that learn deephierarchical hidden representations of the data. These models are being applied in a widerange of problems, some of which have important implications. However, they need largeamounts of annotated data and are not able to output confidence values in their predictions.For that reason, the topic of this Ph.D. is to improve models using unlabeled data, makethem aware of new situations, and able to avoid uninformed decisions.