Semi-supervised learning of deep hierarchical hidden representations

直到20世纪末，大多数计算机程序都是人工执行的，以执行可以自动执行的重复性任务，从而减轻了人工劳动。然而，在本世纪末，机器学习领域出现了，目的是创建能够通过数据和样本自动生成程序的算法。这些方法再加上可用数据和计算能力的指数增长，使得训练得到了更深层次的模型。如今，深度层次模型在目标识别、自动翻译、语音识别、自主运输和医疗应用等各种任务上都取得了甚至有时超过了人类的表现。当前最先进的模型的主要问题之一是，它们需要完整的注释数据来解决任何特定的任务。这类问题被称为监督性学习任务。因此，训练这些模型的瓶颈之一是生成良好的大型数据集，因为它们需要大量的人工标注，为了解决这一问题，半监督学习领域使用未标注的数据来帮助监督学习部分。例如，在标签稀缺的问题中，可以使用未标记的数据来学习分层隐藏表示，这些表示可以用来提高监督模型的性能。新的方法仍在研究中，这是本博士的主要主题之一。另一个问题是，当前机器学习的大多数文献都假设模型训练期间的数据可用与部署时的未来数据可用遵循相同的分布。然而，这一假设仅在少数受控情况下成立，例如在封闭的工厂中。相反，大多数真正的案例管理者都会随着新的对象、词语或模式而演变和变化。因此，为机器学习模型提供当新模式出现时发出通知的能力，从而避免可能的错误是非常重要的。本博士提出通过半监督学习技术来解决这个问题。使用新技术，我们希望通过赋予现有模型区分已知和未知模式的能力来增强它们。通过这种方式，模型能够对他们的预测表现出信心。这一点很重要，以便在给出熟悉的模式时做出可信的预测，同时能够要求进行进一步的检查。总而言之，学习数据的深度隐藏表示的机器学习模型越来越受欢迎。这些模型正在广泛地应用于各种问题，其中一些具有重要的影响。然而，他们需要大量的注释数据，并且无法在预测中输出置信值。因此，本博士的主题是使用未标记的数据来改进模型，使其了解新的情况，并能够避免做出不知情的决策。

Until the end of the 20th century, most of the computer programs were manuallyimplemented to perform repetitive tasks that could be automated, thus alleviating humanwork. However, at the end of the century, the field of Machine Learning emerged in order tocreate algorithms that could generate programs automatically by means of data andexamples. These methods together with an exponential growth of available data andcomputational power allowed the training deep hierarchical models. Nowadays, deephierarchical models are achieving and occasionally surpassing human performance on avariety of tasks like object recognition, automatic translation, speech recognition,autonomous transportation and medical applications.One of the main problems of the current state-of-the-art models is that they need fullyannotated data to solve any specific task. This type of problems is known as SupervisedLearning tasks. For this reason, one of the bottlenecks for training these models is thegeneration of good and large datasets, as they require lots of manual annotation.To solve this problem, the field of Semi-Supervised learning uses data that has not beenannotated in order to help the Supervised Learning part. For example, in problems wherelabels are scarce, it is possible to use unlabeled data to learn hierarchical hiddenrepresentations that can be used to improve the performance of Supervised models. Newmethods are still being investigated and this is one of the main topics of this Ph.D. Anotherproblem is that most of the current literature in Machine Learning assumes that dataavailable during the training of the models follows the same distribution as the future dataavailable during the deployment time. However, this assumption is only true in a fewcontrolled scenarios; for example in a closed factory. On the contrary, most of the real casescenarios evolve and change with new objects, words or patterns. For this reason, it isimportant to provide Machine Learning models with the ability to notify when new patternsappear, thus avoiding possible mistakes.This Ph.D. proposes to address this problem by means of Semi-Supervised Learningtechniques. Using new techniques we want to enhance existent models by giving them theability to discern between known and unknown patterns. In such a way that models are ableto manifest the confidence on their predictions. This is important in order to make confidentpredictions given familiar patterns while being able to ask for further inspection otherwise.In conclusion, there is an increasing popularity of machine learning models that learn deephierarchical hidden representations of the data. These models are being applied in a widerange of problems, some of which have important implications. However, they need largeamounts of annotated data and are not able to output confidence values in their predictions.For that reason, the topic of this Ph.D. is to improve models using unlabeled data, makethem aware of new situations, and able to avoid uninformed decisions.