Semi-supervised learning of deep hierarchical hidden representations

深层层次隐藏表示的半监督学习

• 批准号: 1793885
• 金额: --
• 依托单位: University of Bristol
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2016
• 资助国家: 英国
• 起止时间: 2016 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-1793885
• 关键词: Semi; supervised; learning; deep; hierarchical

Until the end of the 20th century, most of the computer programs were manuallyimplemented to perform repetitive tasks that could be automated, thus alleviating humanwork. However, at the end of the century, the field of Machine Learning emerged in order tocreate algorithms that could generate programs automatically by means of data andexamples. These methods together with an exponential growth of available data andcomputational power allowed the training deep hierarchical models. Nowadays, deephierarchical models are achieving and occasionally surpassing human performance on avariety of tasks like object recognition, automatic translation, speech recognition,autonomous transportation and medical applications.One of the main problems of the current state-of-the-art models is that they need fullyannotated data to solve any specific task. This type of problems is known as SupervisedLearning tasks. For this reason, one of the bottlenecks for training these models is thegeneration of good and large datasets, as they require lots of manual annotation.To solve this problem, the field of Semi-Supervised learning uses data that has not beenannotated in order to help the Supervised Learning part. For example, in problems wherelabels are scarce, it is possible to use unlabeled data to learn hierarchical hiddenrepresentations that can be used to improve the performance of Supervised models. Newmethods are still being investigated and this is one of the main topics of this Ph.D. Anotherproblem is that most of the current literature in Machine Learning assumes that dataavailable during the training of the models follows the same distribution as the future dataavailable during the deployment time. However, this assumption is only true in a fewcontrolled scenarios; for example in a closed factory. On the contrary, most of the real casescenarios evolve and change with new objects, words or patterns. For this reason, it isimportant to provide Machine Learning models with the ability to notify when new patternsappear, thus avoiding possible mistakes.This Ph.D. proposes to address this problem by means of Semi-Supervised Learningtechniques. Using new techniques we want to enhance existent models by giving them theability to discern between known and unknown patterns. In such a way that models are ableto manifest the confidence on their predictions. This is important in order to make confidentpredictions given familiar patterns while being able to ask for further inspection otherwise.In conclusion, there is an increasing popularity of machine learning models that learn deephierarchical hidden representations of the data. These models are being applied in a widerange of problems, some of which have important implications. However, they need largeamounts of annotated data and are not able to output confidence values in their predictions.For that reason, the topic of this Ph.D. is to improve models using unlabeled data, makethem aware of new situations, and able to avoid uninformed decisions.

直到世纪末，大多数计算机程序都是手工实现的，用于执行可以自动化的重复性任务，从而减轻了人类的工作。然而，在世纪末，机器学习领域出现了，以创建可以通过数据和示例自动生成程序的算法。这些方法与可用数据和计算能力的指数增长一起允许训练深度层次模型。如今，深度层次模型在物体识别、自动翻译、语音识别、自主交通和医疗应用等各种任务上的表现已经达到甚至超过人类。当前最先进的模型的主要问题之一是，它们需要完全注释的数据来解决任何特定的任务。这种类型的问题被称为监督学习任务。因此，训练这些模型的瓶颈之一是生成好的和大的数据集，因为它们需要大量的手动注释。为了解决这个问题，半监督学习领域使用未被注释的数据来帮助监督学习部分。例如，在标签稀缺的问题中，可以使用未标记的数据来学习可用于提高监督模型性能的分层hiddenrepresentation。新的方法仍在研究中，这是这个博士的主要课题之一。另一个问题是，目前机器学习领域的大多数文献都假设模型训练期间可用的数据与部署期间可用的未来数据遵循相同的分布。然而，这种假设仅在少数受控场景中成立;例如在封闭的工厂中。相反，大多数真实的案例都随着新的对象、词语或模式而演变和变化。因此，重要的是为机器学习模型提供新模式出现时通知的能力，从而避免可能的错误。提出通过半监督学习技术来解决这个问题。使用新技术，我们希望通过赋予现有模型识别已知和未知模式的能力来增强它们。以这样一种方式，模型能够表现出对他们预测的信心。这一点很重要，因为它可以在给定熟悉模式的情况下做出自信的预测，同时还可以要求进一步的检查。总之，机器学习模型越来越受欢迎，这些模型可以学习数据的深层隐藏表示。这些模型被应用于广泛的问题，其中一些具有重要的意义。然而，他们需要大量的注释数据，并且无法在预测中输出置信度值。是使用未标记的数据来改进模型，使它们意识到新的情况，并能够避免不知情的决策。

项目成果

Background Check: A General Technique to Build More Reliable and Versatile Classifiers

• DOI: 10.1109/icdm.2016.0150
• 发表时间: 2016-12
• 期刊: 2016 IEEE 16th International Conference on Data Mining (ICDM)
• 作者: Miquel Perello-Nieto;Telmo de Menezes e Silva Filho;Meelis Kull;Peter A. Flach

Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration

• 发表时间: 2019-09
• 期刊: ArXiv
• 作者: Meelis Kull;Miquel Perello-Nieto;Markus Kängsepp;Telmo de Menezes e Silva Filho;Hao Song;Peter A. Flach-

Recycling weak labels for multiclass classification

回收弱标签进行多类分类

• DOI: 10.1016/j.neucom.2020.03.002
• 发表时间: 2020
• 期刊: Neurocomputing
• 影响因子: 6
• 作者: Perello-Nieto M