Deep learning with limited data

数据有限的深度学习

• 批准号: RGPIN-2017-05117
• 负责人: Trappenberg, Thomas
• 金额: $ 1.89万
• 依托单位: Dalhousie University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=667039
• 关键词: Deep; learning; limited; data

Deep neural networks is a machine learning technique based on a model of neuron-like elements. The "deep" refers here to networks with many layers that are capable of representing complex functions. Such models have been proven to be extremely valuable as they can learn hierarchical data representations from examples and predict previously unseen data. Such techniques enabled much of the recent progress in computer vision and pattern recognition, and the advancements of deep neural networks was made possible by the availability of large data sets (Big Data), some refinement of algorithms, and the availability of increased computational power with specialized processors such as graphical processing units (GPUs). Companies like Google, Microsoft, Amazon and Facebook, and also numerous smaller engineering firms and market analysis firms are embracing this new technology. ******However, many applications do not have the luxury of having a large set of examples available to train complex models. In the research program proposed here I am trying to develop methods for applications with limited data, which includes a variety of situations such as small number of measurements, biased examples, or examples from peripheral data. Applications with limited data are common. For example, in the ocean science community it is common to have a fairly small coverage of measurements compared to the vastness of the oceans. My lab has started to investigate how artificial data can augment pre-training and facilitate the narrowing of the search space. For example, there exist a variety of physical models of ocean dynamics such as the exchange of CO2 with the atmosphere. While these models are generally considered to be insufficient for predictions on the required scale, it is possible that these simulations can provide data for the pre-training of deep networks.******Another area that promises a whole new level of techniques to training deep networks is the guidance of the learning process by domain experts, also called transfer learning. Domain experts include hereby either deep neural networks that have been trained on complementary or similar domains, or even human experts. While transfer learning has been considered for some time, the use of deep networks to learn appropriate communication channels for learning opened exciting new possibilities for much progress in this area. Such expert training would provide network training that parallels human learning. Such techniques could further drastically reduce the need of large training sets. ******The long-term goal of this research program is to develop specific methods and tools to both evaluate a machine-learning problem in terms of its complexity and data need, and to provide a comprehensive toolbox to apply pre-learning and expert transfer learning strategies to problem domains that suffer from limited data.

深度神经网络是一种基于类神经元模型的机器学习技术。这里的“深度”是指具有许多层的网络，这些层能够表示复杂的功能。这些模型已经被证明是非常有价值的，因为它们可以从示例中学习分层数据表示，并预测以前看不见的数据。这些技术使计算机视觉和模式识别领域的许多最新进展成为可能，而深度神经网络的进步是由于大数据集（大数据）的可用性，算法的一些改进以及图形处理单元（GPU）等专用处理器的计算能力增加。谷歌、微软、亚马逊和Facebook等公司，以及许多小型工程公司和市场分析公司都在接受这项新技术。** 然而，许多应用程序并没有足够的大量示例来训练复杂的模型。在这里提出的研究计划中，我试图为有限数据的应用开发方法，其中包括各种情况，如少量测量，有偏见的例子，或来自外围数据的例子。数据有限的应用程序很常见。例如，在海洋科学界，与浩瀚的海洋相比，测量的覆盖面很小是很常见的。我的实验室已经开始研究人工数据如何增强预训练并缩小搜索空间。例如，存在各种海洋动力学的物理模型，如二氧化碳与大气的交换。虽然这些模型通常被认为不足以进行所需规模的预测，但这些模拟可能可以为深度网络的预训练提供数据。另一个有望为训练深度网络带来全新技术水平的领域是领域专家对学习过程的指导，也称为迁移学习。因此，领域专家包括已经在互补或类似领域上训练过的深度神经网络，甚至包括人类专家。虽然迁移学习已经被考虑了一段时间，但使用深度网络来学习适当的学习沟通渠道为这一领域的许多进展开辟了令人兴奋的新可能性。这种专家培训将提供与人类学习并行的网络培训。这种技术可以进一步大幅减少对大型训练集的需求。该研究计划的长期目标是开发特定的方法和工具，以评估机器学习问题的复杂性和数据需求，并提供一个全面的工具箱，将预学习和专家迁移学习策略应用于数据有限的问题领域。