Deep Neural Networks with Non-iterative Learning Strategies for Pattern Recognition and Data Agumentation Applicable to Computer Vision and Healthcare

在过去的几年里，深度网络蓬勃发展，包括深度自动编码器、深度卷积网络、深度递归网络、长期短期记忆、生成性对手网络(GAN)等。这些方法最近极大地推动了语言理解、机器人、游戏、医疗保健等不同领域的最新技术。然而，许多应用领域需要具有快速识别能力的机器学习算法。因此，快速学习的适用性非常高。遗憾的是，目前众所周知的深度网络采用迭代训练策略。随着网络深度的增加，深度学习者需要广泛的计算能力，而且可能需要数周时间进行培训，这使得广泛应用变得困难。因此，以下问题作为一个动机：我们能否为深度网络设计有效的学习算法，以改善性能和提高训练速度？此外，深度网络本身具有前所未有的参数，必须提供一定比例的示例。然而，目前的基于深度网络的数据增强算法，如GAN，仅适用于图像，而表格数据，如雷达、超声波传感器和生物数据，可以应用于更广泛的领域。针对上述问题，本文的目标是设计有效的数据扩充算法来生成表格数据，并将扩充后的数据重新用于深度网络训练。我的五年研究计划是设计一种适用于各种深度网络的统一的非迭代学习策略，从而使训练计算成本更低，适用于数据增强和模式识别的性能更高。具体目标如下：1)设计非迭代学习算法来训练深度网络，包括适用于计算机视觉和医疗保健的深度卷积神经网络、深度自动编码器和产生式对抗性网络，从而减少训练计算量，提高性能。2)设计适用于表格数据的创新增强算法，包括生物信号、雷达和生理信号。然后，重复使用所创建的合成表格数据来提高神经网络的识别性能。我认为，从方法论和工程应用的角度来看，实现一个完整和有效的学习系统都是一项重大成就。该方法将解决深度网络学习速度的主要瓶颈问题，在计算机视觉和医疗保健领域具有显著的应用前景。此外，这一研究计划将提供独特的机会，在加拿大人工智能行业认为需求很大的主题上培训HQP。

The past few years have witnessed a boom of deep networks, including deep auto-encoders, deep convolutional networks, deep recurrent networks, long short-term memory, generative adversarial network (GAN), etc. These methods have recently dramatically pushed forward the state of the art in diverse domains such as language understanding, robotics, game playing, healthcare, among others. However, many application areas require machine learning algorithms that have fast recognition capability. Thus, the applicability of fast learning is exceptionally high. Unfortunately, the currently well-known deep networks use iterative training strategy. With network depth increasing, deep learners require extensive computational power and can take weeks to train, making it difficult for broad applications. Thus, the following issue serves as a motivation: Can we design effective learning algorithms for deep networks in order to improve performance and increase training speed? Furthermore, the deep network itself has unprecedented parameters where a proportional number of examples must be fed. However, the recent deep network-based data augmentation algorithms such as GAN only work for images, but tabular data such as radars, ultrasonic sensors, and biological data can be applied to wide areas. Motivated by the above problems, the goal is to design effective data augmentation algorithms for tabular data generation and re-use the augmented data for deep networks training. My five-year research plan is to design a unified non-iterative learning strategy that can be used in various deep networks, resulting in lower training computational cost and higher performance applicable to data augmentation and pattern recognition. The specific objectives are as follows: 1) Designing non-iterative learning algorithms to train deep networks, including deep convolutional neural network, deep auto-encoder, and generative adversarial network applicable to computer vision and healthcare, resulting in less training computation with better performance. 2) Designing innovative augmentation algorithms that work for tabular data, including biosignal, radar, and physiological signal. Then, reusing the created synthetic tabular data to boost the recognition performance of neural networks. I believe the realization of a complete and efficient learning system is a significant achievement from both a methodological and engineering application perspective. The developed methods will solve the major bottleneck of learning speed in deep networks, which has a striking application perspective in computer vision and healthcare domains. Furthermore, this research program will provide unique opportunities to train HQPs in topics considered highly in-demand by the Canadian AI industry.