Deep Neural Networks with Non-iterative Learning Strategies for Pattern Recognition and Data Agumentation Applicable to Computer Vision and Healthcare

在过去的几年里，深度网络蓬勃发展，包括深度自动编码器、深度卷积网络、深度递归网络、长短期记忆、生成对抗网络（GAN）等，这些方法最近极大地推动了语言理解、机器人、游戏、医疗保健等不同领域的技术发展。 然而，许多应用领域需要具有快速识别能力的机器学习算法。因此，快速学习的适用性非常高。不幸的是，目前众所周知的深度网络使用迭代训练策略。随着网络深度的增加，深度学习器需要大量的计算能力，并且可能需要数周的时间来训练，这使得它难以广泛应用。因此，以下问题可以作为动机：我们能否为深度网络设计有效的学习算法，以提高性能并提高训练速度？ 此外，深度网络本身具有前所未有的参数，必须提供一定比例的示例。然而，最近基于深度网络的数据增强算法（如GAN）仅适用于图像，但雷达，超声波传感器和生物数据等表格数据可以应用于广泛的领域。受上述问题的启发，我们的目标是设计有效的数据增强算法来生成表格数据，并将增强后的数据重新用于深度网络训练。 我的五年研究计划是设计一种统一的非迭代学习策略，可用于各种深度网络，从而降低训练计算成本，提高适用于数据增强和模式识别的性能。具体目标如下： 1)设计非迭代学习算法来训练深度网络，包括适用于计算机视觉和医疗保健的深度卷积神经网络，深度自动编码器和生成对抗网络，从而减少训练计算，提高性能。 2)设计适用于表格数据的创新增强算法，包括生物信号、雷达和生理信号。然后，重用创建的合成表格数据，以提高神经网络的识别性能。 我认为，从方法论和工程应用的角度来看，实现一个完整而有效的学习系统是一个重大成就。所开发的方法将解决深度网络中学习速度的主要瓶颈，在计算机视觉和医疗保健领域具有惊人的应用前景。此外，该研究计划将提供独特的机会，在加拿大人工智能行业高度需求的主题中培训HQP。

The past few years have witnessed a boom of deep networks, including deep auto-encoders, deep convolutional networks, deep recurrent networks, long short-term memory, generative adversarial network (GAN), etc. These methods have recently dramatically pushed forward the state of the art in diverse domains such as language understanding, robotics, game playing, healthcare, among others. However, many application areas require machine learning algorithms that have fast recognition capability. Thus, the applicability of fast learning is exceptionally high. Unfortunately, the currently well-known deep networks use iterative training strategy. With network depth increasing, deep learners require extensive computational power and can take weeks to train, making it difficult for broad applications. Thus, the following issue serves as a motivation: Can we design effective learning algorithms for deep networks in order to improve performance and increase training speed? Furthermore, the deep network itself has unprecedented parameters where a proportional number of examples must be fed. However, the recent deep network-based data augmentation algorithms such as GAN only work for images, but tabular data such as radars, ultrasonic sensors, and biological data can be applied to wide areas. Motivated by the above problems, the goal is to design effective data augmentation algorithms for tabular data generation and re-use the augmented data for deep networks training. My five-year research plan is to design a unified non-iterative learning strategy that can be used in various deep networks, resulting in lower training computational cost and higher performance applicable to data augmentation and pattern recognition. The specific objectives are as follows: 1) Designing non-iterative learning algorithms to train deep networks, including deep convolutional neural network, deep auto-encoder, and generative adversarial network applicable to computer vision and healthcare, resulting in less training computation with better performance. 2) Designing innovative augmentation algorithms that work for tabular data, including biosignal, radar, and physiological signal. Then, reusing the created synthetic tabular data to boost the recognition performance of neural networks. I believe the realization of a complete and efficient learning system is a significant achievement from both a methodological and engineering application perspective. The developed methods will solve the major bottleneck of learning speed in deep networks, which has a striking application perspective in computer vision and healthcare domains. Furthermore, this research program will provide unique opportunities to train HQPs in topics considered highly in-demand by the Canadian AI industry.