Towards More Efficient and Accurate Deep Learning Models for Segmentation, Classification, and Tracking

语义分割和分类任务已成为几个基于视觉智能（VI）的应用程序的组成部分，如机器人导航，计算机辅助诊断，视频监控和自动生产监控。这些应用程序的鲁棒性和性能在很大程度上取决于上述低层次的视觉感知：密集标记任务，分配一个标签到图像中的每个像素，和全局标记任务，分配一个标签到整个图像。尽管随着深度学习（DL）的发展，人们已经做出了巨大的努力来改进执行这些任务的模型，但不幸的是，最好的模型是计算和内存密集型的。例如，RefineNet [1]架构包含超过8500万个参数，在4字节系统中需要至少340 MB的内存。因此，这些模型需要专家标记的大数据和高性能计算（HPC）设备，如专用图形处理单元（GPU）或张量处理单元（TPU）。此外，大多数现有算法在独立的学习管道中处理分割和分类任务。这些限制阻碍了模型在资源有限的环境（低计算能力平台或数据稀缺的条件）和具有严格推理延迟的安全关键型应用程序中的实际采用。因此，拟议的研究计划是及时的，其长期目标是开发创新和具有成本效益的DL模型，可以联合分割和分类视觉表示，包括图像和视频，以实现准确的实时感知。因此，提出的模型应该保持经济的内存使用、高推理速度和出色的性能，特别是在资源有限的平台上，以便出于实用目的进行部署。因此，这将是利用谱域分析、图网络理论和深度学习优势的尖端算法的研究密集型开发。根据长期目标，研究计划分为三个不同的主题：1。共同开发先进的深度学习模型，以实现更准确的图像分割和分类。2.构建创新的同步视频对象分割和跟踪。3.提出有效的DL模型优化策略。这些目标具有多方面的挑战，需要具有丰富背景知识、专业知识和编码技能的HQP为计划做出贡献并改进现有方法。因此，该计划的一个重点是培养下一代研究人员，所要求的资金将支持1个博士学位，4个硕士学位，和5名本科生在未来5年。HQP将在图像处理、计算机视觉和深度学习等快速增长的领域发展宝贵的能力，这些领域是工业界和学术界都非常需要的。因此，通过该计划培训的HQP将对加拿大的研究和发展产生重大影响。

Semantic segmentation and classification tasks have become an integral part of several vision intelligence (VI) based applications, like robot navigation, computer-aided diagnosis, video surveillance, and automatic production monitoring. Robustness and performance of these applications largely depend on the aforesaid low-level visual perceptions: the dense labeling task that assigns a label to each pixel in the image, and the global labeling task that assigns a single label to the whole image. Although enormous efforts have been undertaken with the advancement of deep learning (DL) to improve the models performing these tasks, unfortunately, the best models are compute and memory intensive. For instance, the RefineNet [1] architecture contains more than 85 million parameters, needing a minimum of 340 MB of memory in a 4-byte system. Hence, such models require expert labeled big data and high-performance computing (HPC) devices, like specialized graphic processing units (GPUs) or tensor processing units (TPUs). Additionally, most of the existing algorithms handle the segmentation and classification tasks in independent learning pipelines. These limitations hinder the practical adoption of the models in resource-limited environments (low-compute capacity platforms or a condition of data scarcity) and safety-critical applications with strict inference latency. Thus, the proposed research program is timely, and its long-term goal is to develop innovative and cost-effective DL models that can jointly segment and classify visual representations, including images and videos, for accurate real-time perception. Hence, the proposed models should maintain economic memory usage, high inference speed, and remarkable performance, especially on resource-limited platforms, to be deployed for practical purposes. Thus, it will be a research-intensive development of cutting-edge algorithms leveraging the strength of spectral-domain analysis, graph network theories, and DL. In line with the long-term goal, the research program is divided into three distinct themes: 1. Developing advanced DL models for more accurate image segmentation and classification, jointly. 2. Building innovative simultaneous video object segmentation and tracking. 3. Proposing efficient strategies for DL model optimization. These objectives have multifaceted challenges and require HQP with significant background knowledge, expertise, and coding skills to contribute to the program and improve upon the existing methods. Thus, a key focus of this program is training the next generation of researchers, and the funding requested will support 1 Ph.D., 4 MASc., and 5 undergraduate students over the next 5 years. The HQP will develop valuable competencies in the rapidly growing areas of image processing, computer vision, and deep learning that are in high demand by both the industry and academia. Thus, the HQP trained through this program will impact significantly Canadian research and development.