Towards More Efficient and Accurate Deep Learning Models for Segmentation, Classification, and Tracking

语义分割和分类任务已经成为一些基于视觉智能（VI）的应用的组成部分，如机器人导航、计算机辅助诊断、视频监控和自动生产监控。这些应用程序的鲁棒性和性能在很大程度上取决于上述低级视觉感知：为图像中的每个像素分配标签的密集标记任务和为整个图像分配单个标签的全局标记任务。尽管深度学习（DL）的进步已经付出了巨大的努力来改进执行这些任务的模型，但不幸的是，最好的模型是计算和内存密集型的。例如，RefineNet[1]架构包含超过8500万个参数，在一个4字节的系统中至少需要340 MB的内存。因此，这样的模型需要专家标记的大数据和高性能计算（HPC）设备，如专用图形处理单元（gpu）或张量处理单元（tpu）。此外，大多数现有算法在独立的学习管道中处理分割和分类任务。这些限制阻碍了在资源有限的环境（低计算能力平台或数据稀缺的条件）和具有严格推理延迟的安全关键型应用程序中实际采用模型。因此，提出的研究计划是及时的，其长期目标是开发创新和具有成本效益的深度学习模型，这些模型可以共同分割和分类视觉表示，包括图像和视频，以实现准确的实时感知。因此，建议的模型应该保持经济的内存使用、高推理速度和卓越的性能，特别是在资源有限的平台上，以便为实际目的部署。因此，它将是利用谱域分析、图网络理论和深度学习的优势的尖端算法的研究密集型开发。根据长期目标，研究计划分为三个不同的主题：1。共同开发先进的深度学习模型，以实现更准确的图像分割和分类。2. 建立创新的同步视频对象分割和跟踪。3. 提出了有效的深度学习模型优化策略。这些目标具有多方面的挑战，要求HQP具有重要的背景知识、专业知识和编码技能，为项目做出贡献，并改进现有方法。因此，该项目的重点是培养下一代研究人员，所申请的资金将支持1名博士，4名硕士。，并在未来5年内招收5名本科生。HQP将在快速发展的图像处理、计算机视觉和深度学习领域培养有价值的能力，这些领域在工业界和学术界都有很高的需求。因此，通过该项目培训的HQP将对加拿大的研究和发展产生重大影响。

Semantic segmentation and classification tasks have become an integral part of several vision intelligence (VI) based applications, like robot navigation, computer-aided diagnosis, video surveillance, and automatic production monitoring. Robustness and performance of these applications largely depend on the aforesaid low-level visual perceptions: the dense labeling task that assigns a label to each pixel in the image, and the global labeling task that assigns a single label to the whole image. Although enormous efforts have been undertaken with the advancement of deep learning (DL) to improve the models performing these tasks, unfortunately, the best models are compute and memory intensive. For instance, the RefineNet [1] architecture contains more than 85 million parameters, needing a minimum of 340 MB of memory in a 4-byte system. Hence, such models require expert labeled big data and high-performance computing (HPC) devices, like specialized graphic processing units (GPUs) or tensor processing units (TPUs). Additionally, most of the existing algorithms handle the segmentation and classification tasks in independent learning pipelines. These limitations hinder the practical adoption of the models in resource-limited environments (low-compute capacity platforms or a condition of data scarcity) and safety-critical applications with strict inference latency. Thus, the proposed research program is timely, and its long-term goal is to develop innovative and cost-effective DL models that can jointly segment and classify visual representations, including images and videos, for accurate real-time perception. Hence, the proposed models should maintain economic memory usage, high inference speed, and remarkable performance, especially on resource-limited platforms, to be deployed for practical purposes. Thus, it will be a research-intensive development of cutting-edge algorithms leveraging the strength of spectral-domain analysis, graph network theories, and DL. In line with the long-term goal, the research program is divided into three distinct themes: 1. Developing advanced DL models for more accurate image segmentation and classification, jointly. 2. Building innovative simultaneous video object segmentation and tracking. 3. Proposing efficient strategies for DL model optimization. These objectives have multifaceted challenges and require HQP with significant background knowledge, expertise, and coding skills to contribute to the program and improve upon the existing methods. Thus, a key focus of this program is training the next generation of researchers, and the funding requested will support 1 Ph.D., 4 MASc., and 5 undergraduate students over the next 5 years. The HQP will develop valuable competencies in the rapidly growing areas of image processing, computer vision, and deep learning that are in high demand by both the industry and academia. Thus, the HQP trained through this program will impact significantly Canadian research and development.