Intelligent and Efficient Transfer Learning with Applications in Edge AI and Healthcare

在过去的十年里，深度学习彻底改变了机器学习，传统的基于特征加工的计算机视觉方法已经被边缘化。然而，这种卓越的性能是以高计算和大量数据需求为代价的，这些都阻碍了深度学习在大型数据集不可用的领域的使用。虽然这些限制可以通过迁移学习部分缓解，但要使用的预训练模型的选择是不平凡的。如果没有对深度学习功能的正确理解，迁移学习通常会导致不必要的大型模型，从而阻碍部署到边缘设备，并导致昂贵的云部署，从而影响功耗。事实上，考虑到大量的预训练模型，如果相应的特征生成器可以合并到一个通用的存储库中，为看不见的任务创建准确有效的模型，这将是非常有益的。在这个意义上，数据和计算要求可以最小化。我们认为，当前机器学习模型开发的范式可以通过组合相关的特征生成器来转变，每个特征生成器都是预训练神经网络的子网络、层或过滤器。由于不存在整合任意预训练模型组件的知识和技术，因此该提案旨在研究实现这一目标的潜在科学和工程可能性。换句话说，我们的长期愿景是建立科学知识和技术诀窍，以实现一种新的网络建设模式，这种模式对组件具有选择性，并且在添加组件时保持保守。这开辟了几条研究途径，包括研究量化任务和网络相似性的方法，视觉应用中的视线跟踪，以量化网络不同组件在分析图像中感兴趣或“注意”区域时的重要性，以及进一步理解网络不同层中过滤器的交互方式。成功实现我们的目标将在若干领域带来切实的好处。一个例子是在医疗保健和其他应用的边缘AI领域，其中部署在传感器站点的网络效率至关重要。另一个重要的成果是在绿色人工智能中，减少用于通过网络运行样本的能量要求更慎重地选择网络架构。拟议研究可能影响的另一个领域是需要密集特征集的应用，例如医疗保健中的放射组学应用。

Deep learning has revolutionized machine learning in the past decade, and traditional computer vision methods based on feature crafting have been marginalized. Nevertheless, this exceptional performance comes with the price of high computation and enormous data requirements, and these hinder the use of deep learning in areas where large datasets are unavailable. Although these limitations can be partially alleviated by transfer learning, the choice of pre-trained models to be used is nontrivial. Without proper understanding of the deep learning features, transfer learning usually results in unnecessarily large models that impede deployment to edge devices and lead to expensive cloud deployments with implications for power consumption. In fact, given the myriad of pre-trained models available, it is vastly beneficial if the corresponding feature generators can be consolidated into a general repository to create accurate and efficient models for unseen tasks. In this sense, the data and computational requirements can be minimized. We think that the current paradigm of machine learning model development can be transformed by combining relevant feature generators, each being a sub-network, layer, or filters of a pre-trained neural network. As the knowledge and techniques of consolidating arbitrary pre-trained model components do not exist, this proposal aims at studying the underlying scientific and engineering possibilities to achieve this goal. In other words, Our long term vision is to build the scientific knowledge and technological know-how to enable a new paradigm of network building that is selective for components and conservative in adding them. This opens up several avenues of research, including studying methods to quantify task and network similarity, eye-gaze tracking in vision applications to quantify the importance of different components of networks in analyzing areas of interest or "attention" in images, and furthering our understanding of the way filters in different layers of networks interact. The success in implementing our goals will deliver tangible benefits in several areas. One example is in the area of edge AI for healthcare and other applications where the efficiency of networks deployed at the sensor site is paramount. Another important outcome is in green AI where reducing the energy used to run samples through a network mandates a more deliberate approach to choosing network architectures. Another area where the proposed research can impact is in applications where dense feature sets are desired, for example in radiomics applications in healthcare.