Interpretable and Robust L2S - Optimization

可解释且稳健的 L2S - 优化

• 批准号: 498557872
• 负责人: Professorin Dr. Margret Keuper
• 金额: --
• 依托单位: Professur für Maschinelles Lernen
• 依托单位国家: 德国
• 项目类别: Research Units
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/498557872?language=en
• 关键词: Interpretable; Robust; L2S; Optimization

Driven by the recent successes in machine learning, the goal of L2S is an end-to-end learning pipeline of image sensing and analysis systems which allows a downstream application-driven sensor design.Yet, while an end-to-end optimization is promising from a machine learning perspective, the complex interplay between input sensor data, neural network architecture and learned network weights amplifies several research questions that are to a certain extent, already prevalent in classical deep learning.First, it is to be expected that currently well performing architectures are somewhat tailored to the conventional RGB sensor design. Therefore, we argue that the sensor optimization should be accompanied by an optimization of the neural architecture along with the network weights. We will therefore address both, sensor and architecture optimization, in a joint discrete optimization problem. Bayesian Optimization over a joint architecture and sensor parameter search space can be employed to find optima in an efficient way. We will thereby explore different sensor layouts. For efficiency, we will consider architecture and sensor embeddings computed using graph neural networks.Second, the optimization with respect to improved accuracy on a downstream task such as image classification, semantic segmentation or optical flow computation, might yield architecture/sensor pairs that perform very well on the data available at training time. Yet, the robustness of such models even under slight domain shifts is expected to be more brittle than in conventional learning driven approaches, where the sensor is not subject to the end-to-end optimization. Therefore, we will investigate methods to ensure a certain robustness of the resulting model/sensor pairs using for example regularization and data augmentation techniques.Third, while neural networks often issue highly accurate predictions, their interpretability is usually low and one has very limited understanding of the prediction uncertainty. Both issues are reinforced when addressing a joint sensor/machine learning optimization. On the one hand, we will investigate how existing methods for network visualization, that help to make the decision process more interpretable, can be transferred to optimized sensors that do not necessarily output image-like data. On the other hand, we will analyze how locally optimized sampling on the sensor side interferes with decision uncertainties, measured for example using Monte-Carlo drop-out. All questions will be considered in the context of sensor modalities ranging from optimized RGB sensors over terahertz measurements to lightfield microscopic recordings and with respect to applications that require varying localization precision, such as classification (no localization), segmentation (pixel accurate localization) or optical flow estimation (pixel accurate localizaiton, sub-pixel accurate mapping).

在最近机器学习成功的推动下，L2S的目标是一个端到端的图像传感和分析系统学习管道，允许下游应用驱动的传感器设计。然而，虽然从机器学习的角度来看，端到端优化很有希望，但输入传感器数据、神经网络架构和学习网络权重之间复杂的相互作用放大了经典深度学习中已经普遍存在的几个研究问题。首先，可以预期的是，目前性能良好的架构在某种程度上是为传统的RGB传感器设计量身定制的。因此，我们认为传感器优化应该伴随着神经结构的优化以及网络权重的优化。因此，我们将在联合离散优化问题中同时解决传感器和架构优化问题。利用联合结构和传感器参数搜索空间上的贝叶斯优化可以有效地找到最优点。因此，我们将探索不同的传感器布局。为了提高效率，我们将考虑使用图神经网络计算的架构和传感器嵌入。其次，在下游任务（如图像分类、语义分割或光流计算）上提高精度的优化可能会产生在训练时可用数据上表现良好的架构/传感器对。然而，即使在轻微的领域转移下，这种模型的鲁棒性也比传统的学习驱动方法更脆弱，在传统的学习驱动方法中，传感器不受端到端优化的影响。因此，我们将研究使用正则化和数据增强技术来确保结果模型/传感器对具有一定鲁棒性的方法。第三，虽然神经网络经常发布高度准确的预测，但它们的可解释性通常很低，人们对预测不确定性的理解非常有限。在解决联合传感器/机器学习优化时，这两个问题都得到了加强。一方面，我们将研究如何将现有的网络可视化方法转移到优化的传感器上，这些传感器有助于使决策过程更具可解释性，而这些传感器不一定输出类似图像的数据。另一方面，我们将分析传感器侧的局部优化采样如何干扰决策不确定性，例如使用蒙特卡罗dropout进行测量。所有问题都将在传感器模式的背景下进行考虑，从太赫兹测量的优化RGB传感器到光场显微记录，以及需要不同定位精度的应用，例如分类（无定位），分割（像素精确定位）或光流估计（像素精确定位，亚像素精确映射）。