Understanding, Predicting and Controlling AI Hallucination in Diffusion Models for Image Inverse Problems

人工智能幻觉的问题已经在一系列生成式深度学习模型中被观察到。对于这项研究，我们考虑幻觉，其中模型输出是现实的或似是而非的，但实际上可能不正确或不一致。例如，语言模型可能会生成不真实的事实，或者图像恢复模型可能会生成语义上不同于基础真实图像的图像。本课题主要研究幻觉图像恢复/反演问题。图像恢复的目的是从添加了噪声、模糊或其他退化的输入图像中恢复出高质量的图像。特别是，我们考虑了基于扩散模型的方法。扩散模型是一类生成式深度学习模型，它迭代地向信号中添加噪声并学习反向去噪过程。扩散模型在图像生成任务中已经达到了最先进的性能，并且已经证明了在图像域上学习表达性先验分布的能力。利用各种调节方法，可以通过退化的输入图像来引导生成过程。虽然这些模型产生了高度逼真的图像，但在输入明显退化的情况下，经常出现幻觉图像。在经典（非深度学习）算法中没有观察到这种现象，其中较差的恢复可能包含非自然工件或残余扭曲，但保持语义一致性。虽然幻觉通常是不受欢迎的图像恢复，它可能是有利的，甚至是必要的创造性应用。目前对人工智能幻觉的研究是有限的，特别是在图像领域。我们的研究目标有两个：首先，我们的目标是研究扩散模型中幻觉的来源。我们假设生成过程可能受到与输入相似的训练图像的过度影响，导致语义元素在输出中重复。对扩散模型隐私性的研究表明，如果给予适当的输入，该模型可以记忆并再现一些训练数据图像。另一个影响因素可能是，当前调节生成过程的方法可能无法有效地在迭代生成过程中建立输入的语义内容。其次，随着对幻觉原因的理解，我们的目标是设计能够检测幻觉何时可能发生的系统，这允许识别潜在的不可靠结果。可以向用户提供结果是幻觉的估计概率，或者向用户提供指示可能包含幻觉内容的图像区域的“幻觉地图”。在生成过程中使用该系统来减少或增强幻觉效果的方法将被探索。首先，我们计划使用预训练的扩散模型进行实验，重点研究条件调节方法和迭代采样过程。涵盖人脸和自然图像的各种图像域和数据集将被考虑。我们对扩散模型中人工智能幻觉来源的研究可以更深入地了解信息扩散模型的学习过程以及在推理过程中如何生成图像语义和细节。希望更好地理解和控制幻觉将使基于生成式深度学习的方法的使用具有对结果的信心。这对于医学图像处理或其他科学成像应用的应用尤其有利，在这些应用中，准确可靠的解决方案至关重要。

The problem of AI hallucination has been observed in a range of generative deep learning models. For this research we consider hallucination where the model outputs are realistic or plausible but may be factually incorrect or inconsistent. For example, a language model may generate untrue facts, or an image restoration model could produce an image which is semantically different from the ground truth image. This research focusses on hallucination in image restoration/inverse problems. The objective of image restoration is to recover a high-quality image from an input image with added noise, blur, or other degradation. In particular, we consider diffusion model-based methods. Diffusion models are a class of generative deep learning models which iteratively add noise to a signal and learn the reverse denoising process. Diffusion models have attained state-of-the-art performance in image generation tasks and have demonstrated ability to learn expressive prior distributions over image domains. Using various conditioning methods, the generation process can be guided by the degraded input image.While these models produce highly realistic images, hallucinated images occur frequently where the input is significantly degraded. This phenomenon is not observed for classical (non deep learning) algorithms, where a poor restoration may contain non-natural artifacts or residual distortions but maintain semantic consistency. While hallucination is generally undesirable for image restoration, it may be advantageous or even necessary for creative applications.Current research into AI hallucination is limited, particularly for the image domain. Our research aims are twofold: firstly, we aim to investigate the source of hallucination in diffusion models. We hypothesise that the generation process may be overly influenced by a training image which is similar to the input, leading to semantic elements being duplicated in the output. Research into the privacy of diffusion models has shown that the model memorises and can reproduce some training data images if given appropriate inputs. Another contributing factor could be that current methods of conditioning the generation process may not effectively establish the semantic contents of the input in the iterative generation process.Secondly, with an understanding of the causes of hallucination, we aim to design systems which can detect when hallucination may be occurring, which allows potentially unreliable results to be identified. The user could be provided an estimated probability that the result is hallucinated, or a "hallucination map" indicating regions of the image which are likely to contain hallucinated content. Methods of using this system during the generation process to either reduce or enhance hallucination effects will be explored. Initially we plan to conduct experiments with pre-trained diffusion models, focussing on the conditioning method and iterative sampling process. A variety of image domains and datasets covering faces and natural images will be considered. Our investigations of the source of AI hallucination in diffusion models could provide deeper insight into the information diffusion models learn and how image semantics and details are generated during inference. It is hoped that better understanding and control of hallucination would enable the use of generative deep learning-based methods with an indication of confidence in the results. This could hold particular benefit for applications in medical image processing or other scientific imaging applications, where accurate and reliable solutions are vital.