Understanding, Predicting and Controlling AI Hallucination in Diffusion Models for Image Inverse Problems

人工智能幻觉的问题已经在一系列生成性深度学习模型中观察到。在这项研究中，我们考虑幻觉，即模型的输出是真实的或看似合理的，但可能是事实不正确或不一致的。例如，语言模型可能产生不真实的事实，或者图像恢复模型可能产生与地面真实图像在语义上不同的图像。本研究主要研究图像恢复/逆问题中的幻觉问题。图像恢复的目标是从增加了噪声、模糊或其他退化的输入图像中恢复高质量的图像。特别地，我们考虑基于扩散模型的方法。扩散模型是一类生成性深度学习模型，它迭代地向信号中添加噪声，并学习反向去噪过程。扩散模型已经在图像生成任务中获得了最先进的性能，并展示了学习图像域上富有表现力的先验分布的能力。使用不同的条件化方法，生成过程可以由退化的输入图像来指导。虽然这些模型产生的图像非常逼真，但在输入严重退化的地方经常出现幻觉图像。对于经典(非深度学习)算法不会观察到这种现象，在这些算法中，较差的恢复可能包含非自然伪像或残余失真，但保持语义一致性。虽然幻觉对于图像恢复通常是不可取的，但对于创造性的应用来说，它可能是有利的，甚至是必要的。目前对人工智能幻觉的研究有限，特别是在图像领域。我们的研究目的有两个：首先，我们的目标是研究扩散模型中幻觉的来源。我们假设生成过程可能受到与输入相似的训练图像的过度影响，导致语义元素在输出中重复。对扩散模型隐私性的研究表明，如果给出适当的输入，该模型可以记忆并重现一些训练数据图像。另一个因素可能是，目前限制产生过程的方法可能不能有效地在迭代产生过程中建立输入的语义内容。第二，在理解了幻觉的原因后，我们的目标是设计能够检测何时可能发生幻觉的系统，这允许识别潜在的不可靠的结果。可以向用户提供结果是幻觉的估计概率，或指示图像中可能包含幻觉内容的区域的“幻觉地图”。将探索在产生过程中使用该系统以减少或增强幻觉效果的方法。首先，我们计划用预先训练好的扩散模型进行实验，重点是条件化方法和迭代抽样过程。将考虑涵盖人脸和自然图像的各种图像域和数据集。我们对扩散模型中人工智能幻觉来源的研究可以更深入地了解扩散模型学习的信息，以及图像语义和细节是如何在推理过程中产生的。希望更好地理解和控制幻觉将能够使用基于生成性深度学习的方法，并表明对结果的信心。这可以为医学图像处理或其他科学成像应用程序带来特别的好处，在这些应用程序中，准确和可靠的解决方案至关重要。

The problem of AI hallucination has been observed in a range of generative deep learning models. For this research we consider hallucination where the model outputs are realistic or plausible but may be factually incorrect or inconsistent. For example, a language model may generate untrue facts, or an image restoration model could produce an image which is semantically different from the ground truth image. This research focusses on hallucination in image restoration/inverse problems. The objective of image restoration is to recover a high-quality image from an input image with added noise, blur, or other degradation. In particular, we consider diffusion model-based methods. Diffusion models are a class of generative deep learning models which iteratively add noise to a signal and learn the reverse denoising process. Diffusion models have attained state-of-the-art performance in image generation tasks and have demonstrated ability to learn expressive prior distributions over image domains. Using various conditioning methods, the generation process can be guided by the degraded input image.While these models produce highly realistic images, hallucinated images occur frequently where the input is significantly degraded. This phenomenon is not observed for classical (non deep learning) algorithms, where a poor restoration may contain non-natural artifacts or residual distortions but maintain semantic consistency. While hallucination is generally undesirable for image restoration, it may be advantageous or even necessary for creative applications.Current research into AI hallucination is limited, particularly for the image domain. Our research aims are twofold: firstly, we aim to investigate the source of hallucination in diffusion models. We hypothesise that the generation process may be overly influenced by a training image which is similar to the input, leading to semantic elements being duplicated in the output. Research into the privacy of diffusion models has shown that the model memorises and can reproduce some training data images if given appropriate inputs. Another contributing factor could be that current methods of conditioning the generation process may not effectively establish the semantic contents of the input in the iterative generation process.Secondly, with an understanding of the causes of hallucination, we aim to design systems which can detect when hallucination may be occurring, which allows potentially unreliable results to be identified. The user could be provided an estimated probability that the result is hallucinated, or a "hallucination map" indicating regions of the image which are likely to contain hallucinated content. Methods of using this system during the generation process to either reduce or enhance hallucination effects will be explored. Initially we plan to conduct experiments with pre-trained diffusion models, focussing on the conditioning method and iterative sampling process. A variety of image domains and datasets covering faces and natural images will be considered. Our investigations of the source of AI hallucination in diffusion models could provide deeper insight into the information diffusion models learn and how image semantics and details are generated during inference. It is hoped that better understanding and control of hallucination would enable the use of generative deep learning-based methods with an indication of confidence in the results. This could hold particular benefit for applications in medical image processing or other scientific imaging applications, where accurate and reliable solutions are vital.