Understanding, Predicting and Controlling AI Hallucination in Diffusion Models for Image Inverse Problems

理解、预测和控制图像逆问题扩散模型中的 AI 幻觉

• 批准号: 2906295
• 金额: --
• 依托单位: Imperial College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2906295
• 关键词: Understanding; Predicting; Controlling; AI; Hallucination

The problem of AI hallucination has been observed in a range of generative deep learning models. For this research we consider hallucination where the model outputs are realistic or plausible but may be factually incorrect or inconsistent. For example, a language model may generate untrue facts, or an image restoration model could produce an image which is semantically different from the ground truth image. This research focusses on hallucination in image restoration/inverse problems. The objective of image restoration is to recover a high-quality image from an input image with added noise, blur, or other degradation. In particular, we consider diffusion model-based methods. Diffusion models are a class of generative deep learning models which iteratively add noise to a signal and learn the reverse denoising process. Diffusion models have attained state-of-the-art performance in image generation tasks and have demonstrated ability to learn expressive prior distributions over image domains. Using various conditioning methods, the generation process can be guided by the degraded input image.While these models produce highly realistic images, hallucinated images occur frequently where the input is significantly degraded. This phenomenon is not observed for classical (non deep learning) algorithms, where a poor restoration may contain non-natural artifacts or residual distortions but maintain semantic consistency. While hallucination is generally undesirable for image restoration, it may be advantageous or even necessary for creative applications.Current research into AI hallucination is limited, particularly for the image domain. Our research aims are twofold: firstly, we aim to investigate the source of hallucination in diffusion models. We hypothesise that the generation process may be overly influenced by a training image which is similar to the input, leading to semantic elements being duplicated in the output. Research into the privacy of diffusion models has shown that the model memorises and can reproduce some training data images if given appropriate inputs. Another contributing factor could be that current methods of conditioning the generation process may not effectively establish the semantic contents of the input in the iterative generation process.Secondly, with an understanding of the causes of hallucination, we aim to design systems which can detect when hallucination may be occurring, which allows potentially unreliable results to be identified. The user could be provided an estimated probability that the result is hallucinated, or a "hallucination map" indicating regions of the image which are likely to contain hallucinated content. Methods of using this system during the generation process to either reduce or enhance hallucination effects will be explored. Initially we plan to conduct experiments with pre-trained diffusion models, focussing on the conditioning method and iterative sampling process. A variety of image domains and datasets covering faces and natural images will be considered. Our investigations of the source of AI hallucination in diffusion models could provide deeper insight into the information diffusion models learn and how image semantics and details are generated during inference. It is hoped that better understanding and control of hallucination would enable the use of generative deep learning-based methods with an indication of confidence in the results. This could hold particular benefit for applications in medical image processing or other scientific imaging applications, where accurate and reliable solutions are vital.

AI幻觉的问题已经在一系列生成式深度学习模型中被观察到。在这项研究中，我们考虑幻觉，其中模型输出是现实的或合理的，但可能是事实上不正确或不一致的。例如，语言模型可以生成不真实的事实，或者图像恢复模型可以产生在语义上不同于地面真实图像的图像。本研究的重点是图像恢复/逆问题中的幻觉。图像恢复的目的是从具有添加的噪声、模糊或其他退化的输入图像恢复高质量的图像。特别是，我们考虑基于扩散模型的方法。扩散模型是一类生成式深度学习模型，它迭代地将噪声添加到信号中，并学习逆去噪过程。扩散模型在图像生成任务中已经达到了最先进的性能，并且已经证明了在图像域上学习表达性先验分布的能力。使用各种调节方法，生成过程可以由退化的输入图像来指导。虽然这些模型生成高度逼真的图像，但在输入严重退化的情况下，经常会出现幻觉图像。这种现象在经典（非深度学习）算法中没有观察到，其中不良恢复可能包含非自然伪影或残余失真，但保持语义一致性。虽然幻觉通常不适合图像恢复，但它可能对创造性应用有利，甚至是必要的。目前对AI幻觉的研究有限，特别是在图像领域。我们的研究目标有两个：第一，我们的目标是调查的幻觉扩散模型的来源。我们假设生成过程可能会受到与输入相似的训练图像的过度影响，导致输出中的语义元素重复。对扩散模型隐私的研究表明，如果给予适当的输入，该模型可以记忆并复制一些训练数据图像。另一个影响因素可能是，目前的方法调节的生成过程中可能无法有效地建立在迭代生成process. Second输入的语义内容，与幻觉的原因的理解，我们的目标是设计系统，可以检测幻觉可能发生时，这使得潜在的不可靠的结果被识别。可以向用户提供结果是超分辨率的估计概率，或者指示可能包含超分辨率内容的图像区域的“超分辨率图”。将探索在生成过程中使用该系统以减少或增强幻觉效果的方法。最初，我们计划使用预先训练的扩散模型进行实验，重点关注条件反射方法和迭代采样过程。将考虑各种图像域和数据集，包括人脸和自然图像。我们对扩散模型中AI幻觉来源的研究可以更深入地了解信息扩散模型的学习以及在推理过程中如何生成图像语义和细节。希望更好地理解和控制幻觉将使基于生成深度学习的方法能够得到使用，并对结果有信心。这对于医学图像处理或其他科学成像应用中的应用特别有利，其中准确和可靠的解决方案至关重要。