From Spectra To Colour: A Big-Data Computational Approach

感知颜色对我们的行为至关重要：它帮助我们识别物体，更快地看到它们并更好地记住它们。然而，理解光的物理性质与其颜色之间的关系是一个复杂的计算问题。表面的颜色不能仅仅根据光谱来预测。事实上，从物体反射的光取决于表面光谱反射特性和照明的光谱组成。至关重要的是，当光照变化时，物体的颜色变化并不像我们所期望的那样多，这是基于反射光的变化。这种现象，通常被称为颜色恒定性（CC），是颜色视觉的基本属性，因为它允许我们根据颜色识别物体及其属性，尽管我们在真实的生活中经历了可变的照明条件。虽然在我们的大脑中，表面的颜色在不同的照明下保持不变，但实现它的神经计算是未知的。一个好的CC模型应该预测颜色如何在照明中变化，以及我们如何感知这些变化。评估计算模型的CC的一个重要限制是缺乏自然的光谱图像与不断变化的照明。令人惊讶的是，感知往往不被考虑在内。色觉的一个基本计算挑战是高维光谱信息（每个波长一维，例如400维）被简化为三个值：我们三个光感受器的反应。这些值用于在视觉系统的不同处理阶段构建不同的颜色表示，其中一些由颜色空间正式描述。对感光器对自然物体反射光的反应的统计分析表明，一个简单的线性变换，意味着有效地将信息从眼睛传递到大脑，可以解释视网膜中如何实现颜色的一致性。非线性方法（例如深度学习）能够解释感知机制，作为对感官输入进行有效编码的结果。通过压缩信息，有可能发现代表感官输入的物理原因的潜在维度，即我们所感知的世界的属性。我们之前已经证明，通过压缩触觉感官输入，深度神经网络可以学习类似于我们皮肤上触觉受体的敏感性功能。类似的方法可以解释在人类视觉系统的不同阶段实现的颜色变换是如何相互联系的，并且由于暴露于我们环境的统计特性而演变。虽然这个想法遵循了近年来计算神经科学的一个有希望的趋势，但深度学习需要大量的光谱图像数据集。计算机图形学（CG）的发展允许在复杂场景中对光反射进行物理精确的模拟。自然主义和视觉场景的复杂性似乎对理解CC的机制至关重要，因为复杂自然场景呈现的上下文信息（例如，内反射、镜面反射高光）起着重要的作用。我们将使用CG来渲染一个大型的自然场景数据集，这些场景具有已知的光谱光源和表面反射率。因此，我们可以评估现有的CC计算模型，基于光谱地面实况和我们将在线收集和在实验室中与人类参与者的颜色恒定性的测量。在这些结果的基础上，我们将利用我们在进化计算方面的专业知识，为CC设计一个新颖、复杂但可解释的模型，可能优于现有的模型。最后，我们将使用深度学习技术将光谱数据减少到更低维的表示，以解释现有的神经生理和感知数据。

Perceiving colour is of vital importance for our behaviour: it helps us to identify objects, see them quicker and remember them better. However, understanding the relationship between the physical properties of light and its colour is a complex computational problem. The colour of a surface cannot be predicted solely based on light spectra. In fact, the light reflected off an object depends on both the surface spectral reflectance properties and the spectral composition of the illumination. Crucially, when the illumination changes, objects don't change colour as much as we would expect based on how much the reflected lights change. This phenomenon, usually referred to as Colour Constancy (CC), is a fundamental property of colour vision, as it allows us to recognise objects and their properties based on colour, despite the variable illumination conditions we experience in real life. Although in our brain there is a representation of the colour of surfaces that remains constant under different illuminations, the neural computations to realise it are unknown. A good CC model should predict how colour changes across illuminations and how we perceived these changes. A significant limit for evaluating computational models of CC is the lack of naturalistic spectral images with changing illuminations. Surprisingly, perception is not often taken into account.A fundamental computational challenge for colour vision is given by the fact that high dimensional spectral information (one dimension per wavelength, e.g. 400 dimensions) is reduced to three values: the responses of our three photoreceptors. Such values are used to build different colour representations at different stages of processing in the visual system, some of which are formally described by colour spaces. Statistical analysis of photoreceptor responses to light reflected from natural objects showed that a simple linear transformation, meant to efficiently transmit information from the eye to the brain, could explain how colour opponency is realised in the retinae.Non-linear approaches (e.g. Deep Learning) are able to explain perceptual mechanisms as a result of efficient encoding of the sensory input. By compressing information, it is possible to discover latent dimensions which represent the physical causes of the sensory input, i.e. the properties of the world that we perceive. We have previously shown that by compressing tactile sensory input a deep neural network can learn sensitivity functions similar to the ones of tactile receptors on our skin. A similar approach could explain how the colour transformation implemented at different stages in the human visual system are linked with each other and have evolved as a result of being exposed to the statistical properties of our environment. Although this idea follows a promising trend of the last years in computational neuroscience, Deep Learning requires a large dataset of spectral images.Developments in Computer Graphics (CG) allow for physically accurate simulation of light reflections in complex scenes. Naturalism and complexity of visual scenes seem crucial to understand the mechanisms of CC, because contextual information presented by complex natural scenes (e.g., inter-reflections, specular highlights) plays an important role. We will use CG to render a large dataset of naturalistic scenes with known spectral illuminants and surface reflectances. We could thus evaluate existing computational models for CC, based on the spectral ground-truth and on measures of colour constancy that we will collect online and in the laboratory with human participants. Building on these results, we will use our expertise on evolutionary computing to device a novel, complex, but interpretable model for CC, potentially outperforming existing ones. Finally, we will use Deep Learning techniques to reduce spectral data to a lower dimensional representation that explains existing neurophysiological and perceptual data.