From Spectra To Colour: A Big-Data Computational Approach

感知颜色对我们的行为至关重要：它帮助我们识别物体，更快地看到它们，更好地记住它们。然而，理解光的物理性质和颜色之间的关系是一个复杂的计算问题。表面的颜色不能仅仅根据光谱来预测。事实上，物体反射的光取决于表面光谱反射特性和照明的光谱组成。关键是，当光照变化时，物体的颜色变化并不像我们根据反射光的变化所期望的那样大。这种现象，通常被称为色彩恒常性（CC），是色彩视觉的基本属性，因为它允许我们根据颜色识别物体及其属性，尽管我们在现实生活中经历了可变的照明条件。虽然在我们的大脑中，表面颜色的表征在不同的光照下保持不变，但实现它的神经计算是未知的。一个好的CC模型应该预测颜色如何在不同的光照下变化，以及我们如何感知这些变化。评估CC计算模型的一个重要限制是缺乏随光照变化的自然光谱图像。令人惊讶的是，人们的感觉往往不被考虑在内。高维光谱信息（每个波长一个维度，例如400维）被简化为三个值：我们的三个光感受器的响应，这是色觉的一个基本计算挑战。这些值用于在视觉系统的不同处理阶段构建不同的颜色表示，其中一些是由颜色空间正式描述的。对光感受器对自然物体反射的光的反应的统计分析表明，一个简单的线性转换，意味着有效地将信息从眼睛传递到大脑，可以解释视网膜中颜色对抗是如何实现的。非线性方法（如深度学习）能够解释感知机制，这是对感觉输入进行有效编码的结果。通过压缩信息，有可能发现代表感官输入的物理原因的潜在维度，即我们所感知的世界的属性。我们之前已经证明，通过压缩触觉感官输入，深度神经网络可以学习类似于我们皮肤上的触觉感受器的敏感功能。类似的方法可以解释在人类视觉系统的不同阶段实现的颜色转换是如何相互联系的，并且由于暴露于我们环境的统计特性而进化。虽然这个想法遵循了过去几年计算神经科学的一个有希望的趋势，但深度学习需要一个大的光谱图像数据集。计算机图形学（CG）的发展允许在复杂场景中对光反射进行物理精确模拟。自然主义和视觉场景的复杂性似乎对理解CC机制至关重要，因为复杂自然场景所呈现的上下文信息（例如，相互反射，高光反射）起着重要作用。我们将使用CG渲染具有已知光谱光源和表面反射率的自然场景的大型数据集。因此，我们可以评估现有的CC计算模型，该模型基于光谱基础真值和我们将在网上和实验室中与人类参与者一起收集的颜色恒常性测量值。在这些结果的基础上，我们将利用我们在进化计算方面的专业知识，为CC设计一个新颖、复杂但可解释的模型，其性能可能优于现有的模型。最后，我们将使用深度学习技术将光谱数据简化为较低维度的表示，以解释现有的神经生理和感知数据。

Perceiving colour is of vital importance for our behaviour: it helps us to identify objects, see them quicker and remember them better. However, understanding the relationship between the physical properties of light and its colour is a complex computational problem. The colour of a surface cannot be predicted solely based on light spectra. In fact, the light reflected off an object depends on both the surface spectral reflectance properties and the spectral composition of the illumination. Crucially, when the illumination changes, objects don't change colour as much as we would expect based on how much the reflected lights change. This phenomenon, usually referred to as Colour Constancy (CC), is a fundamental property of colour vision, as it allows us to recognise objects and their properties based on colour, despite the variable illumination conditions we experience in real life. Although in our brain there is a representation of the colour of surfaces that remains constant under different illuminations, the neural computations to realise it are unknown. A good CC model should predict how colour changes across illuminations and how we perceived these changes. A significant limit for evaluating computational models of CC is the lack of naturalistic spectral images with changing illuminations. Surprisingly, perception is not often taken into account.A fundamental computational challenge for colour vision is given by the fact that high dimensional spectral information (one dimension per wavelength, e.g. 400 dimensions) is reduced to three values: the responses of our three photoreceptors. Such values are used to build different colour representations at different stages of processing in the visual system, some of which are formally described by colour spaces. Statistical analysis of photoreceptor responses to light reflected from natural objects showed that a simple linear transformation, meant to efficiently transmit information from the eye to the brain, could explain how colour opponency is realised in the retinae.Non-linear approaches (e.g. Deep Learning) are able to explain perceptual mechanisms as a result of efficient encoding of the sensory input. By compressing information, it is possible to discover latent dimensions which represent the physical causes of the sensory input, i.e. the properties of the world that we perceive. We have previously shown that by compressing tactile sensory input a deep neural network can learn sensitivity functions similar to the ones of tactile receptors on our skin. A similar approach could explain how the colour transformation implemented at different stages in the human visual system are linked with each other and have evolved as a result of being exposed to the statistical properties of our environment. Although this idea follows a promising trend of the last years in computational neuroscience, Deep Learning requires a large dataset of spectral images.Developments in Computer Graphics (CG) allow for physically accurate simulation of light reflections in complex scenes. Naturalism and complexity of visual scenes seem crucial to understand the mechanisms of CC, because contextual information presented by complex natural scenes (e.g., inter-reflections, specular highlights) plays an important role. We will use CG to render a large dataset of naturalistic scenes with known spectral illuminants and surface reflectances. We could thus evaluate existing computational models for CC, based on the spectral ground-truth and on measures of colour constancy that we will collect online and in the laboratory with human participants. Building on these results, we will use our expertise on evolutionary computing to device a novel, complex, but interpretable model for CC, potentially outperforming existing ones. Finally, we will use Deep Learning techniques to reduce spectral data to a lower dimensional representation that explains existing neurophysiological and perceptual data.