CAREER: On the Fairness of Light Transport for Unbiased Low-level Vision

光的物理特性是否对某些皮肤类型不利？人类皮肤的视觉外观是非常多样化的。解剖学上的差异不仅在亮度上，而且在油性、质地和厚度上都会影响图像的显示方式。这种解剖差异与人口统计学有关，并可能解释为什么人工智能(AI)管道对某些皮肤类型(例如，较黑的皮肤、较老的皮肤、女性与男性皮肤)不利。现有的减轻偏见的努力集中在人工智能堆栈的更高层的偏见上。相比之下，该研究奖通过探索光线如何与皮肤相互作用形成图像，自下而上地研究偏见。通过这样做，不仅可以识别成像中的基于物理的偏差，而且还可以纠正这种偏差。今天，基于物理学的偏见导致了日常成像系统的性能差距，如面部识别或医学成像。如果成功，研究奖可以让这些系统变得更加公平。该项目将研究与教育相结合，并延伸到初中生和高中生。该研究奖由三个支柱组成。第一个支柱寻求定性和数学地识别图像如何根据皮肤的变化而变化。调查小组将重点关注肤色、厚度、皱纹和油性的变化。还将与医生合作研究这些变异的人口统计学联系，使计算机图形渲染能够与真实的人类受试者数据进行交叉检查。第二个支柱旨在评估图像外观的这些变化随后如何影响下游图像处理和人工智能管道。例如，研究小组打算研究光度立体--一种广泛使用的获得3D形状的技术--如何追踪其性能偏差的来源与皮肤的变化。第三个支柱综合了这些洞察力，创建了新型的计算成像系统，可以抵抗由于皮肤变化而产生的偏见。很可能在寻求设计公平的成像系统时，整体性能可能会受到影响。如果在公平和绩效之间存在这样的帕累托折衷，调查团队将设计一种灵活的系统，可以在帕累托曲线上对多个点进行采样。总体选择--折衷点是最佳的--取决于社会和社区目标。总而言之，这一奖项希望为分析计算图像中的偏见奠定一个独特的基础。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Does the physics of light disadvantage certain skin types? The visual appearance of human skin is remarkably diverse. Anatomical variations in not just shades of lightness, but also oiliness, texture, and thickness all influence how images will appear. Such anatomical variations are linked to demographics and may explain why artificial intelligence (AI) pipelines disadvantage certain skin types (e.g., darker skin, older skin, female vs male skin). Existing efforts in mitigating bias focus on biases at higher-layers of the AI stack. In contrast, this research award studies bias from the bottom-up by probing how light interacts with skin to form images. In doing so, it is possible to not only identify, but also correct for physics-based bias in imaging. Today, physics-based bias leads to performance gaps in everyday imaging systems, such as facial identification or medical imaging. If successful, the research award can make these systems fairer. The project integrates the research with education and outreaches to middle and high school students.This research award rests on three pillars. The first pillar seeks to qualitatively and mathematically identify how images vary based on skin variations. The investigative team will focus on variations in skin tone, thickness, wrinkles, and oiliness. The demographic link to these variations will also be studied in collaboration with physicians enabling computer graphics renderings to be cross-checked with real, human subject data. The second pillar aims to assess how these variations in image appearance subsequently affect downstream image processing and AI pipelines. The investigative team intends to study, for instance, how photometric stereo- a widely used technique to obtain 3D shape - can trace its source of performance bias to variations in skin. The third pillar synthesizes these insights to create novel computational imaging systems that resist bias due to skin variations. It is quite possible that in seeking to design a fair imaging system, the overall performance may be affected. In the event there is such a Pareto tradeoff between fairness and performance, the investigative team will design a flexible system that can sample multiple points on the Pareto curve. The overall choice - of which tradeoff point is optimal - lies with societal and community goals. In summary, this award hopes to set a unique foundation for analyzing bias in computational imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.