CAREER: Art and Vision: Scene Layout from Pictorial Cues

职业生涯：艺术与视觉：《画报》中的场景布局PI：Stella(兴兴)Yu机构：波士顿大学艺术家是视觉感知的大师。同时研究艺术和视觉可以为计算机视觉中的基本问题提供新的解决方案。我们专注于从一幅图像中推断场景布局。从人工智能研究的最早时期起，人们就一直在研究这个问题，导致了许多所谓的从X到形状的方法，其中X可以是阴影、透视等。不幸的是，这些方法都是在自己的假设下工作的，而这些假设在真实图像中往往不成立。这些线索是如何相互作用和整合的，目前仍不得而知。画家经常使用四种技术的组合：遮挡、透视、明暗处理和形状，以有效地从2D图片中唤起3D感知。研究它们的技术可以为从像素值恢复场景布局的计算提供更多的见解。PI建议将艺术家和视觉科学家聚集在一起，解决根据图片线索进行场景布局的计算问题。该项目从教育、实验和计算模型三个方面实现了这一点。波士顿学院开发了一门新的跨学科课程--艺术与视觉感知，以全面交叉检验艺术如何有助于理解视觉，以及视觉如何有助于艺术的产生和观看。学生积极参与艺术实践和视觉实验。与单独研究每一门学科相比，一起学习艺术和视觉会带来更深层次的理解。学生的作业也为视觉研究带来了有价值的数据集。在本项目中，根据图片线索进行场景布局的计算方法是在谱图理论框架下，从多个图片线索的全局整合中将像素分组到空间组织的表面。其目标是将这些线索如何相互作用的艺术渲染知识转化为计算现实。PI将使用眼睛跟踪和心理物理学实验和计算模型研究几何(遮挡和透视)、外观(亮度和颜色)和形状。这些工作被组织成两个阶段，从由平面组成的场景(房间和街道)推断空间布局到由曲面组成的场景(景观和通用场景)。智能价值视觉最引人注目的是它从一张2D图像感知3D空间布局的能力。与统计学习方法相比，分组方法不仅是通用的，因此可以很好地随场景的数量而缩放，而且可以产生场景中表面的精确组织。与传统的从X形状的方法相比，分组方法结合其他图像线索来检查每一幅图片线索。这些多个图画线索的整合使它们第一次适用于真实图像。PI发展了谱图理论中用于深度分离的基本分组机制。与大多数现有的关于这一主题的公式相比，它具有无与伦比的概念简单性、计算效率和保证的近全局最优性。关于亮度和颜色感知的拟议研究，与从阴影到形状和表面组织有关，将有助于澄清低水平和高级机制在Hering和Helmholtz之间关于颜色感知的长期科学辩论中的作用。广泛影响该项目不仅在研究中，而且在教育中，通过开发一种横跨神经科学、心理学、计算机科学和视觉艺术领域的新课程，通过让学生参与艺术实践和科学实验，并为艺术家和科学家提供一个交流视觉感知思想的论坛，来弥合艺术和科学之间的鸿沟。这些跨学科的努力符合波士顿学院的文科教育传统。该项目不仅将受益于校园内强大的美术系，还将培养计算机科学意识和接触非技术人员，并促进波士顿学院年轻的计算机科学系的成长。网址：http://www.cs.bc.edu/~syu/artvis/

CAREER: Art and Vision: Scene Layout from Pictorial CuesPI: Stella (XingXing) Yu Institution: Boston CollegeArtists are the masters of visual perception. Studying art and vision together can provide new solutions to fundamental problems in computer vision. We focus on inferring scene layout from a single image. This problem has been studied since the earliest days of Artificial Intelligence research, resulting in a host of so-called Shape-from-X methods, where X could be shading, perspective, etc.Unfortunately, each of these methods works under its own assumptions which often do not hold in real images. How these cues interact and integrate remains elusive. Painters constantly use a combination of four techniques: occlusion, perspective, shading, and form to effectively evoke a 3D percept from a 2D picture. Studying their techniques can lend insights into the computation of recovering scene layout from pixel values. The PI proposes to bring artists and vision scientists together to solve the computational problem of scene layout from pictorial cues. This project realizes it in three areas:education, experiments and computational modeling.A new interdisciplinary course, Art and Visual Perception, has been developed at Boston College to give a comprehensive cross-examination of how art contributes to the understanding of vision, and how vision contributes to the generation and viewing of art. Students are actively engaged in both art practice and vision experiments.Learning art and vision together results in a deeper understanding than studying each discipline separately. Students' assignments also result in valuable datasets for vision research.The computational approach to scene layout from pictorial cues in this project is to group pixels into spatially organized surfaces from a global integration of multiple pictorial cues in a spectral graph-theoretic framework. The goal is to turn artistic rendering knowledge on how these cues interact into a computational reality.The PI will study geometry (occlusion and perspective), appearance (brightness and color), and form using eye tracking and psychophysics experiments and computational models. These efforts are organized into two phases that progress from inferring the spatial layout from scenes made of planar surfaces (rooms and streets) to scenes made of curved surfaces (landscape and generic scenes).Intellectual MeritWhat is most remarkable about vision is its ability to perceive 3D spatial layout from a single 2D image. The proposed research replicates this ability in computation from a grouping perspective.Compared to statistical learning approaches, the grouping method is not only generic and thus scales well with the number of scenes, but can also produce a precise organization of surfaces in the scene.Compared to traditional Shape-from-X approaches, the grouping method examines each pictorial cue in conjunction with others. The integration of these multiple pictorial cues allows them for the first time to become applicable to real images. The PI has developed the essential grouping machinery in spectral graph theory for depth segregation. Compared to most existing formulations on this topic, it has unparalleled conceptual simplicity, computational efficiency, and guaranteed near-global optimality. The proposed research on brightness and color perception, in connection with Shape-from- Shading and surface organization, will help clarify the role of low- level and high-level mechanisms in the long-standing scientific debate between Hering and Helmholtz on color perception.Broader ImpactThis project bridges the gap between art and science not only in research but also in education by developing a new curriculum that traverses the areas of neuroscience, psychology, computer science, and visual arts, by involving students in art practice and scientific experiments, and by providing a forum for artists and scientists to exchange ideas on visual perception. These interdisciplinary efforts befit the liberal arts education tradition at Boston College. This project will not only benefit from the strong Fine Arts department on campus, but also cultivate computer science awareness and outreach to non-technical people, and promote the growth of the young Computer Science department at Boston College.URL: http://www.cs.bc.edu/~syu/artvis/