RUI: Intermediate-level Vision: Grouping of generic features for image and video processing

人类的视觉系统能够识别物体，并仅从物体的边界理解场景。它的健壮性令人惊讶，因此它可以在非边界点和边界点的稀疏采样的干扰下保持这种能力。该系统如何实现这一壮举在很大程度上是未知的。这个项目研究计算机如何在算法上复制它。这个问题是根本性的，与知觉组织和中级视觉问题密切相关。因此，这项研究有可能对广泛的计算机视觉应用产生影响。由于输入(一小部分孤立点)与整个图像相比很小，并且没有颜色信息，该算法是有效的，对光照和对比度的变化具有鲁棒性，适用于任何成像模型。主要问题是将边界点内插到感知显著的一组曲面上，而不受虚假的非边界点的干扰。由直线骨架进行的内插带来了点集的时间可逆的多尺度表示，其中显著的边界点倾向于持久地形成多边形曲面，而虚假的非边界点往往随着比例的增加而迅速消失。由于时间的可逆性，每个比例尺上的曲面都可以追溯到原始比例尺或原始点集。因此，该技术可用于从原始点集形成一组显著曲面。本研究开发了一种基于直骨架内插的通用特征分组算法，并将其应用于2D和3D域中的一些中间视觉问题。研究人员积极吸引本科生参与研究，并通过演讲、演示和外展活动促进美国东北部的STEM教育。源代码和工具箱将公开提供，并用于促进STEM教育。

The human vision system is able to recognize objects and understand the scene from the boundary of the objects alone. It is astonishingly robust so that it can sustain this capability under distraction by non-boundary points and sparse sampling of the boundary points. How the system achieves this feat is largely unknown. This project investigates how a computer can replicate it algorithmically. The problem is fundamental and closely related to perceptual organization and intermediate level vision problems. Thus, this research has the potential to impact a wide range of computer vision applications. Since the input (a small set of isolated points) is small compared to the whole image and has no color information, the algorithm is efficient, robust against changes in illumination and contrast, and applicable to any imaging modalities.The main problem is to interpolate boundary points into a perceptually salient set of surfaces without being distracted by spurious non-boundary points. Interpolation by straight skeletons brings a time-reversible, multi-scale representation of a point set where salient boundary points tend to form a polygonal surface persistently while spurious non-boundary points tend to disappear quickly as the scale increases. Because of the time-reversible nature, a surface at each scale can be traced back to the original scale or the original point set. Thus, this technique can be used to form a set of salient surfaces from the original point set. This research develops a general purpose feature grouping algorithm using the straight skeleton interpolation and applies it to a number of intermediate vision problems in 2D and 3D domains. The investigator actively involves undergraduate students into the research and promotes STEM education in the northeastern part of the U.S.A through presentations, demonstrations, and outreach activities. The source code and toolboxes will be made publicly available and used to promote STEM education.