RUI: Intermediate-level Vision: Grouping of generic features for image and video processing

人类的视觉系统能够仅仅从物体的边界来识别物体和理解场景。它具有惊人的鲁棒性，可以在非边界点干扰和边界点稀疏采样的情况下保持这种能力。该系统如何实现这一壮举在很大程度上是未知的。这个项目研究计算机如何通过算法复制它。这个问题是基本的，与知觉组织和中级视觉问题密切相关。因此，这项研究有可能影响广泛的计算机视觉应用。由于输入（一组孤立的点）与整个图像相比很小，并且没有颜色信息，因此该算法效率高，对光照和对比度的变化具有鲁棒性，适用于任何成像模式。主要问题是将边界点插入到感知上显著的曲面集合中，而不会被虚假的非边界点分散注意力。直线骨架插值带来了一个时间可逆的多尺度点集表示，其中显著的边界点倾向于持久地形成多边形表面，而虚假的非边界点则倾向于随着尺度的增加而迅速消失。由于时间可逆的性质，每个尺度的曲面都可以追溯到原始尺度或原始点集。因此，该技术可用于从原始点集形成一组显著面。本研究开发了一种基于直线骨架插值的通用特征分组算法，并将其应用于二维和三维领域的一些中间视觉问题。研究者积极地让本科生参与到研究中，并通过演讲、演示和外展活动促进美国东北部的STEM教育。源代码和工具箱将向公众开放，并用于促进STEM教育。

The human vision system is able to recognize objects and understand the scene from the boundary of the objects alone. It is astonishingly robust so that it can sustain this capability under distraction by non-boundary points and sparse sampling of the boundary points. How the system achieves this feat is largely unknown. This project investigates how a computer can replicate it algorithmically. The problem is fundamental and closely related to perceptual organization and intermediate level vision problems. Thus, this research has the potential to impact a wide range of computer vision applications. Since the input (a small set of isolated points) is small compared to the whole image and has no color information, the algorithm is efficient, robust against changes in illumination and contrast, and applicable to any imaging modalities.The main problem is to interpolate boundary points into a perceptually salient set of surfaces without being distracted by spurious non-boundary points. Interpolation by straight skeletons brings a time-reversible, multi-scale representation of a point set where salient boundary points tend to form a polygonal surface persistently while spurious non-boundary points tend to disappear quickly as the scale increases. Because of the time-reversible nature, a surface at each scale can be traced back to the original scale or the original point set. Thus, this technique can be used to form a set of salient surfaces from the original point set. This research develops a general purpose feature grouping algorithm using the straight skeleton interpolation and applies it to a number of intermediate vision problems in 2D and 3D domains. The investigator actively involves undergraduate students into the research and promotes STEM education in the northeastern part of the U.S.A through presentations, demonstrations, and outreach activities. The source code and toolboxes will be made publicly available and used to promote STEM education.