MCS: Research on Detection and Classification of 2D and 3D Shapes in Cluttered Point Clouds

图像是我们数字社会中最大的数据和信息来源。许多应用领域，如医疗诊断、国土安全、军事监视和互联网通信，可以受益于用于分析图像的自动化技术，特别是用于检测、分类和分析图像中的对象的自动化技术。用于图像中的对象检测的快速技术通常分两步工作：（1）使用快速技术从图像中提取某些基本基元（突出点、边缘、弧等），以及（2）在这些提取的基元中收集感兴趣的对象。第二步-基于形状的2D和3D点云中的对象发现的统计框架-是本研究的重点。由于提取的基元可能属于对象或背景，给定的数据是嘈杂和混乱。简单地说，这个问题类似于在繁星点点的夜晚寻找天空中的北斗七星。简单的组合搜索是不可能的，因为将点组织成多边形形状的计算成本过高。这里提出的框架是通过合成进行分析：首先合成连续的形状--2D的轮廓和3D的表面--对于感兴趣的形状类，并将这些形状采样为点集。然后将这些合成的点集与给定的点云进行（概率）比较，以确定图像中是否存在形状类。本研究将采用贝叶斯方法进行形状分类，其中估计给定点云中存在的不同形状类别的后验概率。这涉及到一个基本步骤，整合出某些滋扰变量：（i）未知形状的对象存在于数据中，（ii）未知的姿态和规模，它出现在场景中，以及（iii）未知的采样连续形状到离散点。研究人员将开发特定类别的统计模型来捕获这些讨厌的变量的变化，并将使用蒙特卡罗方法来模拟这些模型以近似所需的后验。该框架依赖于以下成分：（1）统计形状模型：首先，研究人员将导出形状（曲线和曲面）的数学表示，在其形状空间上施加黎曼度量，并开发用于计算测地线的算法。其次，他们将定义和估计概率模型的形状空间和模拟形状从这些模型用于生成随机推断。由于形状空间通常是无限维的非线性流形，因此形状的有效统计推断的前景既新颖又具有挑战性。(2)形状采样：为了合成一个假设的点集，我们需要一个连续的形状，并用有限数量的点对其进行采样。研究人员将为这个采样过程开发数学表示和随机模型。(3)可能性评估：最后，需要计算给定点云的可能性。这涉及最佳地配准和变换（旋转、平移和缩放）合成点集以匹配给定数据。成本函数基于观测噪声和背景杂波的概率模型。该项目将研究，开发和实施这些基本要素，用于在点云中查找形状。所提出的框架将进行测试的性能和效率，在图像中的对象检测和分类。

Images form the largest source of data and information in our digital society. Many application domains, such as medical diagnostics, homeland security, military surveillance, and Internet communication, can benefit from automated techniques for analyzing images and, in particular, for detection, classification, and analysis of objects in images. Fast techniques for object detection in images often work in two steps: (1) Extract certain basic primitives (prominent points, edges, arcs, etc) from images using fast techniques and, (2) glean objects of interest in these extracted primitives. This second step -- a statistical framework for shape-based discovery of objects in 2D and 3D point clouds -- is the focus of this research. Since the extracted primitives may belong either to objects or backgrounds, the given data is both noisy and cluttered. In simple terms, this problem is akin to finding the big dipper in the sky on a starry night. A simple combinatorial search is impossible, as the computational cost of organizing points into polygonal shapes are prohibitive. The framework proposed here is analysis by synthesis: one starts by synthesizing continuous shapes Ð contours for 2D and surfaces for 3D -- for the shape classes of interest and samples these shapes into sets of points. These synthesized point sets are then (probabilistically) compared with the given point cloud to decide if a shape class is present in the image. This research will take a Bayesian approach to shape classification where one estimates the posterior probabilities of different shape classes being present in the given point cloud. This involves a fundamental step of integrating out certain nuisance variables: (i) the unknown shape of object present in the data, (ii) the unknown pose and scale at which it appears in the scene, and (iii) the unknown sampling of a continuous shape into discrete points. The investigators will develop class-specific statistical models to capture variability of these nuisance variables, and will use a Monte Carlo approach that simulates from these models to approximate the desired posterior. This framework relies on the following ingredients: (1) Statistical shape models: Firstly, the investigators will derive mathematical representations of shapes (of curves and surfaces), impose Riemannian metrics on their shape spaces and develop algorithms for computing geodesics. Secondly, they will define and estimate probability models on the resulting shape spaces and simulate shapes from those models for use in generating stochastic inferences. Since shape spaces are typically infinite-dimensional, nonlinear manifolds, the prospect of efficient statistical inferences of shapes is both novel and challenging. (2) Shape Sampling: To synthesize a hypothesized point set, one takes a continuous shape and samples it with a finite number of points. The researchers will develop mathematical representations and stochastic models for this sampling process. (3) Likelihood Evaluation: Lastly, one needs to calculate the likelihood of the given point cloud. This involves optimally registering and transforming (rotating, translating, and scaling) the synthetic point set to match the given data. The cost function is based on probability models for the observation noise and the background clutter. This project will research, develop, and implement these fundamental ingredients for finding shapes in point clouds. The proposed framework will be tested for performance and efficiency in detecting and classifying objects in images.