Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
Generative PointNet: Deep Energy-Based Learning on Unordered Point Sets for 3D Generation, Reconstruction and Classification
复制标题
DOI：
10.1109/cvpr46437.2021.01473
复制
发表时间：
2020-04
期刊：
影响因子：
--
通讯作者：
Jianwen Xie;Yifei Xu;Zilong Zheng;Song-Chun Zhu;Y. Wu
中科院分区：
文献类型：
--
作者：
Jianwen Xie;Yifei Xu;Zilong Zheng;Song-Chun Zhu;Y. Wu
我们提出了一种基于能量模型的无序点集(如点云)的生成模型，其中能量函数由输入置换不变的自底向上神经网络来参数化。能量函数学习每个点的坐标编码，然后将所有单独的点要素聚合为整个点云的能量。我们称我们的模型为生成性PointNet，因为它可以从区分PointNet派生出来。我们的模型可以通过基于MCMC的最大似然学习(及其变体)来训练，而不需要像GANS和VAE中那样的任何辅助网络的帮助。与大多数依赖于手动创建的距离度量的点云生成器不同，我们的模型不需要任何手动创建的距离度量，因为它通过根据能量函数定义的统计属性匹配观测样本来合成点云。此外，我们还可以学习到基于能量的模型的短期运行的MCMC，作为点云重建和内插的流生成器。学习的点云表示可用于点云分类。实验证明了本文提出的点云生成模型的优越性。
We propose a generative model of unordered point sets, such as point clouds, in the forms of an energy-based model, where the energy function is parameterized by an input-permutation-invariant bottom-up neural network. The energy function learns a coordinate encoding of each point and then aggregates all individual point features into an energy for the whole point cloud. We call our model the Generative PointNet because it can be derived from the discriminative PointNet. Our model can be trained by MCMC-based maximum likelihood learning (as well as its variants), without the help of any assisting networks like those in GANs and VAEs. Unlike most point cloud generators that rely on hand-crafted distance metrics, our model does not require any hand-crafted distance metric for the point cloud generation, because it synthesizes point clouds by matching observed examples in terms of statistical properties defined by the energy function. Furthermore, we can learn a short-run MCMC toward the energy-based model as a flow-like generator for point cloud reconstruction and interpolation. The learned point cloud representation can be useful for point cloud classification. Experiments demonstrate the advantages of the proposed generative model of point clouds.