Estimating Probabilistic Models on Curved Surfaces using Score Matching

经典的统计建模，如最大似然估计，依赖于概率密度模型的归一化常数的知识。在某些情况下，例如，在通用截断域上观察数据时，归一化常数是难以处理的。虽然传统的方法通常通过数值积分来逼近这一项，但诸如分数匹配(Hyvärinen，2005)和最小Stein偏差估计(Barp等人，2019；Liu等人，2016)等方法完全绕过了它的评估。在第三章中，我们提出了一种使用分数匹配来定义一般截断区域上的密度估计的方法。这种方法依赖于在区域边界处等于零的加权函数，对于该区域，我们提出了一个距离函数，该距离函数是从最大化Stein差异自然产生的。数值实验表明了所提方法的潜力，包括真实的芝加哥犯罪数据，以及校正异常值调整的过度补偿。在第四章中，我们扩展了第三章中的工作，其中数据既被截断又位于流形的表面。我们给出了该方法在球面上的一个应用，并提出了两个距离函数作为相应的加权函数。在球面上的实验表明，该方法比以往的欧几里德域方法获得了更低的估计误差。在第五章中，我们定义了一个近似的Stein类，对其Stein恒等式成立。这使得能够构建截断的核化Stein差异，该差异只需要访问一组边界点。在实验中，我们证明了即使在这种宽松的假设下，该方法的性能与以前的方法相当。最后，在第六章中，我们使用时间得分匹配得到了非归一化指数族密度参数的时变导数的估计(Choi等人，2022)。我们提出了一种基于该估计量的渐近方差的变点检测方法，该方法不需要对期望的变点类型做任何先验假设。这种高度灵活的方法提供了与以前流行的变点检测方法相当的结果。我们展示了它对真实世界应用的适用性，以及检测何时使用神经网络表示将不分布的图像引入到数据集的能力。

Classical statistical modelling such as maximum likelihood estimation relies on knowledge of the normalising constant of a probability density model. Under certain cases, for example where data are observed on a generic truncated domain, the normalising constant is intractable. Whilst conventional methods usually approximate this term via numerical integration, methods such as score matching (Hyvärinen, 2005) and minimum Stein discrepancy estimators (Barp et al., 2019; Liu et al., 2016) bypass its evaluation entirely.In chapter 3, we present a method that uses score matching to define a density estimator on a generic truncation domain. This method relies on a weighting function that is equal to zero at the boundary of the domain, for which we propose a distance function which arises naturally from maximising a Stein discrepancy. Numerical experiments show the potential of the proposed method across a range of experiments, including real world Chicago crime data, and correcting the over-compensation from outlier trimming.In chapter 4, we extend the work from chapter 3 where data are both truncated and lie on the surface of a manifold. We present an application of this method to the sphere, and propose two distance functions as the corresponding weighting function. Experiments on the sphere show that this method achieves lower estimation error than prior Euclidean domain methods. We present a real-world application estimating the mean of storm locations truncated over the continental United States.In chapter 5, we define an approximate Stein class for which the Stein identity holds approximately. This enables construction of a truncated kernelised Stein discrepancy, which only requires access to a set of boundary points. In experiments, we show that even with this relaxed set of assumptions, the method is comparable in performance to previous methods.Finally, in chapter 6, we derive an estimator of the time-varying derivative of the parameter of an unnormalised exponential family density using time score matching (Choi et al., 2022). We present a changepoint detection method based on the asymptotic variance of this estimator which needs no prior assumptions on the type of changepoint expected. This highly flexible method provides comparable results to previous popular changepoint detection methods. We show its applicability to real-world applications, as well as the ability to detect when out-of distribution images are introduced to a dataset using a neural network representation.