Learning Highly Structured Sparse Latent Variable Models

技术进步带来了收集和分析高维数据库中模式的能力。一种特殊类型的分析涉及的问题是，记录的变量间接测量隐藏的因素，解释了观察到的关联。例如，最近的2009年全国国民保健服务工作人员调查，超过10万名工作人员，载有几个问题的工作满意度。观察到的答案模式是一些未被记录的常见隐藏因素的结果，这是很自然的。特别是，这些答案可以通过对工作实践质量的看法、同事的支持等因素进行分组，这些因素只能间接测量。在实践中，当从高维数据源中获得意义时，将观察结果减少到少数共同因素是有用的。由于记录受到与实际因素无关的可变性来源的影响（想想某人有一个糟糕的一天，甚至错误地输入错误的信息），因此删除这些工件也是统计问题的一部分。估计这种转换的模型被称为“降维”和“平滑”。有多种方法可以完成这些任务。在光谱的一端，有一些模型假设数据与一些非常简单的模式相匹配，例如钟形曲线和预定因素。另一些则非常强大，允许灵活的模式，甚至在一些非常温和的假设下从数据中推断出无限数量的因素。拟议的工作试图弥合这些极端：非常灵活的模型的缺点是微妙的，但重要的。特别是，它们可能对数据的变化非常敏感-这意味着如果提供稍微不同的一组观察结果，可能会得出一些关于隐藏因素的非常不同的结论。此外，还有计算方面的问题：计算所需的估计值通常需要一个迭代过程，这个过程需要对这些估计值进行一些初始猜测。因此，即使对于一个固定的数据集，如果不仔细选择这样的初始猜测，结果也会有很大的不同。我们的动机是，如果一个人确实有这些顾虑，那么他也可以不嫌麻烦地整合关于该领域的领域知识。结论：我们的目标不是通用的，而是针对存在一些合理领域知识的应用程序。特别是，我们专注于问题的隐藏目标的兴趣是预先指定的，但无限多的其他可能存在。当我们将数据映射到一个固定的隐变量空间时，我们提供了一种方法，该方法对存在无限数量的其他隐式公共因素具有鲁棒性。所提出的模型是自适应的：它们考虑到了给定隐藏因素之间可能存在的额外变异性，而这些因素可能会被更简单的模型遗漏。同时，它们被设计成对初始条件不太敏感，同时对数据集中的小变化不太敏感。

Technological advances have brought the ability of collecting and analysing patterns in high-dimensional databases. One particular type of analysis concerns problems where the recorded variables indirectly measure hidden factors that explain away observed associations. For instance, the recent National NHS Staff Survey of 2009, taken by over one hundred thousand staff members, contained several questions on job satisfaction. It is only natural that the patterns of observed answers are the result of some common hidden factors that remain unrecorded. In particular, such answers could arguably be grouped by factors such as perceptions of the quality of work practice, support of colleagues and so on, that are only indirectly measured.In practice, when making sense out of a high-dimensional data source, it is useful to reduce the observations to a small number of common factors. Since records are affected by sources of variability that are unrelated to the actual factors (think of someone having a bad day, or even typing wrong information by mistake), removing such artifacts is also part of the statistical problem. A model that estimates such transformations is said to perform "dimensionality reduction" and "smoothing".There are a variety of methods to accomplish such tasks. At one end of the spectrum, there are models that assume the data match some very simple patterns such as bell curves and pre-determined factors. Others are very powerful, allowing for flexible patterns and even an infinite number of factors that are inferred from data under some very mild assumptions. The proposed work tries to bridge these extremes: the shortcomings of the very flexible models are subtle but important. In particular, they can be very sensitive to changes in the data - meaning some very different conclusions about the hidden factors might be achieved if a slightly different set of observations is provided. Moreover there are computational concerns: calculating the desired estimates usually requires an iterative process, a process that needs some initial guess about these estimates. So, even for a fixed dataset, results can vary considerably if such an initial guess is not carefully chosen. Our motivation is that if one does have these concerns, one might as well take the trouble of incorporating domain knowledge about the domain. The upshot: we do not aim to be general, and instead target applications where some reasonable domain knowledge exists. In particular, we focus on problems where the hidden targets of interest are pre-specified, but infinitely many others might exist. While we map our data to a fixed space of hidden variables, we provide an approach that is robust to the presence of an unbounded number of other, implicit, common factors. The proposed models are adaptive: they account for possible extra variability between the given hidden factors that would be missed by the simpler models. At the same time, they are designed to be less sensitive to initial conditions while being less sensitive to small changes in the datasets.