Developing the maximum likelihood estimation for component analysis: A state space approach

传统上，科学家在探索性数据分析中使用成分分析(CA)作为描述性工具。在数学上，分量被定义为观察变量的线性组合。CA的原型是主成分分析。CA的其他基本方法包括典型相关分析和冗余分析。在过去的几十年里，这些基本组件模型的各种推广被开发出来，包括同时组件分析、三向组件分析、多集典型相关分析和扩展冗余分析，仅举几例。 在统计文献中，大多数分量模型都是用最小二乘法估计的，这种方法只能给出有限的参数估计信息。相比之下，非常流行和最常用的估计程序-最大似然法(ML)-对于CA来说有点不发达。尽管ML方法要求变量服从正态分布的假设，但在许多情况下，ML方法往往是首选的，因为1)ML估计器是一致的，2)比LS估计器更有效，3)提供了更多可用于统计推断的信息，包括单个参数的标准误差、似然比检验统计量和用于模型选择的几个信息标准。此外，在正态分布下得到的最大似然估计和似然比检验统计量对某些违反正态假设的情况保持了稳健性。最后，ML方法构造了似然函数，在假设违反和/或模型错误指定的情况下，该似然函数可用于计算稳健标准误差。 鉴于ML方法的许多优点，我对我未来的研究有两个主要目标：1)为一些现有的构件模型开发ML方法；2)提出适合于心理和行为研究的新的构件模型。在许多情况下，第一个目标可以通过对特定组件模型的状态空间模型(SSM)进行适当的规范来实现，这给应用研究人员带来了实际的便利，即可以使用为SSM设计的软件程序来估计各种组件模型，而现代文献中开发的许多组件模型的实现需要专门的程序。 在头三年，为这两个目标规划了六个项目，每个项目预计将产生一篇同行评议的期刊文章。在过去的两年和更长的时间里，可以做更多与ML方法和CA相关的主题，例如线性和非线性组件模型的惩罚ML方法，以及开发更好的数值优化算法。总之，这一研究方向将为多元统计分析做出重要贡献。

Traditionally, scientists have used component analysis (CA) as descriptive tools in exploratory data analysis. Mathematically, a component is defined as a linear combination of the observed variables. The prototype of CA is principal component analysis. Other basic methods of CA include canonical correlation analysis and redundancy analysis. Over the past several decades, various generalizations of these basic component models were developed, including simultaneous component analysis, three-way component analysis, multiple-set canonical correlation analysis, and extended redundancy analysis, just to name a few. In the statistical literature, most component models are estimated by the least square method, which only gives limited information for parameter estimation. In contrast, the very popular and most commonly used estimation procedure—the maximum likelihood (ML) method—is somewhat under-developed for CA. Despite the fact that the ML method requires the assumption of normal distribution on the variables, the ML method is often preferred in many instances, because 1) the ML estimator is consistent, 2) is more efficient than the LS estimator, and 3) provides more information that can be used for statistical inferences, including the standard errors for individual parameters, the likelihood ratio test statistic, and several information criteria for model selection purpose. Moreover, the ML estimator and the likelihood ratio test statistic derived under the normal distribution retain their robustness to some violations of the normality assumption. Finally, the ML method constructs the likelihood function that can be used to calculate the robust standard errors in case of assumption violations and/or model misspecifications. Given the many advantages of the ML method, I have two major objectives for my future research including 1) developing the ML method for some existing component models and 2) proposing new component models suitable for psychological and behavioral research. The first objective, in many situations, can be achieved through a proper specification of a state space model (SSM) for a particular component model, which brings a practical convenience to applied researchers that software programs designed for SSM can be used to estimate various component models, whereas the implementation of many component models developed in the modern literature requires a specialized program. For the first three years, six projects are planned for the two objectives, and each project is expected to produce a peer-reviewed journal article. For the last two years and beyond, more topics related to the ML method and CA can be done, such as the penalized ML method for both linear and nonlinear component models and the development of better algorithms for numeric optimization. In sum, this line of research will make important contributions to multivariate statistical analysis.