Parameter estimation with constraints and its stability in statistical model

在这个项目中，我们研究了具有二次和线性约束的最大相关系数问题。这个问题是为了最大化目标变量和解释变量的加权和w‘x之间的相关性来确定权重向量w，其中参数w必须满足二次约束(w-w_0)’(w-w_0)≦^2和线性约束A_w≦c。为了解析地解决这个问题，我们首先对矩阵(i-q)Z进行奇异值分解，其中q是投影器，Z是设计矩阵。利用奇异值分解产生的线性变换，将样本空间上定义的量变换为参数空间上定义的量。然后考虑包含对应于二次约束和线性约束的集中椭球体的凸锥，并通过几何洞察得到最优解。为了将这些最优解与最小二乘解进行比较，我们系统地生成了三维人工数据集。我们还使用了实用的数据集，如入学考试数据集。在此基础上，我们考察了两种解决方案之间的差异。结果发现，当从目标变量向量转换而来的向量远离约束所指定的椭球体时，差异非常大，因此得到的相关系数非常不同；例如，0.75和0.34。此外，我们还考察了数据值的微小变化对参数估计的影响。结果发现，当目标变量向量转换成的向量与约束椭球中心所指定的向量几乎垂直时，最优解可能会发生灾难性的变化。

In this project, we investigated a problem of maximizing correlation coefficient with quadratic and linear constraints. This problem is to determine the weight vector w so as to maximize the correlation between an objective variable and a weighted sum w'x of explanatory variables, where the parameter w must satisfy a quadratic constraint (w-w_0)' (w-w_0)≦^2 and a linear constraint A_w≦c. To solve this problem analytically, we first made the singular value decomposition (SVD) of the matrix (I-Q)Z, where Q is the projector and Z is the design matrix. Using the linear transformation induced by the SVD, we transformed the quantities defined on sample space to ones defined on parameter space. We then considered the convex cone which included the concentrated ellipsoid corresponding to the quadratic and linear constraints, and obtained the optimal solution through geometrical insight. In order to compare these optimal solutions with the least squares solutions, we systematically generated three dimensional artificial data sets. Also we used practical data sets such as entrance examination data sets. Based on these we examined the differences between two kinds of solutions. As the result, we found that the difference was very large when the vector transformed from an objective variable vector was far from the ellipsoid specified by the constraints, and so the resulting correlation coefficients were very different ; for example, 0.75 and 0.34. Further we investigated the influence on the parameter estimates by small change of data values. As the result, we found that some catastrophic change might happen in the optimal solution when the vector transformed from an objective variable vector was nearly perpendicular to the vector specified by the center of the constraint ellipsoid.