Parameter estimation with constraints and its stability in statistical model

点击翻译按钮获取中文摘要

In this project, we investigated a problem of maximizing correlation coefficient with quadratic and linear constraints. This problem is to determine the weight vector w so as to maximize the correlation between an objective variable and a weighted sum w'x of explanatory variables, where the parameter w must satisfy a quadratic constraint (w-w_0)' (w-w_0)≦^2 and a linear constraint A_w≦c. To solve this problem analytically, we first made the singular value decomposition (SVD) of the matrix (I-Q)Z, where Q is the projector and Z is the design matrix. Using the linear transformation induced by the SVD, we transformed the quantities defined on sample space to ones defined on parameter space. We then considered the convex cone which included the concentrated ellipsoid corresponding to the quadratic and linear constraints, and obtained the optimal solution through geometrical insight. In order to compare these optimal solutions with the least squares solutions, we systematically generated three dimensional artificial data sets. Also we used practical data sets such as entrance examination data sets. Based on these we examined the differences between two kinds of solutions. As the result, we found that the difference was very large when the vector transformed from an objective variable vector was far from the ellipsoid specified by the constraints, and so the resulting correlation coefficients were very different ; for example, 0.75 and 0.34. Further we investigated the influence on the parameter estimates by small change of data values. As the result, we found that some catastrophic change might happen in the optimal solution when the vector transformed from an objective variable vector was nearly perpendicular to the vector specified by the center of the constraint ellipsoid.