Multicollinearity: A Bayesian Interpretation
Multicollinearity: A Bayesian Interpretation
复制标题
多重共线性：贝叶斯解释
DOI：
10.2307/1927962
复制
发表时间：
1973
期刊：
影响因子：
--
通讯作者：
Edward E. Leamer
中科院分区：
文献类型：
--
作者：
Edward E. Leamer
在单变量多元回归模型的背景下，共线数据集的问题已经产生了一系列令人困惑的论文，评论和脚注。细读这些文献并不能使读者得出结论，这个问题甚至已经被严格定义。本文件的目的是提供几个严格的定义，并提出几个实质性的数量总结的程度的问题。具体地，我们关注回归模型Y = X，B + u，其中，B是具有元素81，12，.的k维参数向量。其中Y和u是T X 1向量，X是T X k矩阵。根据对Y和X的观测，可以对矢量B作出推论。当X的列正交时，设计矩阵X 'X是对角的。X的相关列表示非对角设计矩阵。共线性问题与在这两种情况下可能得出的推论的差异有关。本文的主要主张是，共线性问题的最重要的方面来自非支配的不确定的先验信息的存在，导致在解释数据证据的主要问题。它声称在这里，如果我们的先验知识的参数值是完全肯定的或“完全不确定”的方面的共线性问题，我们大多数人担心会消失。作为对这一命题的实证检验，考虑共线性被确定为罪魁祸首的情况。通常，符号是错误的，或者点估计是特殊的。有时，置信区间会与参数空间中不太可能的区域重叠。然而，说这些事情就是说存在非支配的不确定先验信息。经典的推理，与可能的例外的预测试文献，必然排除非支配的不确定的先验信息。因此，大多数关于共线性问题的讨论都忽略了一个关键点。包括Theil（1971，p. 149），Malinvaud（1970，p. 218），和Goldberger（1964，p. 192）在内的教科书讨论观察到，当设计矩阵X 'X变为奇异时，最小二乘估计量是非唯一的，并且抽样分布仅对某些“可估计”函数具有有限方差。因此，极端共线性被隐含地定义为完全缺乏关于某些参数的样本信息。不太极端的共线性的情况下，没有处理这么平凡，因为有没有在最小二乘定理，这显然是依赖于“近不可逆性”的设计矩阵。这一事实使Kmenta（1971，第391页）得出结论：“高度的多重共线性只是样本的一个特征，它导致了估计系数的不可靠性，但与由于这种不可靠性而得出的结论无关。换句话说，定义共线性的问题可以通过确定一个距离函数来解决，该距离函数用于测量设计矩阵与某些不可逆矩阵的接近程度，其中共线性问题是明确极端的。由于极端情况与参数的无穷大边际方差有关，Theil（1971，第152页）、Malinvaud（1970，第218页）和Goldberger（1964，第193页）等作者使用了与系数的抽样方差非正式相关的距离函数。共线性定义为大方差。这个定义的失败之处在于，它没有定义一个新的问题，而是确定了一个已经很好理解的问题的新原因-证据不足。虽然共线性作为弱证据问题的原因可以与其他原因（如小样本或大残差方差）区分开来，但共线性于1972年11月7日收到供发表。修订版于1973年1月23日接受出版。* 本文的研究得到了NSF Grant GS 319.29的支持。作者从与加里·张伯伦和理查德·科普克关于这个问题的谈话中受益匪浅。讨论和内容都大大受益于裁判的评论。早期的版本是在197，2年10月在明尼苏达大学举行的NBER-NSF计量经济学贝叶斯推理研讨会上提出的。[1]关于定义完全不确定性问题的讨论，见泽尔纳（1971年，第2章）。
T HE problem of collinear data sets in the context of the univariate multiple regression model has generated a confusing array of papers, comments and footnotes. Perusal of this literature does not lead the reader to conclude that the problem has even been rigorously defined. The purpose of this paper is to offer several rigorous definitions and to suggest several substantive quantitative summaries of the degree of the problem. Specifically we address our attention to the regression model Y = X,B + u where ,B is a k-dimensional parameter vector with elements 81,12, ... *k, where Y and u are T X 1 vectors and where X is a T X k matrix. Inferences are to be made about the vector ,B from observations of Y and X. When the columns of X are orthogonal, the design matrix X'X is diagonal. Correlated columns of X imply a nondiagonal design matrix. The collinearity problem has to do with the differences in the inferences that may be drawn in these two situations. The principal claim of this paper is that the most important aspects of the collinearity problem derive from the existence of undominated uncertain prior information which causes major problems in interpreting the data evidence. It is claimed here that if our a priori knowledge of parameter values were either completely certain or "completely uncertain" the aspects of the collinearity problem that most of us worry about would disappear.' As an empirical test of this proposition consider the situations when collinearity is identified as a culprit. Usually signs are wrong or point estimates are otherwise peculiar. Occasionally confidence intervals overlap unlikely regions of the parameter space. Yet to say these things is to say there exists undominated uncertain prior information. Classical inference, with the possible exception of the pretesting literature, necessarily excludes undominated uncertain prior information. As a result most discussions of the collinearity problem miss a critical point. The textbook discussions including Theil (1971, p. 149), Malinvaud (1970, p. 218), and Goldberger (1964, p. 192), observe that when the design matrix X'X becomes singular, the least squares estimator is non-unique and the sampling distribution has finite variance only for certain "estimable" functions. Thus extreme collinearity is implicitly defined as total lack of sample information about some parameters. The case of less extreme collinearity is not dealt with so trivially since there is nothing in the least-squares theorems that is obviously dependent on the "near non-invertibility" of the design matrix. This fact has led Kmenta (1971, p. 391) to conclude "that a high degree of multicollinearity is simply a feature of the sample that contributes to the unreliability of the estimated coefficients, but has no relevance for the conclusions drawn as a result of this unreliability." To put this another way, the problem of defining collinearity may be solved by identifying a distance function for measuring the closeness of the design matrix to some noninvertible matrix in which the collinearity problem is unambiguously extreme. Since the extreme case is associated with infinite marginal variances on the parameters, authors such as Theil (1971, p. 152), Malinvaud (1970, p. 218), and Goldberger (1964, p. 193) use a distance function informally related to the sampling variance of the coefficients. Collinearity is defined as large variances. The failure of this definition is that instead of defining a new problem, it identifies a new cause of an already well-understood problem weak evidence. Although collinearity as a cause of the weak evidence problem can be distinguished from other causes such as small samples or large residual error variances, collinearity as Received for publication November 7, 1972. Revision accepted for publication January 23, 1973. * Research for this paper was supported by NSF Grant GS 319.29. The author has benefited from conversations on the subject with Gary Chamberlain and Richard Kopcke. Both the discussion and the content have benefited significantly from a referee's comments. An earlier version was presented at the NBER-NSF Seminar in Bayesian Inference in Econometrics at the University of Minnesota, October, 197,2. 1 See Zellner (1971, chapter 2) for a discussion of the problem of defining complete uncertainty.