Contingency matrix theory : Statistical dependence in a contingency table
Contingency matrix theory : Statistical dependence in a contingency table
复制标题
列联矩阵理论：列联表中的统计依赖性
DOI：
10.1016/j.ins.2008.11.023
复制
发表时间：
2009
影响因子：
8.1
通讯作者：
Shusaku Tsumoto
中科院分区：
文献类型：
--
作者：
Masao OKABE;Akiko YOSHIOKA;Keido KOBAYASHI;Takahira YAMAGUCHI;Shusaku Tsumoto
偶然发现的目的是从意外关系的角度理解函数依赖的意义。最重要的观察之一是，这种机会隐藏在从给定数据中提取的大量coocurrencies之下。另一方面，传统的数据挖掘方法强烈依赖于频率和统计或相关性，而不是兴趣或意外性。本文讨论了统计相关性思想的局限性，特别是从线性代数的角度讨论了辛普森悖论的形式特征。理论结果表明，当列联表作为矩阵是非正则的，即列联矩阵的秩不满时，可以观察到这种辛普森悖论。因此，数据有序证据给出了一些局限性，这应该通过面向人的推理来弥补。
Chance discovery aims at understanding the meaning of functional dependency from the viewpoint of unexpected relations. One of the most important observations is that such a chance is hidden under a huge number of coocurrencies extracted from a given data. On the other hand, conventional data-mining methods are strongly dependent on frequencies and statistical or dependence rather than interestingness or unexpectedness. This paper discusses some limitations of ideas of statistical dependence, especially focusing on the formal characteristics of Simpson’s paradox from the viewpoint of linear algebra. Theoretical results show that such a Simpson’s paradox can be observed when a given contingency table as a matrix is not regular, in other words, the rank of a contingency matrix is not full. Thus, data-ordered evidence gives some limitations, which should be compensated by human-oriented reasoning.