Effect of removing outliers on statistical inference: implications to interpretation of experimental data in medical research.
Effect of removing outliers on statistical inference: implications to interpretation of experimental data in medical research.
复制标题
DOI：
10.18590/mjm.2018.vol4.iss2.9
复制
发表时间：
2018-01-01
影响因子：
--
通讯作者：
Shapiro, Joseph I
中科院分区：
文献类型：
--
作者：
Gress, Todd W;Denvir, James;Shapiro, Joseph I
背景：消除“离群值”的数据编辑在生物医学科学中是很常见的。这种类型的数据编辑的效果可能会影响研究结果，随着医学研究的大量和不断扩大，这些影响将被放大。方法和结果：我们首先对美国各地机构的医学院教职员工进行了匿名调查，发现确实有很大比例的受访者进行了某种形式的异常值排除。接下来，我们进行了蒙特卡罗模拟，从同一正态分布的样本中排除高值和低值。我们发现，随着样本量增加到数千，去除一对“异常值”，特别是分别去除两个样本的高值和低值，对I类误差有可测量的影响。我们开发了一种对t分数的调整，它考虑了类型I错误的预期变化(tadj=TOBS-2(log(N)^0.5/n^0.5))，并建议在参数分析之前消除异常值时使用这一调整。结论：数据编辑通过消除异常值，包括分别从两个样本中去除高值和低值，可以显著影响类型I型错误的发生。这种类型的数据编辑可能会对大容量研究领域产生深远的影响，特别是在医学领域，我们建议使用对t分数的调整来减少出错的可能性。
BACKGROUND: Data editing with elimination of "outliers" is commonly performed in the biomedical sciences. The effects of this type of data editing could influence study results, and with the vast and expanding amount of research in medicine, these effects would be magnified.METHODS AND RESULTS: We first performed an anonymous survey of medical school faculty at institutions across the United States and found that indeed some form of outlier exclusion was performed by a large percentage of the respondents to the survey. We next performed Monte Carlo simulations of excluding high and low values from samplings from the same normal distribution. We found that removal of one pair of "outliers", specifically removal of the high and low values of the two samplings, respectively, had measurable effects on the type I error as the sample size was increased into the thousands. We developed an adjustment to the t score that accounts for the anticipated alteration of the type I error (tadj=tobs-2(log(n)^0.5/n^0.5)), and propose that this be used when outliers are eliminated prior to parametric analysis.CONCLUSION: Data editing with elimination of outliers that includes removal of high and low values from two samples, respectively, can have significant effects on the occurrence of type 1 error. This type of data editing could have profound effects in high volume research fields, particularly in medicine, and we recommend an adjustment to the t score be used to reduce the potential for error.