Assessing the reliability of textbook data in syntax: Adger's Core Syntax1
Assessing the reliability of textbook data in syntax: Adger's Core Syntax1
复制标题
评估教科书数据语法的可靠性：Adger 的核心语法1
DOI：
10.1017/s0022226712000011
复制
发表时间：
2012
影响因子：
1.1
通讯作者：
Diogo Almeida
中科院分区：
文献类型：
--
作者：
Jon Sprouse;Diogo Almeida
至少50年来，对句法中可接受性判断数据的可靠性一直存在一致的批评模式（例如，Hill 1961），在过去的十年中，在几个高调的批评中达到高潮（Edelman & Christiansen 2003，Ferreira 2005，Wasow & Arnold 2005，吉布森& Fedorenko 2010，在印刷中）。这些批评者的基本主张是，传统的可接受性判断收集方法，与实验心理学的方法相比，往往是相对非正式的，导致了不可容忍的大量假阳性结果。在本文中，我们通过正式测试所有469个（独特的，美国英语）数据点从流行的语法教科书（Adger 2003）使用440天真的参与者，两个判断任务（幅度估计和是非），和三种不同类型的统计分析（标准频率检验，线性混合效应模型，贝叶斯因子分析）的经验评估这一说法。结果表明，传统方法和正式实验方法之间的最大差异为2%。这表明，即使在（可能没有根据的）假设下，不一致的结果都是由于传统方法的缺点而进入句法文献的假阳性，这469个数据点的最低复制率为98%。我们讨论了这些结果的句法数据的可靠性问题的影响，以及这些结果的实际后果，可供syntocurcians的方法选择。
There has been a consistent pattern of criticism of the reliability of acceptability judgment data in syntax for at least 50 years (e.g., Hill 1961), culminating in several high-profile criticisms within the past ten years (Edelman & Christiansen 2003, Ferreira 2005, Wasow & Arnold 2005, Gibson & Fedorenko 2010, in press). The fundamental claim of these critics is that traditional acceptability judgment collection methods, which tend to be relatively informal compared to methods from experimental psychology, lead to an intolerably high number of false positive results. In this paper we empirically assess this claim by formally testing all 469 (unique, US-English) data points from a popular syntax textbook (Adger 2003) using 440 naïve participants, two judgment tasks (magnitude estimation and yes–no), and three different types of statistical analyses (standard frequentist tests, linear mixed effects models, and Bayes factor analyses). The results suggest that the maximum discrepancy between traditional methods and formal experimental methods is 2%. This suggests that even under the (likely unwarranted) assumption that the discrepant results are all false positives that have found their way into the syntactic literature due to the shortcomings of traditional methods, the minimum replication rate of these 469 data points is 98%. We discuss the implications of these results for questions about the reliability of syntactic data, as well as the practical consequences of these results for the methodological options available to syntacticians.