The reliability and validity of automated tools for examining variation in syntactic complexity across genres
The reliability and validity of automated tools for examining variation in syntactic complexity across genres
复制标题
用于检查跨流派句法复杂性变化的自动化工具的可靠性和有效性
DOI：
10.1111/ijal.12200
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Hyung
中科院分区：
文献类型：
--
作者：
C. Polio;Hyung
本研究调查了两个自动化系统，即L2句法复杂性分析器（SCA）和Coh‐Metrix，以分析英语句法复杂性，作为捕获两种体裁之间差异的一种方法。本文的目的是部分复制以前的研究，发现叙事和议论文之间的复杂性差异，同时也评估两个自动化的工具，可用于分析在其他情况下的变化。我们首先通过将SCA和Coh‐ Meetings的结果与30篇论文的手工编码结果进行比较来测试其可靠性。然后，我们使用81名ESL学生写的162篇文章（记叙文和议论文）来检验这两个系统检测体裁对句法复杂性的影响的能力。结果表明，这两个系统的大多数措施是可靠的，但有些措施是不透明和一致的。此外，SCA和Coh-Meetings的措施表明，更高的句法复杂性的议论文比叙述，确认了以前的研究结果，这两种工具可能是有效的研究复杂性的变化。研究结果进行了讨论的体裁差异的动机。
This study investigates two automated systems, the L2 syntactic complexity analyzer (SCA) and Coh‐Metrix, for their analysis of English syntactic complexity as a way to capture variation across two genres. The purpose of the paper is to partially replicate previous studies that have found complexity differences between narrative and argumentative essays while also evaluating two automated tools that can be used for analysing variation in other contexts. We first test the reliability of SCA and Coh‐Metrix by comparing their results with the hand‐coded results of 30 essays. We then examine the two systems' ability to detect genre effects on syntactic complexity using 162 essays (narrative and argumentative) written by 81 ESL students. Results indicate that the majority of measures from the two systems are reliable, but some measures were not transparent and consistent. Additionally, both SCA and Coh‐Metrix measures show higher syntactic complexity in the argumentative essays than the narratives, confirming both the findings of previous research and that both tools may be valid for studying variation in complexity. Findings are discussed in terms of the motivation for genre differences.