CAREER: Structured Scientific Evidence Extraction: Models and Corpora

科学证据主要在自由文本期刊文章中传播。因此，利用这些证据来做出决策或为政策提供信息需要仔细阅读相关文章并手动提取感兴趣的调查结果。不幸的是，这一过程非常耗时，而且还没有扩大规模，以满足科学证据库的急剧扩张带来的要求。这项工作旨在设计一种新的自然语言处理(NLP)方法，这种方法可以自动“阅读”并理解非结构化的已发表科学证据。这一点至关重要，因为政策制定者、照顾者和个人的决定应以所有相关已发表的科学证据为依据；但由于证据主要是非结构化的--因此不能直接采取行动--这在实践中是不可能的。以临床医学为例，它是本提案的目标领域(尽管框架和模型将推广到其他科学领域)。2015年，每天大约有100篇描述试验的文章发表。医疗保健专业人员不可能理解这一点，因此必须在没有充分考虑现有证据的情况下做出治疗决定。从大量非结构化文献中自动推断出哪些治疗方法实际上得到了证据的支持，将有助于更好地、以证据为基础的决定。为此，本研究试图设计NLP模型，能够将描述研究或试验的自然语言科学文章映射到结构化的“证据框架”，这些“证据框架”将所研究的干预和结果以及与此相关的报告结果编成代码。NLP技术目前还不能胜任这项任务。因此，该项目将支持核心方法学贡献，推动长篇文章的数据提取和机器阅读系统；这些影响将超出目前的激励应用。从技术角度来看，这项工作的重点是开发新的、可解释的(透明)神经网络模型，用于从长篇文章中提取和推理。具体地说，该项目旨在设计能够从自由文本中自动识别治疗方法和相关结果的模型，然后推断前者相对于后者的已报道的比较效果。这打破了现有语言技术能力的限制。特别是，这需要对个别的、可能很长的技术文档进行深入分析的模型。此外，模型透明度在这里至关重要，因为领域专家必须能够从文件中推断出证据声明的位置恢复过来。为该项目策划的新语料库(将与更广泛的社区共享)将促进对此类模型的核心NLP研究。为了实现上述方法论目标，领导该项目的研究人员将开发条件和动态的“注意力”神经模型。要探索的具体研究方法路线包括：(I)配备条件稀疏注意机制的模式，这些机制反映了科学的语篇结构，以实现对所报告证据的准确和透明的提取和推理。(Ii)神经序列标记模型，该模型对文本进行多次读取，利用迭代调整的条件文档表示作为全局上下文来通知局部预测。项目网站(http://www.byronwallace.com/evidence-extraction)提供对论文、数据集和其他项目输出的访问。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Scientific evidence is primarily disseminated in free-text journal articles. Drawing upon this evidence to make decisions or inform policies therefore requires perusing relevant articles and manually extracting the findings of interest. Unfortunately, this process is time-consuming and has not scaled to meet the demands imposed by the torrential expansion of the scientific evidence base. This work seeks to design novel Natural Language Processing (NLP) methods that can automatically "read" and make sense of unstructured published scientific evidence. This is critically important because decisions by policy-makers, care-givers and individuals should be informed by the entirety of the relevant published scientific evidence; but because evidence is predominantly unstructured -- and hence not directly actionable -- this is currently impossible in practice. Consider clinical medicine, an important example which serves as the target domain of this proposal (although the framework and models will generalize to other scientific areas). Roughly 100 articles describing trials were published every single day in 2015. Healthcare professionals cannot possibly make sense of this, and thus treatment decisions must be made without full consideration of the available evidence. Methods that can automatically infer from this torrential mass of unstructured literature which treatments are actually supported by the evidence would facilitate better, evidence-based decisions. Toward this end, this research seeks to design NLP models capable of mapping from natural language scientific articles describing studies or trials to structured "evidence frames" that codify the interventions and outcomes studied, and the reported findings concerning these. NLP technology is not presently up to this task. Therefore, this project will support core methodological contributions that will advance systems for data extraction and machine reading of lengthy articles; these will have impact beyond the present motivating application. From a technical perspective, the focus of this work concerns developing novel, interpretable (transparent) neural network models for extraction from and inference over lengthy articles. Specifically, this project aims to design models that can automatically identify treatments and associated outcomes from free-texts, and then infer the reported comparative effects of the former with respect to the latter. This pushes against limits of existing language technology capabilities. In particular, this necessitates models that perform deep analysis of individual, potentially lengthy, technical documents. Furthermore, model transparency is critical here, as domain experts must be able to recover from where in documents evidential claims were inferred. New corpora curated for this project (to be shared with the broader community) will facilitate core NLP research on such models. To realize the aforementioned methodological aims, the researchers leading this project will develop conditional and dynamic "attentive" neural models. Specific methodological lines of research to be explored include: (i) Models equipped with conditional, sparse attention mechanisms over textual units that reflect scientific discourse structure to achieve accurate and transparent extraction of, and inference concerning, reported evidence. (ii) Neural sequence tagging models that take multiple 'reads' of a text, exploiting iteratively adjusted conditional document representations as global context to inform local predictions. A project website (http://www.byronwallace.com/evidence-extraction) provides access to papers, datasets and other project outputs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.