Explainable Natural Language Inference over Cancer Clinical Trial Texts

该项目旨在开发专门的自然语言推理(NLI)技术，以支持对癌症临床试验(CCT)报告的推理。该项目旨在解决以下研究问题：1：如何构建能够支持对CCT报告的推理的基于解释的多跳NLI模型？设计和构建了名为NLI4CT的NLI CCT数据集，以测试该任务的各种模型，在NLI4CT上运行共享任务，该共享任务的第二次迭代将在NLI4CT的更新版本上运行。1.1：我们如何对医学定义和CCT报告进行编码，以支持基于解释的多跳NLI？关系和定义从肿瘤学本体中提取，并编码成双曲线和欧几里得模型，目前正在评估中，并在使用大语言模型(LLM)检索用于患者资料查询的CCT报告方面进行了额外的实验。1.2：我们如何构建高质量的解释来支持CCT的多跳NLI模型预测？我们将试验可微凸优化来构建解释，从相关的领域事实中构建图形，并提取基于约束的子图来模拟自然解释。1.3：我们如何才能为CCT开发能够解释的数值NLI的模型？实验将在数值NLI4CT实例上进行，测试自动形式化、LLMS和SymPy求解。2：我们如何定量和定性地描述我们的模型关于RQ1-1.3的行为？在定量上，我们用F1分数和平均精度进行了检验，并定义了两个新的定性度量，一致性和忠诚度，用于NLI模型的因果分析。定性地，我们探索了双曲嵌入空间的遍历，进行了中点分析，并研究了加法/乘法行为。其他实验将包括探测、分类分析、消融、概括和对抗性研究。

The project aims to develop specialized Natural Language Inference (NLI) techniques to support inference over cancer clinical trial (CCT) reports. The project aims to address the following research questions:1: How can we construct explanation-based multi-hop NLI models capable of supporting inference over CCT reports? An NLI CCT dataset called NLI4CT was designed and constructed to test various models on this task, a shared task was run on NLI4CT, and a second iteration of this shared task will be run on an updated version of NLI4CT.1.1: How can we encode medical definitions and CCT reports to support explanation-based multi-hop NLI? Relations and definitions were extracted from an oncology ontology, and encoded into a hyperbolic and Euclidean model, currently under evaluation, with additional experimentation on the retrieval of CCT reports for patient profile queries using Large Language Models (LLM).1.2: How can we construct high quality explanations to support model predictions over multi-hop NLI for CCTs? We will experiment with Differentiable Convex Optimization for explanation construction, building graphs from relevant domain facts, and extracting constraint-based subgraphs to simulate natural explanations.1.3: How can we develop models capable of explainable numerical NLI for CCTs?Experiments will be carried out on numerical NLI4CT instances, testing Auto formalisation, LLMs and SymPy solvers.2: How can we quantitatively and qualitatively characterise the behaviour of our model with regards to RQ1-1.3? Quantitatively we have tested with F1 score and Mean Average Precision, as well as defining two novel qualitative measures, Consistency and Faithfulness, designed for causal analyses of NLI models. Qualitatively, we explore the traversal of the hyperbolic embedding space, performing midpoint analysis, and studying additive/multiplicative behaviours. Additional experiments will include probing, categorical analysis, ablation, generalisation and adversarial studies