Explainable Natural Language Inference over Cancer Clinical Trial Texts

该项目旨在开发专门的自然语言推理（NLI）技术，以支持对癌症临床试验（CCT）报告的推理。该项目旨在解决以下研究问题：1：我们如何构建基于推理的多跳NLI模型，能够支持CCT报告的推理？设计并构建了一个名为NLI 4CT的NLI CCT数据集来测试此任务上的各种模型，在NLI 4CT上运行了一个共享任务，该共享任务的第二次迭代将在NLI 4CT的更新版本上运行。从肿瘤学本体中提取关系和定义，并编码为双曲线和欧几里得模型，目前正在评估中，使用大型语言模型（LLM）检索CCT报告以进行患者资料查询。1.2：我们如何构建高质量的解释以支持CCT的多跳NLI模型预测？我们将尝试用可微凸优化来解释构造，从相关领域事实构建图形，并提取基于约束的子图来模拟自然的解释。1.3：我们如何开发能够为CCT解释数值NLI的模型？实验将在数值NLI 4CT实例上进行，测试自动形式化，LLM和SymPy求解器。2：我们如何定量和定性地描述我们的模型在RQ 1 -1.3方面的行为？我们已经测试了F1分数和平均精度，以及定义两个新的定性措施，一致性和忠实性，设计用于因果分析的NLI模型的定量。定性地，我们探索双曲嵌入空间的遍历，执行中点分析，并研究加法/乘法行为。额外的实验将包括探测，分类分析，消融，概括和对抗性研究

The project aims to develop specialized Natural Language Inference (NLI) techniques to support inference over cancer clinical trial (CCT) reports. The project aims to address the following research questions:1: How can we construct explanation-based multi-hop NLI models capable of supporting inference over CCT reports? An NLI CCT dataset called NLI4CT was designed and constructed to test various models on this task, a shared task was run on NLI4CT, and a second iteration of this shared task will be run on an updated version of NLI4CT.1.1: How can we encode medical definitions and CCT reports to support explanation-based multi-hop NLI? Relations and definitions were extracted from an oncology ontology, and encoded into a hyperbolic and Euclidean model, currently under evaluation, with additional experimentation on the retrieval of CCT reports for patient profile queries using Large Language Models (LLM).1.2: How can we construct high quality explanations to support model predictions over multi-hop NLI for CCTs? We will experiment with Differentiable Convex Optimization for explanation construction, building graphs from relevant domain facts, and extracting constraint-based subgraphs to simulate natural explanations.1.3: How can we develop models capable of explainable numerical NLI for CCTs?Experiments will be carried out on numerical NLI4CT instances, testing Auto formalisation, LLMs and SymPy solvers.2: How can we quantitatively and qualitatively characterise the behaviour of our model with regards to RQ1-1.3? Quantitatively we have tested with F1 score and Mean Average Precision, as well as defining two novel qualitative measures, Consistency and Faithfulness, designed for causal analyses of NLI models. Qualitatively, we explore the traversal of the hyperbolic embedding space, performing midpoint analysis, and studying additive/multiplicative behaviours. Additional experiments will include probing, categorical analysis, ablation, generalisation and adversarial studies