xARA: ARA through Explainable AI

为响应NIH FOA OTA-19009《生物医学翻译器：开发》，我们建议 构建一个自主中继代理(ARA)，该代理可以表征和评估 从多个多尺度不同的知识提供者(KPS)返回的信息。 生物医学研究人员通过以下方式与知识提供商(KP)建立信任关系 频繁和持续的使用。随着时间的推移，熟悉感的发展推动了他们的理解和 洞察1)如何构建和调用更有效的查询，2)它们的结果质量 可能期望响应不同的查询参数和特征值，以及3)如何评估 特定查询结果的相关性。 尽管这种信息检索范式已经适度地服务于研究界 在过去，它是不可扩展的，KPS的数量、范围和复杂性正在以 惊人的速度(截至2019年1月，报告了1613个分子生物学数据库)。在这一切之中 不断变化的信息格局，生物医学研究人员现在有两个选择--要么 继续使用他们已经学会信任的少数KPS，但在可操作的方面仍然受到限制 他们将收到信息，或投入时间并接受使用一系列新的 很少或根本不熟悉的信息资源，因此有效性不确定。如果研究人员 现在将从NIH和行业赞助的大量信息资产中受益 现有的和不断扩大的新的信息检索和质量评估技术 必填项。 我们建议构建一个解释性自主中继代理(XARA)，该代理能够表征 查询结果通过对多尺度异质KPS返回的信息质量进行评级。 XARA将利用多重信息检索和可解释人工智能(XAI) 跨多个异类KP执行查询并按以下顺序对其结果进行排名的策略 质量和相关性，同时识别和解释以下各项之间的不一致 数据库获取相同的查询响应。为了兑现这一承诺，我们将利用基于案例的 使用生物医学数据(即BioBERT和CUSTOM)训练的推理和语言模型 通过Reactome和UniProt嵌入注释)，从而将查询分析提升到一个新的级别 和评估。 我们的策略将允许1)通过测试替代查询模式来填补信息空白 其产生不同表面句法但具有语义相关和可操作的概念， 2)对于给定的查询特征值要识别的不一致性，以及3)识别和 通过以下方式丰富的相似性度量消除或合并语义冗余的查询结果 基于案例的推理策略在可解释人工智能(XAI)社区中用于识别 机器学习模型的行为和性能。 这里提出的XARA功能将基于韦伯博士的实验室开发的策略 对于信息检索，当推理时希望获得更大的透明度 实验数据是我们的首要目标。我们的多机构团队由资深人士组成 研究人员和软件工程师在计算机和数据方面接受过正式培训并具有丰富的经验 科学、化学信息学、生物信息学、分子生物学和生物化学。 查询异类KPS的固有风险包括存在不一致的标签 在独特的KP数据结构中使用相同的生物医学概念。人工工程可能是 克服这些障碍是必要的，但对于初始阶段不会是一个重大挑战 原型机，因为只有两个有充分记录的KPS正在接受评估。另一个值得注意的风险是 UniProt和Reactome生成的单词嵌入的质量可能不是 足够，需要对像PubMed这样的生物医学文本进行进一步的文本分析，这是可行的 在我们项目计划的时间范围内。

In response to the NIH FOA OTA-19009 “Biomedical Translator: Development” we propose to build an Autonomous Relay Agent (ARA) that can characterize and rate the quality of information returned from multiple multiscale heterogeneous knowledge providers (KPs). Biomedical researchers develop a trust relationship with a knowledge provider (KP) through frequent and continued use. Over time a familiarity develops that drives their understanding and insight on 1) how to structure and invoke more effective queries, 2) the quality of the results they may expect in response to different query parameters and feature values, and 3) how to assess the relevancy of a specific query’s results. Although this information retrieval paradigm has served the research community moderately well in the past it is not scalable and the number, scope and complexity of KPs is increasing at a dramatic pace (1,613 molecular biology databases reported as of Jan. 2019). Within this ever changing information landscape, a biomedical researcher now has two choices -- either continue using the few KPs they have learned to trust but remain limited in the actionable information they will receive, or invest the time and accept the risk of using a range of new information resources with little or no familiarity and thus uncertain effectiveness. If researchers are to benefit from the vast array of NIH and industry sponsored information assets now available and expanding new information retrieval and quality assessment technologies will be required. We propose to build an Explanatory Autonomous Relay Agent (xARA) that can characterize query results by rating the quality of information returned from multi-scale heterogeneous KPs. The xARA will utilize multiple information retrieval and explainable Artificial Intelligence (xAI) strategies to perform queries across multiple heterogeneous KPs and rank their results by quality and relevancy while also identifying and explaining any inconsistencies among databases for the same query response. To deliver on this promise, we will utilize case-based reasoning and language models trained with biomedical data (i.e., BioBERT and custom annotation embeddings through Reactome and UniProt) permitting a new level of query profiling and assessment. Our strategies will permit 1) information gaps to be filled by testing alternative query patterns that produce different surface syntax yet possess semantically related and actionable concepts, 2) inconsistencies to be identified for a given query feature value, and 3) the identification and elimination or merging of semantically redundant query results via similarity metrics enriched by case-based reasoning strategies employed in the explainable AI (xAI) community to identify machine learning model behavior and performance. The xARA capabilities proposed herein will be based on strategies developed in Dr. Weber’s lab for information retrieval where the desire for greater transparency when reasoning over experimental data is our primary aim. Our multi-institutional team is comprised of senior researchers and software engineers formally trained and experienced in the computer and data sciences, cheminformatics, bioinformatics, molecular biology, and biochemistry. Inherent risks in querying heterogeneous KPs include the presence of inconsistent labeling of the same biomedical concept within unique KP data structures. Manual engineering may be necessary to overcome such hurdles, but will not be a significant challenge for the initial prototype, since only two well documented KPs are being evaluated. Another noteworthy risk is that the quality of word embeddings generated from UniProt and Reactome may not be sufficient, requiring further textual analysis of biomedical text like PubMed, which is feasible within the timeframe of our project plan.