BioThings Explorer: A platform for distributed knowledge integration across biomedical APIs

该提案将BioThings Explorer(BTE)平台描述为自主中继 代理(ARA)。大多数数据集成工作(包括许多在Translator内部进行的工作)都是 以建设集中的数据资源为基础。这些工作通常涉及构建特定于资源的 数据摄取解析器，其输出被组装成知识图和 加载到本地数据库中(在翻译器中，ne4j是一个流行的选项)。 BTE平台对数据集成采取了补充方法，重点是 通过应用编程接口连接的知识提供者的分布式网络 (API)。BTE的概念验证实施现已完成(演示可在 [1，2])。该实现有三个组件(图1)：分布式知识图 (分布式-KG)基于多个单独的Web API(在野外)，一个元知识图谱 (META-KG)，描述每个API和BTE之间的输入和输出的兼容性 在整个API网络中自动规划和执行查询的Python客户端。 相对于集中式解决方案，这种分布式模型提供了几个优势。首先，API 生态系统很容易被社区扩展，因为没有看门人控制 对集中式资源的访问。BTE系统的范围和产出将持续和 随着社区维护的API组件注册继续 成长。其次，检索到的数据始终与源保持同步，而不依赖于 具有频繁的同步时间表的。第三，分布式模型的可伸缩性更强 由于BTE客户端运行在每个用户自己的计算基础设施上，因此使用率很高 任何可能成为单点故障的集中式组件。 BTE还通过使用语义精确的 API存在时的注释，而不是强制对API本身进行任何更改。这些 API注释足够详细地描述API输入和输出，以支持启动API 调用并解释结果。相比之下，其他联合方法(包括内部的一些方法 Translator)要求API符合通用API结构。然而，像这样的要求 这些在历史上一直是广泛采用的重大障碍，而瘦API“包装器” 不合规的API带来了额外的复杂性和技术故障点。 当前BTE实施的优点是易于添加新资源和 查询计划和执行的自动化。我们在这方面应对的关键挑战 一项提案正在开发新的方法来对检索到的查询路径结果进行排名和排序 从BTE查询。

This proposal describes the BioThings Explorer (BTE) platform as an Autonomous Relay Agent (ARA). Most data integration efforts (including many being pursued within Translator) are based on building centralized data resources. These efforts typically involve building resource-specific data ingestion parsers, the outputs of which are assembled into knowledge graphs and loaded into a local database (with neo4j being a popular option within Translator). The BTE platform takes a complementary approach to data integration, focusing on a distributed network of knowledge providers connected by application programming interfaces (APIs). A proof-of-concept implementation of BTE is now complete (with demos available in [1,2]). This implementation has three components (Figure 1): a distributed knowledge graph (distributed-KG) based on many individual web APIs "in the wild", a meta-knowledge graph (meta-KG) that describes compatibility of inputs and outputs between each API, and a BTE python client that automates the planning and execution of queries across the API network. Relative to centralized solutions, this distributed model offers several advantages. First, the API ecosystem is easily extensible by the community since there is no gatekeeper controlling access to a centralized resource. The scope and output of the BTE system will continuously and automatically improve as the community-maintained registry of API components continues to grow. Second, the data retrieved are always up-to-date with the source and not dependent on having a frequent synchronization schedule. Third, the distributed model is more scalable to heavy usage since the BTE client is run on each user's own computing infrastructure, bypassing any centralized component that could become a single point of failure. BTE also distinguishes itself from other federated API solutions by using semantically-precise annotations of APIs as they exist, rather than mandating any changes to the API itself. These API annotations describe API inputs and outputs in enough detail to support initiating the API call and interpreting the result. In contrast, other federated approaches (including some within Translator) require APIs to conform to a common API structure. However, requirements like these have historically been significant barriers to widespread adoption, and thin API "wrappers" around non-compliant APIs introduce additional complexity and technical points of failure. The strengths of the current BTE implementation are the ease of adding new resources and the automation of the query planning and execution. The critical challenge that we address in this proposal is developing new methods to rank and sort the query path results that are retrieved from BTE queries.