BioThings Explorer: A platform for distributed knowledge integration across biomedical APIs

该提案将BioThings Explorer（BTE）平台描述为自主中继 代理（ARA）。大多数数据集成工作（包括在Translator中进行的许多工作） 建立集中的数据资源。这些工作通常涉及构建特定于资源的 数据摄取解析器，其输出被组装到知识图中， 加载到本地数据库（neo4j是Translator中的一个常用选项）。 BTE平台采用一种互补的数据集成方法， 由应用程序编程接口连接的知识提供者的分布式网络 （API）。BTE的概念验证实现现已完成（演示可在 [1，2]）。该实现有三个组件（图1）： （分布式KG）基于许多单独的Web API“在野外”，元知识图 描述每个API和BTE之间的输入和输出的兼容性的元KG python客户端，可跨API网络自动规划和执行查询。 相对于集中式解决方案，这种分布式模型提供了几个优点。首先，API 生态系统很容易被社区扩展，因为没有看门人控制 一个集中的资源。BTE系统的范围和产出将不断扩大， 随着社区维护的API组件注册表的不断完善， 成长其次，检索到的数据始终与源保持最新，并且不依赖于 具有频繁的同步调度。第三，分布式模型的可扩展性更强， 由于BTE客户端运行在每个用户自己的计算基础设施上， 任何可能成为单点故障的集中式组件。 BTE还通过使用语义精确的 API的注释，而不是强制对API本身进行任何更改。这些 API注释足够详细地描述API输入和输出，以支持启动API 调用并解释结果。相反，其他联邦方法（包括 转换器）要求API符合公共API结构。然而，要求，如 这些在历史上一直是广泛采用的重大障碍，而薄的API“包装器” 不兼容的API会带来额外的复杂性和技术故障点。 当前BTE实现的优点是易于添加新资源， 查询规划和执行的自动化。我们在这方面所面临的关键挑战 一项提案正在开发新的方法来对检索到的查询路径结果进行排名和排序 BTE查询。

This proposal describes the BioThings Explorer (BTE) platform as an Autonomous Relay Agent (ARA). Most data integration efforts (including many being pursued within Translator) are based on building centralized data resources. These efforts typically involve building resource-specific data ingestion parsers, the outputs of which are assembled into knowledge graphs and loaded into a local database (with neo4j being a popular option within Translator). The BTE platform takes a complementary approach to data integration, focusing on a distributed network of knowledge providers connected by application programming interfaces (APIs). A proof-of-concept implementation of BTE is now complete (with demos available in [1,2]). This implementation has three components (Figure 1): a distributed knowledge graph (distributed-KG) based on many individual web APIs "in the wild", a meta-knowledge graph (meta-KG) that describes compatibility of inputs and outputs between each API, and a BTE python client that automates the planning and execution of queries across the API network. Relative to centralized solutions, this distributed model offers several advantages. First, the API ecosystem is easily extensible by the community since there is no gatekeeper controlling access to a centralized resource. The scope and output of the BTE system will continuously and automatically improve as the community-maintained registry of API components continues to grow. Second, the data retrieved are always up-to-date with the source and not dependent on having a frequent synchronization schedule. Third, the distributed model is more scalable to heavy usage since the BTE client is run on each user's own computing infrastructure, bypassing any centralized component that could become a single point of failure. BTE also distinguishes itself from other federated API solutions by using semantically-precise annotations of APIs as they exist, rather than mandating any changes to the API itself. These API annotations describe API inputs and outputs in enough detail to support initiating the API call and interpreting the result. In contrast, other federated approaches (including some within Translator) require APIs to conform to a common API structure. However, requirements like these have historically been significant barriers to widespread adoption, and thin API "wrappers" around non-compliant APIs introduce additional complexity and technical points of failure. The strengths of the current BTE implementation are the ease of adding new resources and the automation of the query planning and execution. The critical challenge that we address in this proposal is developing new methods to rank and sort the query path results that are retrieved from BTE queries.