ESPRESSO: Efficient Search over Personal Repositories - Secure and Sovereign

最近关于访问和处理个人数据的争议突出了个人对其个人数据主权的重要性，并导致了基于个人在线数据存储（pod）的应用程序开发的新范式，其中个人可以完全控制哪些应用程序可以访问他们的个人数据以及出于什么目的。新兴的框架和生态系统，如SOLID和Dataswift的生态系统，支持这种分散应用程序的开发，当相关个人授予访问权限时，可以访问存储在pod中的数据，为健康和福祉、社交网络和协作创作等领域的用户提供服务。然而，在大规模搜索或查询存储在pod中的数据时，这种去中心化会带来重大的性能挑战，这对于实现此类应用程序的潜力至关重要。当前通过提供关键字或短语搜索数据的技术需要为每个用户（或对所有可用pod共享相同访问权限的用户组）创建和维护单独的索引，从而导致存储、网络和计算成本的显著增加。类似地，通过提供数据库查询（例如使用SPARQL查询语言）来搜索数据的当前技术需要为每个用户创建和维护单独的元数据，以及在查询评估过程中控制对数据的访问和缓存的附加检查。目前的技术水平并没有提供有效生成和维护必要索引和元信息数据结构的技术，也没有在这种分散设置中大规模评估搜索查询和聚合查询结果的算法。ESPRESSO项目将研究、开发和评估适当的算法、索引和元信息数据结构，以实现跨分布式pod的大规模数据搜索。我们的技术将处理由每个pod所有者设置的不同访问权限和数据缓存需求。我们将处理基于关键字的搜索（可能需要最重要的（top-k）或所有搜索结果）和使用SPARQL的分布式查询。我们将通过使用SOLID框架和SOLID兼容数据生态系统（如基于HAT Microserver技术的Dataswift）实现的pod来评估我们的技术。这些pod的数量、分布和内容将由现有的信息检索基准确定，并扩展了有关所有者访问和缓存限制的esresso特定数据；以及从健康领域引出的现实场景的需求，它提供了大量的设置来调查和演示我们的新技术在pod数据搜索中实现的效率提升。我们不会使用真实的个人数据，而是使用从公开可用的人类受试者匿名数据集中获得的统计模式生成的合成数据。该项目将与新加坡的NExT++中心以及SOLID和HAT项目团队合作。我们将通过专门的活动与学术界和行业利益相关者积极接触。该项目的研究结果将为分布式系统、数据库、数字经济和网络安全以及创新分散应用程序的设计提供信息，并将有助于解决与数据主权和隐私相关的政策挑战。

Recent controversies over access and processing of personal data have highlighted the significance of the sovereignty of individuals over their personal data and are leading to new paradigms for application development based on personal online datastores (pods), where individuals have complete control over which applications can gain access to their personal data and for what purpose. Emerging frameworks and ecosystems, such as SOLID and the one by Dataswift, support the development of such decentralised applications which, when granted access by the individuals concerned, can access the data stored in pods to provide services to users in areas such as health and well- being, social networking, and collaborative authoring. However, this decentralisation presents significant performance challenges when searching or querying data stored in pods on a large scale, which is critical to fulfil the potential of such applications. The current state of the art in searching for data by supplying keywords or phrases would require separate indexes to be created and maintained for each user (or group of users who share identical access rights to all available pods), leading to significant increases in storage, network and computation costs. Similarly, the current state of the art in searching for data by supplying a database query (e.g. using the SPARQL query language) would require separate metadata to be created and maintained for each user, and additional checks to control access to, and caching of, data during the query evaluation process. The current state of the art does not provide techniques for the efficient generation and maintenance of the necessary indexes and meta-information data structures, nor algorithms for evaluating search queries and aggregating query results on a large scale in such decentralised settings. The ESPRESSO project will research, develop and evaluate appropriate algorithms, indexes and meta-information data structures to enable large-scale data search across distributed pods. Our techniques will handle varying access rights and data caching requirements, as set by each individual pod owner. We will address both keyword-based search where the most important (top-k) or all search results may be required, and distributed querying using SPARQL. We will evaluate our techniques over pods that are implemented using the SOLID framework and SOLID-compatible data ecosystems such as the one by Dataswift based on HAT Microserver technology. The numbers, distribution and content of these pods will be determined by existing Information Retrieval benchmarks, extended with ESRESSO-specific data about owners' access and caching restrictions; and by the requirements of real-world scenarios elicited from the Health domain, which provides a wealth of settings to investigate and demonstrate the efficiency gains in pod data search achieved by our new techniques. We will not use real personal data, but will employ synthetic data generated using statistical patterns obtained in the aggregate from publicly available anonymised datasets of human subjects. The project will collaborate with NExT++ centre in Singapore and with the SOLID and HAT project teams. We will actively engage with the academic community and industrial stakeholders through dedicated events. The project's findings will inform current research in distributed systems, databases, the digital economy and cybersecurity, as well as the design of innovative decentralised applications, and will help to address the policy challenges relating to data sovereignty and privacy.