CICI: Data Provenance: Provenance-Based Trust Management for Collaborative Data Curation

数据驱动的科学不仅依赖于统计和机器学习，还依赖于人类的专业知识。随着收集数据以解决日益具有挑战性的科学和医学问题，需要相应地增加专家人力投入（策展和在某些情况下的注释）。 该项目通过开发协作数据管理解决了这一需求：它不依赖于少数专家，而是使具有不同专业知识的用户社区能够进行注释。 由于不同用户的注释质量会有所不同，因此开发了新的定量技术来评估每个用户的可信度，基于他们的行为，并将值得信赖的专家与不熟练和恶意的用户区分开来。算法被开发为基于用户的可信度来组合联合收割机用户的注释。 协同数据管理将极大地增加人工标注的数据量，这反过来又会为生命科学、医学等领域带来更好的大数据分析和检测算法。协同数据管理的核心问题在于用户标注质量的高度可变性，以及数据在注释时所采用的形式的可变性。该提案开发了一些技术，可以在不同的数据视图上采用不同用户所做的注释（诸如具有应用于信号的滤波器和变换的EEG显示），使用出处来推理注释如何与原始数据相关，以及推理每个用户的注释在该数据上的可靠性和可信度。 为了实现这一目标，研究首先定义了捕获时间和空间变化数据的数据和起源模型;用于计算和动态更新个人可靠性和可信度的新可靠性演算算法，基于他们的注释以及这些注释与公认专家和更广泛社区的注释的比较;以及一种名为PAL的高级语言，使研究人员能够实现和比较多种策略。 研究人员最初将在900多个用户的公共数据共享门户（1500多个EEG和其他需要注释的数据集）中开发和验证神经科学和时间序列数据的技术。项目团队后来将该技术扩展到其他数据模式，如成像和基因组学

Data-driven science relies not only on statistics and machine learning, but also on human expertise. As data are being collected to tackle increasingly challenging scientific and medical problems, there is need to scale up the amount of expert human input (curation and, in certain cases, annotation) accordingly. This project addresses this need by developing collaborative data curation: instead of relying on a small number of experts, it enables annotations to be made by communities of users of varying expertise. Since the quality of annotations by different users will vary, novel quantitative techniques are developed to assess the trustworthiness of each user, based on their actions, and to distinguish trustworthy experts from unskilled and malicious users. Algorithms are developed to combine users' annotations based on their trustworthiness. Collaborative data curation will greatly increase the amount of human annotated data, which will, in turn, lead to better Big Data analysis and detection algorithms for the life sciences, medicine, and beyond.The central problems of collaborative data curation lie in the high variability in the quality of users' annotations, and variability in the form the data takes when they annotate it. The proposal develops techniques to take annotations made by different users over different views of data (such as an EEG display with filters and transformations applied to the signal), to use provenance to reason about how the annotations relate to the original data, and to reason about the reliability and trustworthiness of each user's annotations over this data. To accomplish this, the research first defines data and provenance models that capture time- and space-varying data; novel reliability calculus algorithms for computing and dynamically updating the reliability and trustworthiness of individuals, based on their annotations and how these compare to annotations from recognized experts and the broader community; and a high-level language called PAL that enables the researchers to implement and compare multiple policies. The researchers will initially develop and validate the techniques on neuroscience and time series data, within a 900+ user public data sharing portal (with 1500+ EEG and other datasets for which annotations are required). The project team later expands the techniques to other data modalities, such as imaging and genomics