VADA: Value Added Data Systems -- Principles and Architecture

数据无处不在，由越来越多的应用程序、设备和用户生成，但格式、语义和质量几乎没有保证。数据驱动创新的经济潜力巨大，据经济与商业研究中心估计，2017年将达到400亿英镑。为了实现这一潜力，并提供有意义的数据分析，数据科学家必须首先花费大量时间（估计为50%至80%）在“数据争吵”-收集，重组和清理数据的过程。这种沉重的代价是由于所谓的大数据的四个V：Volume -数据的规模，Velocity -变化的速度，Variety -不同形式的数据，Veracity -数据的不确定性。迫切需要为数据科学家提供新一代的工具，以释放数据资产的潜力，并显着减少数据争论的组成部分。由于许多传统工具不再适用于4V的环境，因此需要进行彻底的范式转换。该提案旨在通过增加数据的价值，通过在充分了解数据和用户上下文的环境中处理数据管理任务，以及通过以尚未尝试但在当今数据驱动的经济中许多创新公司迫切需要的方式紧密整合关键数据管理任务来实现这种范式转变。瓦达研究计划将定义增值数据系统的原则和解决方案，支持用户发现、提取、整合、访问和解释与其问题相关的数据。在这样做时，它使用用户的上下文，例如，在完整性和正确性之间的权衡方面的要求，以及数据上下文，例如，其可得性、成本、来源和质量。用户上下文不仅描述了哪些数据是相关的，还描述了它必须展示的属性，以满足目的。因此，增加数据的价值涉及到尽最大努力向用户提供数据，同时沿着关于所提供数据的质量和来源的全面信息。用户可以对获得的结果提供反馈，从而更改所有数据管理任务，从而持续改善用户体验。建立增值数据系统背后的原则需要一种革命性的数据管理方法，并通过在数据提取、数据集成、数据质量、出处、查询应答和推理方面的相互关联的研究提供信息。这将使这些领域中的每一个都能从与其他领域的协同增效中受益。研究已经在这些子学科中开发出了有针对性的结果;瓦达开发这些专业的方式既改变了子学科中的技术，又能够开发将它们结合在一起以增加数据价值的架构。研究领域的商业重要性已得到广泛认可。瓦达计划将大学研究人员与迫切需要新一代数据管理工具的商业合作伙伴聚集在一起。他们将通过资助研究人员和学生、为研究合作提供大量员工时间、支持实习、接待访客、贡献具有挑战性的现实生活案例研究、分享经验以及参加技术会议来为该计划做出贡献。这些合作伙伴既是数据管理技术的开发商（LogicBlox、微软、Neo），也是医疗保健（科视Christie）、电子商务（LambdaTek、PricePanda）、金融（AllianceBernstein）、社交网络（Facebook）、安全（Horus）、智能城市（FutureEverything）和电信（华为）等领域的数据用户组织。

Data is everywhere, generated by increasing numbers of applications, devices and users, with few or no guarantees on the format, semantics, and quality. The economic potential of data-driven innovation is enormous, estimated to reach as much as £40B in 2017, by the Centre for Economics and Business Research. To realise this potential, and to provide meaningful data analyses, data scientists must first spend a significant portion of their time (estimated as 50% to 80%) on "data wrangling" - the process of collection, reorganising, and cleaning data.This heavy toll is due to what is referred as the four V's of big data: Volume - the scale of the data, Velocity - speed of change, Variety - different forms of data, and Veracity - uncertainty of data. There is an urgent need to provide data scientists with a new generation of tools that will unlock the potential of data assets and significantly reduce the data wrangling component. As many traditional tools are no longer applicable in the 4 V's environment, a radical paradigm shift is required. The proposal aims at achieving this paradigm shift by adding value to data, by handling data management tasks in an environment that is fully aware of data and user contexts, and by closely integrating key data management tasks in a way not yet attempted, but desperately needed by many innovative companies in today's data-driven economy.The VADA research programme will define principles and solutions for Value Added Data Systems, which support users in discovering, extracting, integrating, accessing and interpreting the data of relevance to their questions. In so doing, it uses the context of the user, e.g., requirements in terms of the trade-off between completeness and correctness, and the data context, e.g., its availability, cost, provenance and quality. The user context characterises not only what data is relevant, but also the properties it must exhibit to be fit for purpose. Adding value to data then involves the best effort provision of data to users, along with comprehensive information on the quality and origin of the data provided. Users can provide feedback on the results obtained, enabling changes to all data management tasks, and thus a continuous improvement in the user experience.Establishing the principles behind Value Added Data Systems requires a revolutionary approach to data management, informed by interlinked research in data extraction, data integration, data quality, provenance, query answering, and reasoning. This will enable each of these areas to benefit from synergies with the others. Research has developed focused results within such sub-disciplines; VADA develops these specialisms in ways that both transform the techniques within the sub-disciplines and enable the development of architectures that bring them together to add value to data.The commercial importance of the research area has been widely recognised. The VADA programme brings together university researchers with commercial partners who are in desperate need of a new generation of data management tools. They will be contributing to the programme by funding research staff and students, providing substantial amounts of staff time for research collaborations, supporting internships, hosting visitors, contributing challenging real-life case studies, sharing experiences, and participating in technical meetings. These partners are both developers of data management technologies (LogicBlox, Microsoft, Neo) and data user organisations in healthcare (The Christie), e-commerce (LambdaTek, PricePanda), finance (AllianceBernstein), social networks (Facebook), security (Horus), smart cities (FutureEverything), and telecommunications (Huawei).