QuantiCode: Intelligent infrastructure for quantitative, coded longitudinal data

这个跨学科项目旨在开发新的数据挖掘和可视化工具和技术，这将改变人们分析定量和编码纵向数据的能力。这类数据在许多部门都很常见。例如，健康数据使用数十万个读取代码（临床术语词库）的层次结构进行分类，分析师需要为临床调试决策提供商业智能，研究人员需要应对疾病风险分层建模等挑战。Sainsbury's等零售商销售50，000多种产品，并希望联合收割机结合来自购买、人口和其他来源的数据，以了解便利文化等行为现象，以指导投资并减少浪费。为了解决这些需求，公共和私营部门组织需要一个能够提供比当今更强大的分析工具的基础设施。今天的分析工具是有缺陷的，因为它们（a）在评估数据质量方面是粗糙的，（B）通常涉及的分析技术被设计为对聚合数据而不是细粒度数据进行操作，以及（c）通常使用起来很费力，这抑制了用户发现重要模式。QuantiCode项目将通过汇集统计、建模、可视化、用户评价和道德。该项目将设在利兹数据分析研究所（利达），该研究所设有ESRC消费者数据研究中心（500万英镑ES/L011891/1）和MRC医学生物信息学中心（700万英镑ES/L011891/1），并提供了一个开发设施，包括高性能计算（HPC），可视化和敏感数据的安全室。我们的项目将提供概念验证视觉分析系统，我们将与来自利达的合作伙伴和研究人员/外部用户的各种用户进行评估。在项目开始时，我们将与合作伙伴一起确定驱动我们研究细节的分析用例和需求，这些用例和需求分为四个工作包（WP）。WP 1（数据融合）将开发用于分析来自多个来源的细粒度数据的治理原则，实施工具以大幅减少链接这些来源的工作，并开发新技术以可视化完整性，一致性，可扩展性和数据质量的其他方面。WP 2（分析技术）和WP 3（抽象模型）是该项目的技术核心。WP 2将提供一种新的，强大的方法来建模数据，因为它们自然地出现在健康和零售数据中（不规则地分散或随时间采样），通过随机控制来扩展该方法以指导学习和资源使用，并开发一个低工作量的“提出问题”视觉界面，以大大降低调查数据和查找模式的人力。WP3（抽象模型）专注于数据粒度，并将提供一个工具，实现我们在WP 1中开发的治理原则的工作版本，以及用于探索抽象空间以创建适合每个分析方面的输入的新计算和交互技术。WP 4将在我们的概念系统证明的三个版本中实现上述工具和技术，与我们的合作伙伴和利达研究人员/用户一起评估每一项。这将确保我们的解决方案与具有挑战性的现实世界数据分析问题兼容并可扩展。成功的标准将是节省时间，增加分析范围，显著的见解，并解决以前不可行的分析类型-所有这些都与用户当前分析工具提供的基线进行比较。我们将鼓励通过展示，研讨会和许可安装在我们的合作伙伴的网站采用。该项目的遗产将包括作为利达基础设施的一个组成部分嵌入的工具、其持续开发的计划和研究路线图。

This cross-disciplinary project aims to develop novel data mining and visualization tools and techniques, which will transform people's ability to analyse quantitative and coded longitudinal data. Such data are common in many sectors. For example, health data is classified using a hierarchy of hundreds of thousands of Read Codes (a thesaurus of clinical terms), with analysts needing to provide business intelligence for clinical commissioning decisions, and researchers tacking challenges such modelling disease risk stratification. Retailers such as Sainsbury's sell 50,000+ types of products, and want to combine data from purchasing, demographic and other sources to understand behavioural phenomena such as the convenience culture, to guide investment and reduce waste.To solve these needs, public and private sector organisations require an infrastructure that provides far more powerful analytical tools than are available today. Today's analysis tools are deficient because they (a) are crude for assessing data quality, (b) often involve analysis techniques are designed to operate on aggregated, rather than fine-grained, data, and (c) are often laborious to use, which inhibits users from discovering important patterns.The QuantiCode project will address these deficiencies by bringing together experts in statistics, modelling, visualization, user evaluation and ethics. The project will be based in the Leeds Institute for Data Analytics (LIDA), which houses the ESRC Consumer Data Research Centre (£5m ES/L011891/1) and the MRC Medical Bioinformatics Centre (£7m ES/L011891/1), and provides a development facilities complete with high-performance computing (HPC), visualization and safe rooms for sensitive data. Our project will deliver proof of concept visual analytic systems, which we will evaluate with a wide variety of users drawn from our partners and researchers/external users based in LIDA.At the outset of the project we will engage with our partners to identify analysis use cases and requirements that drive the details of our research, which is divided into four workpackages (WPs). WP1 (Data Fusion) will develop governance principles for the analysis of fine-grained data from multiple sources, implement tools to substantially reduce the effort of linking those sources, and develop new techniques to visualize completeness, concordance, plausibility, and other aspects of data quality.WP2 (Analytical Techniques) and WP3 (Abstraction Models) are the project's technical core. WP2 will deliver a new, robust approach for modelling data as they appear naturally in health and retail data (irregularly dispersed or sampled over time), scaling that approach with stochastic control to guide learning and resource usage, and developing a low-effort 'question-posing' visual interface to drastically lower the human effort of investigating data and finding patterns. WP3 (Abstraction Models) focuses on data granularity, and will deliver a tool that implements a working version of the governance principles we develop in WP1, and new computational and interactive techniques for exploring abstraction spaces to create inputs suited to each aspect of analysis.WP4 will implement the above tools and techniques in three versions of our proof of concept system, evaluating each with our partners and LIDA researchers/users. This will ensure that our solutions are compatible with, and scale to, challenging real-world data analysis problems. Success criteria will be time saved, increased analysis scope, notable insights, and tackling previously unfeasible types of analysis - all compared against a baseline provided by users' current analysis tools. We will encourage adoption via showcases, workshops and licensed installations at our partners' sites. The project's legacy will include tools that are embedded as an integral part of the LIDA infrastructure, a plan for their on-going development, and a research roadmap.