Dataflow representation of data analyses: Toward a platform for collaborative data science
Dataflow representation of data analyses: Toward a platform for collaborative data science
复制标题
数据分析的数据流表示：迈向协作数据科学平台
DOI：
10.1147/jrd.2017.2736278
复制
发表时间：
2017
影响因子：
1.3
通讯作者：
Kush R. Varshney
中科院分区：
文献类型：
--
作者：
Evan Patterson;R. McBurney;H. Schmidt;Ioana Baldini;A. Mojsilovic;Kush R. Varshney
数据科学在解决当今科学和社会挑战方面发挥着越来越重要的作用。为了促进治疗多发性硬化症的进展，加速治愈项目创建了一个开放的多发性硬化症患者生物学和调查数据库。在其他领域也正在建立类似的大规模储存库。随着开放的、数据驱动的科学模型的激增，研究界对用于协作数据科学的云平台的需求日益增长。这样的平台应该促进领域专家和数据科学家之间的合作，并拥有组织、推荐和操纵数据分析的人工智能能力。在本文中，我们提出了一些基于这一愿景的基础技术。我们的系统自动从数据分析中提取高级别的Bronchlow图。此图描述了数据如何通过分析管道流动，包括使用哪些统计方法以及它们如何组合在一起。该系统不需要数据分析师的特殊注释，并使用标准工具（如Scikit-learn和Statsmodels）使用Python编写的分析。在本文中，我们解释了我们的系统是如何工作的，以及它如何适应我们对协作数据科学平台的更大愿景。
Data science plays an increasingly important role in solving today's scientific and social challenges. To promote progress toward a cure for multiple sclerosis, the Accelerated Cure Project has created an open repository of biological and survey data on patients with multiple sclerosis. Similar large-scale repositories are being created in other domains. As the open, data-driven model of science proliferates, the research community faces a growing need for a cloud platform for collaborative data science. Such a platform should facilitate collaboration between domain experts and data scientists and possess artificial intelligence capabilities for organizing, recommending, and manipulating data analyses. In this paper, we present some foundational technologies motivated by this vision. Our system automatically extracts a high-level dataflow graph from a data analysis. This graph describes how data flows through an analysis pipeline, including which statistical methods are used and how they fit together. The system requires no special annotations from the data analyst and consumes analyses written in Python using standard tools, such as Scikit-learn and Statsmodels. In this paper, we explain how our system works and how it fits into our larger vision for a collaborative data science platform.
DOI：
10.1002/asi.23329
发表时间：
2015-11-01
影响因子：
3.5
作者：
Bornmann, Lutz;Mutz, Ruediger
通讯作者：
Mutz, Ruediger