Finding Related Tables in Data Lakes for Interactive Data Science.
Finding Related Tables in Data Lakes for Interactive Data Science.
复制标题
DOI：
10.1145/3318464.3389726
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Ives ZG
中科院分区：
文献类型：
--
作者：
Zhang Y;Ives ZG
许多现代数据科学应用程序都建立在数据湖、与模式无关的数据文件存储库和提供有限组织和管理功能的数据产品之上。需要将数据湖搜索功能构建到数据科学环境中，以便科学家和分析师可以找到对其手头任务有用的表格、模式、工作流程和数据集。我们为 Jupyter Notebook 数据科学平台开发搜索和管理解决方案，使科学家能够增强训练数据、找到提取的潜在特征、清理数据以及查找可连接或可链接的表。我们的核心方法还可以推广到计算任务涉及程序或脚本执行的其他设置。
Many modern data science applications build on data lakes, schema-agnostic repositories of data files and data products that offer limited organization and management capabilities. There is a need to build data lake search capabilities into data science environments, so scientists and analysts can find tables, schemas, workflows, and datasets useful to their task at hand. We develop search and management solutions for the Jupyter Notebook data science platform, to enable scientists to augment training data, find potential features to extract, clean data, and find joinable or linkable tables. Our core methods also generalize to other settings where computational tasks involve execution of programs or scripts.