III: Small: Promoting Reuse and Retargeting in Data Science

过去十年，数据的可获得性发生了翻天覆地的变化。设备和数据获取工具使获取新数据变得容易，云托管和“数据湖”使集中和共享文件变得容易，云数据分析和机器学习工具推动了对这些数据进行集成并从中提取价值的愿望。数据科学家必须在过多的跨越用户、组和组织的数据集、版本和表示法中进行操作。他们经常做多余的工作，因为他们没有有效的方法来找到适当的资源来重用和重定目标到新的应用程序。数据科学家需要一套全面的数据管理工具来发现、标准化和重用数据湖中的现有资源。拟议的工作将满足这些需求，并将加强广泛使用的Jupyter笔记本和JupyterLab数据科学环境。这些工具还将与PI现有的生命科学合作、神经科学家(包括拥有3000多名用户的IEEG.org门户网站)、遗传学家和放射科医生进行整合和评估。这项拟议的工作将利用宾夕法尼亚大学的努力，让女性学生和那些来自代表性不足的群体参与暑期研究；这项研究将被纳入宾夕法尼亚大学数据科学硕士项目和在线学位课程的课程。PI将开发朱诺系统，这是一个新的数据湖数据管理平台。它将为源、中间和最终数据集和数据产品提供高效的版本化存储，使索引、搜索和比较不同的结果变得容易。一套丰富的搜索工具和排序算法用于获取现有的表碎片、模式或任务，并查找相关的表，这将使用户更容易重用以前工作的结果。该项目将开发从过去的数据模式和处理管道中学习的技术，并帮助用户设计不需要从头开始的新技术，使用户更容易重复使用以前开发的数据模型和工具。它还将开发用于分析用户社区执行的计算的技术，并开发用于共享的模式和表示，使基于数据实际使用方式的数据建模变得更容易。最后，研究人员将创建模型和工具，将有限的数据管道类从一个数据处理平台重新定位到另一个数据处理平台，使跨不同语言和工具的代码共享和重用变得更容易。Juneau将可供广大用户访问，因为它将位于广受欢迎的Jupyter笔记本和JupyterLab平台下，这些平台可以部署在本地或云上。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The past decade has brought a sea change in the availability of data. Devices and data acquisition tools make it easy to acquire new data, cloud hosting and "data lakes" make it easy to centralize and share files, and cloud data analytics and machine learning tools have driven a desire to integrate and extract value from that data. Data scientists must operate within a plethora of datasets, versions, and representations that span users, groups, and organizations. They often end up doing redundant work because they have no effective way of finding appropriate resources to reuse and retarget to new applications. Data scientists need a set of holistic data management tools to find, standardize, and reuse the existing resources in the data lake. The proposed work will address these needs, and it will enhance the widely-used Jupyter Notebook and JupyterLab data science environments. The tools will also be integrated and evaluated with the PI's existing life-science collaborations, with neuroscientists (including the IEEG.org portal with over 3000 users), geneticists, and radiologists. The proposed work will leverage Penn's efforts to involve female students, and those from underrepresented groups, in summer research; and the research will be incorporated into the curriculum of Penn's Data Science Master's program and online degree programs.The PI will develop the Juneau system, a novel data management platform for data lakes. It will provide efficient versioned storage of source, intermediate, and final datasets and data products, making it easy to index, search, and compare different results. A rich set of search tools and ranking algorithms for taking an existing table fragment, schema, or task, and finding related tables will make it easier for users to reuse the results of prior work. The project will develop techniques that learn from past data schemas and processing pipelines, and help users design new ones that do not need to start "from scratch," making it easier for users to reuse the data models and tools that were previously developed. It will also develop techniques for analyzing computations performed by the user community, and developing schemas and representations for sharing, making it easier to promote data modeling based on how data is actually being used. Finally, the researchers will create models and tools for "retargeting" limited classes of data pipelines from one data processing platform to another, making it easier to share and reuse code across different languages and tools. Juneau will be accessible to a broad user community because it will layer underneath the popular Jupyter Notebook and JupyterLab platform, which can be deployed locally or on the cloud.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.