SHF:Small:Language Support for Ad Hoc Data Processing

在每个商业、工程和科学学科中，工作人员都在将他们的知识数字化，希望使用计算方法对他们的数据进行分类、查询、筛选、搜索、诊断和可视化。虽然这一努力带来了显著的工业和科学进步，但它也产生了大量的特别数据(即，标准数据处理工具，如查询引擎、统计包、绘图工具或其他软件无法随时获得的数据)。即席数据给其用户带来了巨大的挑战，因为它通常是高度多样化的、文档很少的、充满错误的并且不断发展-然而即席数据也包含许多有价值的信息。这项研究的目标是开发能够有效管理即席数据的通用软件工具和技术。这项研究有可能对社会产生广泛的影响，因为它极大地提高了工业数据分析员、计算机系统管理员和学者的生产率，他们必须在日常工作中处理特别数据。这项研究的核心技术挑战涉及设计、实施和评估一种新的特定于领域的编程语言，以促进特别数据集的管理。这种新的编程语言将允许数据分析员指定特殊数据文件的结构、这些文件在文件系统中的排列方式以及与它们相关联的元数据。一旦规范完成，就可以将其用作数据集或生成数据处理工具的文档。这项研究还将涉及开发新的方法，使用户能够快速而准确地生成规范，而不必真正手工写下所有细节。最后，该研究将开发新的算法，以有效地实现生成的数据处理工具。

In every business, engineering endeavour and scientific discipline, workers are digitizing their knowledge with the hope of using computational methods to categorize, query, filter, search, diagnose, and visualize their data. While this effort is leading to remarkable industrial and scientific advances, it is also generating enormous amounts of ad hoc data (i.e., that data for which standard data processing tools such as query engines, statistical packages, graphing tools, or other software is not readily available). Ad hoc data poses tremendous challenges to its users because it is often highly varied, poorly documented, filled with errors, and continuously evolving --- yet ad hoc data also contains much valuable information. The goal of this research is to develop general-purpose software tools and techniques capable of managing ad hoc data efficiently. This research has the potential for a broad impact on society by dramatically improving the productivity of industrial data analysts, computer systems administrators and academics who must deal with ad hoc data on a day-to-day basis.The central technical challenge of the research involves designing, implementing and evaluating a new domain-specific programming language that facilitates the management of ad hoc data sets. This new programming language will allow data analysts to specify the structure of ad hoc data files, how those files are arranged in a file system and what meta-data is associated with them. Once a specification is complete, it will be possible to use it as documentation for the data set or for generating data-processing tools. The research will also involve developing new methods for enabling users to generate specifications quickly and accurately, without actually having to write down all of the details by hand. Finally, the research will develop new algorithms for implementing the generated data-processing tools efficiently.