NSF-BSF: III: Small: Data Driven Schema

数据需要系统地、严格地组织起来。 有关消费品的数据放在一个表中，有关微生物的数据放在另一个表中。 这使得人类和计算机更容易存储数据，检索和查询数据，并更新数据，但今天人们经常会发现大量嘈杂，不一致，不完整的数据，这些数据是不可能严格组织的。 这些数据的庞大数量使其非常有价值，但如果没有适当的组织，它的效用有限。 这个项目开发了组织嘈杂的，不一致的，不完整的数据的方法，并开发了存储，查询和更新这些数据的技术。 它的研究结果将告知组织如何组织和使用大量的噪声data.This项目开发了一种技术，用于近似模式发现的噪声data.For规范化的数据，根据此模式，并提高查询处理。 输入由一个单一的，大的关系，这可能是嘈杂的，不一致的，不完整的，和系统自动发现一些候选模式，可以代表数据的损失最小，并具有高效用的下游任务。 每个模式都与一个信息理论分数相关联，它表示当我们根据该模式表示数据时可能丢失的信息量。 然后，该项目研究了查询存储在近似模式中的数据的新方法，通过显式记录模式生成的“虚假元组”的数量，或者通过使用概率来量化查询答案的置信度。 该项目中探索的技术将联合收割机信息理论与图形算法和查询优化相结合。该奖项反映了NSF的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

Data needs to be organized systematically and rigorously. Data about consumer products goes into one table, and data about micro-organisms goes into a different table. This makes it easier for humans and computers to store the data, to retrieve and query it, and to update it. But today one often finds large amounts of noisy, inconsistent, incomplete data, which are impossible to organize rigorously. The sheer volume of this data makes it very valuable, yet it is of limited utility without proper organization. This project develops methods for organizing noisy, inconsistent, incomplete data, and develops techniques for storing, querying, and updating such data. Its findings will inform organizations on how to organize and use large amounts of noisy data.This project develops a technique for approximate schema discovery for noisy data, for normalizing the data according to this schema, and for improving query processing. The input consists of a single, large relation, which may be noisy, inconsistent, incomplete, and the system discovers automatically a few candidate schemas that can represent the data with minimal loss and with high utility for downstream tasks. Each schema is associated with an information-theoretic score, which represents the amount of information that may be lost when we represent the data according to that schema. Then, the project researches new approaches for querying the data stored in an approximate schema, by either recording explicitly the number of "spurious tuples" generated by the schema, or by using probabilities to quantify the degree of confidence in the query's answer. The techniques explored in this project combine information theory with graph algorithms and with query optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.