III: Medium: Linear Algebra Operators in Databases to Support Analytic and Machine-Learning Workloads

机器学习工具已经在现代信息系统中变得无处不在。这些工具使用的数据输入通常来自关系数据库。这些工具产生的数据产出通常存储在数据库中，可用于随后的数据分析。然而，通常情况下，学习过程本身在数据库系统之外执行。这个项目调查了在数据库本身内执行更多机器学习工作的机会，避免了昂贵的(而且往往是冗余的)数据导出和导入。哥伦比亚大学团队将与Relational-AI和微软的研究人员合作，设计和构建两个名为Marque和Zork的互动开源系统。这些系统将使驻留在数据库中的信息的数据分析更加高效和有效。效率的提高将带来更快、更具成本效益的机器学习，在DBMS中执行ML将简化操作复杂性，并受益于DBMS的功能，如可伸缩性、访问控制和数据管理。最终，这项工作将扩大机器学习技术在广泛的数据密集型学科中的应用。MARQUE将是一个数据库管理系统，在查询处理引擎的上下文中支持机器学习原语，如线性代数运算。该系统将使用数据库中的嵌入式机器学习模型高效地编译SQL查询，将最先进的查询处理技术与高度工程化的线性代数算法相结合。Marque将允许机器学习管道本身的组件被制定为数据库中的操作，从而避免不必要的数据复制。可以使用像矩阵乘法这样的运算符的扩展来重新制定的传统SQL分析查询可以被优化，以使用涉及此类运算符的专用算法的高效执行计划。为了进一步支持数据库中的机器学习，项目调查人员将建立Zork，这是一个支持大规模机器学习的系统，将利用Marque提供的基础设施。Zork将通过处理数据的分解表示而不是显式地实现大型连接来扩展到非常大的数据集。该项目将为涉及传统关系运算符和广义线性代数运算符的查询开发新的创新查询处理技术。紧密集成将促进操作员内部和操作员之间的查询优化。使用该系统，将开发一系列完全在数据库管理系统内运行的机器学习技术，避免数据导出并简化数据隐私管理等问题。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Machine-learning tools have become ubiquitous in modern information systems. The data inputs used by these tools often originate from relational databases. The data outputs generated by those tools are often stored in databases, where they can be used for subsequent data analysis. Typically, however, the learning process itself is performed outside the database system. This project investigates the opportunity for performing more of the machine learning work within the database itself, avoiding expensive (and often redundant) data export and import. In partnership with researchers from Relational-AI and Microsoft, the Columbia University team will design and build two interacting open-source systems named MARQUE and ZORK. These systems will make data analysis more efficient and effective for database-resident information. Improved efficiency will lead to faster, more cost-effective machine learning, and executing ML within the DBMS will simplify operational complexity and benefit from DBMS features such as scalability, access control, and data management. Ultimately, this work will broaden the adoption of machine learning technologies in a wide range of data-intensive disciplines.MARQUE will be a database management system that supports machine learning primitives such as linear algebra operations within the context of a query processing engine. The system will efficiently compile SQL queries using embedded machine learning models within the database, combining state-of-the-art query processing techniques with highly engineered linear algebra algorithms. MARQUE will allow components of the machine-learning pipeline itself to be formulated as in-database operations, avoiding unnecessary data copying. Conventional SQL analytic queries that can be reformulated using extensions of operators like matrix multiplication can be optimized to use efficient execution plans involving specialized algorithms for such operators. To further support in-database machine learning, the project investigators will build ZORK, a system to support machine learning at scale that will make use of the infrastructure provided by MARQUE. ZORK will scale to very large datasets by processing factorized representations of the data rather than explicitly materializing large joins. This project will develop new and innovative query processing techniques for queries involving both conventional relational operators and generalized linear algebra operators. Tight integration will facilitate query optimization within and between operators. Using this system, a range of machine learning techniques will be developed that operate entirely within the database management system, avoiding data export and simplifying concerns such as data privacy administration.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.