Efficient Static and Dynamic In-Database Tensor Decompositions on Chunk-Based Array Stores
Efficient Static and Dynamic In-Database Tensor Decompositions on Chunk-Based Array Stores
复制标题
基于块的数组存储上的高效静态和动态数据库内张量分解
DOI：
10.1145/2661829.2661864
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
Mijung Kim
中科院分区：
文献类型：
--
作者：
Mijung Kim
随着相关数据集变得越来越大，现有的用于张量分解的内存中方案变得越来越无效，相反，需要与内存无关的解决方案，例如数据库中的分析。在本文中，我们提出了在基于块的数组数据存储上高效实现数据库内张量分解的技术。提出的静态和增量数据库内张量分解算子及其优化解决了在处理大张量和高阶张量数据时受主存限制的限制。首先，我们讨论了如何在基于块的数据存储系统上高效地实现交替最小二乘运算。其次，我们考虑了数据更新频繁的场景，证明了压缩矩阵乘法技术可以有效地降低增量张量分解的维护成本。据我们所知，本文首次尝试开发高效和优化的数据库内张量分解操作。我们在不适合可用内存的张量数据集上对所提出的算法进行了评估，结果表明，所提出的技术显着提高了这种核心数据分析的可扩展性。
As the relevant data sets get large, existing in-memory schemes for tensor decomposition become increasingly ineffective and, instead, memory-independent solutions, such as in-database analytics, are necessitated. In this paper, we present techniques for efficient implementations of in-database tensor decompositions on chunk-based array data stores. The proposed static and incremental in-database tensor decomposition operators and their optimizations address the constraints imposed by the main memory limitations when handling large and high-order tensor data. Firstly, we discuss how to implement alternating least squares operations efficiently on a chunk-based data storage system. Secondly, we consider scenarios with frequent data updates and show that compressed matrix multiplication techniques can be effective in reducing the incremental tensor decomposition maintenance costs. To the best of our knowledge, this paper presents the first attempt to develop efficient and optimized in-database tensor decomposition operations. We evaluate the proposed algorithms on tensor data sets that do not fit into the available memory and results show that the proposed techniques significantly improve the scalability of this core data analysis.