Multi-Label Classification With Hyperdimensional Representations
Multi-Label Classification With Hyperdimensional Representations
复制标题
DOI：
10.1109/access.2023.3299881
复制
发表时间：
2023
期刊：
影响因子：
3.9
通讯作者：
Rishikanth Chandrasekaran;Fatemeh Asgareinjad;Justine Morris;Tajana Rosing
中科院分区：
文献类型：
--
作者：
Rishikanth Chandrasekaran;Fatemeh Asgareinjad;Justine Morris;Tajana Rosing
高维计算（HDC）是一种计算范式，利用一组神经合理的操作来利用高维矢量空间的数学特性将数据作为符号实体操纵。尽管HDC在认知任务中取得了巨大的成功，但其在复杂应用中的潜力（例如多标签分类）尚未探索。在本研究论文中，我们介绍了三种多标签分类方法，这些方法基于问题的复杂性在计算效率和准确性之间取得了平衡。我们提出的第一种方法是Power Set HD，这是一种转换方法，非常适合小型多标签分类，标签基数小于四个，标签设置尺寸小于十。第二种方法，即All HD，是另一种转换方法，适用于具有较高标签基数的更复杂任务，提供了更高的效率 - 精确率在Power Set HD上。但是，由于一VS-All HD的昂贵线性复杂性缩放量，我们提出了一种用于极限尺度任务的新型神经方法，称为Tinyxml HD。该方法通过将学习问题分解为多个子问题来学习高维表示，这些问题是通过基于梯度的优化来神经解决的。重要的是，tinyxml HD将模型的输出大小固定到Hypersensionality的尺寸，无论标签大小如何，因此在具有极大标签空间的数据集上评估时，仅通过一个小常数进行缩放。我们的方法在计算效率和准确性之间提供了有价值的权衡。我们表明，我们的方法在最先进的数据集上提供了16-60倍的加速，同时保持了可比的精度。此外，我们的方法在中等规模的任务上产生的模型较小56倍，而极端尺度数据集则高达836倍，这是模型大小的显着降低，同时仍然可以达到高精度。
Hyperdimensional computing (HDC) is a computational paradigm that leverages the mathematical properties of high-dimensional vector spaces to manipulate data as symbolic entities using a set of neurally plausible operations. Although HDC has demonstrated remarkable success in cognitive tasks, its potential in complex applications such as multi-label classificati has yet to be explored. In this research paper, we introduce three approaches to multi-label classification that strike a balance between computational efficiency and accuracy, based on the complexity of the problem. The first approach we propose is Power Set HD, a transformation method that is ideal for small-scale multi-label classification with label cardinality less than four and label set size less than ten. The second approach, One-vs-All HD, is another transformation method that is suitable for slightly more complex tasks with higher label cardinality, providing a better efficiency-accuracy trade-off over Power Set HD. However, due to the expensive linear complexity scaling of One-vs-All HD, we propose a novel neural approach called TinyXML HD for extreme scale tasks. This method learns hyperdimensional representations by decomposing the learning problem into multiple sub-problems, which are solved neurally through gradient-based optimization. Importantly, TinyXML HD fixes the output size of the model to the dimensionality of the hypervector, regardless of the label size, thereby scaling only by a small constant when evaluated on datasets with extremely large label spaces. Our approaches offer a valuable trade-off between computational efficiency and accuracy. We show that our methods provide a speedup of 16-60x on state of the art datasets, while maintaining comparable accuracy. Furthermore, our methods yield models that are 56x smaller on medium-scale tasks and up to 836x smaller on extreme-scale datasets, which is a significant reduction in model size while still achieving high accuracy.