Symphony in the Latent Space: Provably Integrating High-dimensional Techniques with Non-linear Machine Learning Models
Symphony in the Latent Space: Provably Integrating High-dimensional Techniques with Non-linear Machine Learning Models
复制标题
DOI：
10.48550/arxiv.2212.00852
复制
发表时间：
2022-12
期刊：
影响因子：
--
通讯作者：
Qiong Wu;Jian Li;Zhenming Liu;Yanhua Li;Mihai Cucuringu
中科院分区：
文献类型：
--
作者：
Qiong Wu;Jian Li;Zhenming Liu;Yanhua Li;Mihai Cucuringu
本文回顾了构建涉及实体之间交互的机器学习算法，例如主动管理的投资组合中的金融资产之间的交互，或社交网络中用户之间的交互。我们的目标是预测多变量时间序列集合在此类应用中的未来演变（例如，金融资产的未来回报或Twitter帐户的未来流行度）。为这样的系统设计ML算法需要解决高维交互和非线性的挑战。现有的方法通常采用一个特设的方法来整合高维技术到非线性模型和最近的研究表明，这些方法在时间演化的相互作用系统的有效性值得怀疑。为此，我们提出了一个新的框架，我们称之为加性影响模型。在我们的建模假设下，我们证明了将高维交互的学习与非线性特征交互的学习解耦是可能的。为了学习高维的相互作用，我们利用基于内核的技术，与可证明的保证，嵌入在一个低维的潜在空间的实体。为了学习非线性特征-响应相互作用，我们概括了突出的机器学习技术，包括设计一种新的统计上合理的非参数方法和一种针对向量回归优化的集成学习算法。两个常见的应用程序的广泛实验表明，我们的新算法提供了显着更强的预测能力相比，标准和最近提出的方法。
This paper revisits building machine learning algorithms that involve interactions between entities, such as those between financial assets in an actively managed portfolio, or interactions between users in a social network. Our goal is to forecast the future evolution of ensembles of multivariate time series in such applications (e.g., the future return of a financial asset or the future popularity of a Twitter account). Designing ML algorithms for such systems requires addressing the challenges of high-dimensional interactions and non-linearity. Existing approaches usually adopt an ad-hoc approach to integrating high-dimensional techniques into non-linear models and recent studies have shown these approaches have questionable efficacy in time-evolving interacting systems. To this end, we propose a novel framework, which we dub as the additive influence model. Under our modeling assumption, we show that it is possible to decouple the learning of high-dimensional interactions from the learning of non-linear feature interactions. To learn the high-dimensional interactions, we leverage kernel-based techniques, with provable guarantees, to embed the entities in a low-dimensional latent space. To learn the non-linear feature-response interactions, we generalize prominent machine learning techniques, including designing a new statistically sound non-parametric method and an ensemble learning algorithm optimized for vector regressions. Extensive experiments on two common applications demonstrate that our new algorithms deliver significantly stronger forecasting power compared to standard and recently proposed methods.