Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs
Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs
复制标题
DOI：
--
复制
发表时间：
2018-09
期刊：
影响因子：
--
通讯作者：
R. Murphy;Balasubramaniam Srinivasan;Vinayak A. Rao;Bruno Ribeiro
中科院分区：
文献类型：
--
作者：
R. Murphy;Balasubramaniam Srinivasan;Vinayak A. Rao;Bruno Ribeiro
我们考虑了序列（或多集函数）的置换不变函数的一个简单而全面的表示。我们的方法，我们称之为Janossy池，将置换不变函数表示为应用于输入序列所有重排序的置换敏感函数的平均值。这使我们能够利用丰富而成熟的关于置换敏感函数的文献来构建新颖而灵活的置换不变函数。如果天真地执行，Janossy池可能会在计算上令人望而却步。为了允许计算可跟踪性，我们考虑了三种近似：序列的规范排序，具有$k$阶相互作用的函数，以及具有随机排列的随机优化算法。我们的框架统一了文献中的各种现有工作，并提出了可能的建模和算法扩展。我们在实验中探索了一些方法，它们比当前最先进的方法表现出更高的性能。
We consider a simple and overarching representation for permutation-invariant functions of sequences (or multiset functions). Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence. This allows us to leverage the rich and mature literature on permutation-sensitive functions to construct novel and flexible permutation-invariant functions. If carried out naively, Janossy pooling can be computationally prohibitive. To allow computational tractability, we consider three kinds of approximations: canonical orderings of sequences, functions with $k$-order interactions, and stochastic optimization algorithms with random permutations. Our framework unifies a variety of existing work in the literature, and suggests possible modeling and algorithmic extensions. We explore a few in our experiments, which demonstrate improved performance over current state-of-the-art methods.