Recovery and Generalization in Over-Realized Dictionary Learning
Recovery and Generalization in Over-Realized Dictionary Learning
复制标题
DOI：
--
复制
发表时间：
2020-06
期刊：
影响因子：
--
通讯作者：
Jeremias Sulam;Chong You;Zhihui Zhu
中科院分区：
文献类型：
--
作者：
Jeremias Sulam;Chong You;Zhihui Zhu
在二十多年的研究中，字典学习领域已经收集了大量成功的应用程序，只有在与底层字典相同的模型类中进行优化时，才知道模型恢复的理论保证。这项工作的特点是令人惊讶的现象，字典恢复可以通过搜索更大的过度实现模型的空间来促进。这种观察是普遍的，并且与所使用的特定字典学习算法无关。我们在实践中彻底证明了这一点，并通过将恢复措施与泛化界限联系起来，对这一现象进行了理论分析。我们进一步表明，一个有效的和可证明正确的蒸馏机制可以用来恢复正确的原子从过度实现的模型。因此，我们的元算法提供了字典估计，并始终更好地恢复地面实况模型。
In over two decades of research, the field of dictionary learning has gathered a large collection of successful applications, and theoretical guarantees for model recovery are known only whenever optimization is carried out in the same model class as that of the underlying dictionary. This work characterizes the surprising phenomenon that dictionary recovery can be facilitated by searching over the space of larger over-realized models. This observation is general and independent of the specific dictionary learning algorithm used. We thoroughly demonstrate this observation in practice and provide a theoretical analysis of this phenomenon by tying recovery measures to generalization bounds. We further show that an efficient and provably correct distillation mechanism can be employed to recover the correct atoms from the over-realized model. As a result, our meta-algorithm provides dictionary estimates with consistently better recovery of the ground-truth model.