Prototype memory and attention mechanisms for few shot image generation
Prototype memory and attention mechanisms for few shot image generation
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Tianqin Li;Zijie Li;Andrew Luo;Harold Rockwell;A. Farimani;T. Lee
中科院分区：
文献类型：
--
作者：
Tianqin Li;Zijie Li;Andrew Luo;Harold Rockwell;A. Farimani;T. Lee
最近的发现表明，猕猴初级视觉皮层（V1）表层的神经代码是复杂的，多样的和超稀疏的。这让我们思考这些“祖母细胞”的计算优势和功能作用。“在这里，我们提出这样的细胞可以作为原型记忆先验，在大脑中的图像生成过程中偏置和塑造分布式特征处理。这些记忆原型通过动量在线聚类来学习，并通过基于记忆的注意力操作来利用。结合这一机制，我们提出了记忆概念注意（莫卡），以提高少拍图像生成质量。我们发现，具有注意力机制的原型记忆可以提高图像合成质量，学习可解释的视觉概念簇，并提高模型的鲁棒性。我们的研究结果表明，这些超稀疏的复杂特征检测器可以作为原型的记忆先验调制的视觉系统中的图像合成过程的想法的可行性。
Recent discoveries indicate that the neural codes in the superficial layers of the primary visual cortex (V1) of macaque monkeys are complex, diverse and super-sparse. This leads us to ponder the computational advantages and functional role of these “grandmother cells." Here, we propose that such cells can serve as prototype memory priors that bias and shape the distributed feature processing during the image generation process in the brain. These memory prototypes are learned by momentum online clustering and are utilized through a memory-based attention operation. Integrating this mechanism, we propose Memory Concept Attention ( MoCA ) to improve few shot image generation quality. We show that having a prototype memory with attention mechanisms can improve image synthesis quality, learn interpretable visual concept clusters, and improve the robustness of the model. Our results demonstrate the feasibility of the idea that these super-sparse complex feature detectors can serve as prototype memory priors for modulating the image synthesis processes in the visual system.