Explicit-memory multiresolution adaptive framework for speech and music separation.
Explicit-memory multiresolution adaptive framework for speech and music separation.
复制标题
DOI：
10.1186/s13636-023-00286-7
复制
发表时间：
2023
期刊：
影响因子：
--
通讯作者：
中科院分区：
文献类型：
--
作者：
人类的听觉系统采用了许多原则来促进从复杂的声音混合中选择感知上分离的流。大脑利用输入的多尺度冗余表示，并使用记忆(或先验)来指导从输入混合中选择目标声音。此外，反馈机制改进了记忆结构，从而进一步提高了特定声音对象在动态背景中的选择性。本研究提出了一个统一的端到端计算框架，该框架模仿了适用于语音和音乐混合的声源分离的这些原则。虽然由于每个信号域的限制和特殊性，语音增强和音乐分离的问题通常是分开处理的，但目前的工作假设声源分离的共同原则是领域不可知的。在该方案中，并行和分层卷积路径将输入混合映射到冗余但分布的高维子空间，并利用时间相干的概念来选择属于在内存中抽象的目标流的嵌入。这些显性记忆通过来自输入观测的自我反馈进一步精炼，以提高系统在面对未知背景时的选择性。该模型为语音和音乐混合产生了稳定的源分离结果，并证明了外显记忆作为先验的强大表示的好处，从而指导从复杂输入中选择信息。
The human auditory system employs a number of principles to facilitate the selection of perceptually separated streams from a complex sound mixture. The brain leverages multi-scale redundant representations of the input and uses memory (or priors) to guide the selection of a target sound from the input mixture. Moreover, feedback mechanisms refine the memory constructs resulting in further improvement of selectivity of a particular sound object amidst dynamic backgrounds. The present study proposes a unified end-to-end computational framework that mimics these principles for sound source separation applied to both speech and music mixtures. While the problems of speech enhancement and music separation have often been tackled separately due to constraints and specificities of each signal domain, the current work posits that common principles for sound source separation are domain-agnostic. In the proposed scheme, parallel and hierarchical convolutional paths map input mixtures onto redundant but distributed higher-dimensional subspaces and utilize the concept of temporal coherence to gate the selection of embeddings belonging to a target stream abstracted in memory. These explicit memories are further refined through self-feedback from incoming observations in order to improve the system’s selectivity when faced with unknown backgrounds. The model yields stable outcomes of source separation for both speech and music mixtures and demonstrates benefits of explicit memory as a powerful representation of priors that guide information selection from complex inputs.
登录
查看更多内容
影响因子：
4.3
作者：
Elliott TM;Theunissen FE
通讯作者：
Theunissen FE
DOI：
10.1523/jneurosci.5297-12.2013
发表时间：
2013-03-27
期刊：
The Journal of neuroscience : the official journal of the Society for Neuroscience
影响因子：
--
作者：
Ding N;Simon JZ
通讯作者：
Simon JZ
影响因子：
3.5
作者：
Dyson, Benjamin J.;Ishfaq, Feraz
通讯作者：
Ishfaq, Feraz
影响因子：
25
作者：
Fritz, Jonathan B.;David, Stephen V.;Radtke-Schuller, Susanne;Yin, Pingbo;Shamma, Shihab A.
通讯作者：
Shamma, Shihab A.
DOI：
10.1073/pnas.1520027113
发表时间：
2016-07-05
影响因子：
11.1
作者：
Brady, Timothy F.;Stormer, Viola S.;Alvarez, George A.
通讯作者：
Alvarez, George A.