Neurally driven synthesis of learned, complex vocalizations.
Neurally driven synthesis of learned, complex vocalizations.
复制标题
神经驱动的合成学习，复杂的发声。
DOI：
10.1016/j.cub.2021.05.035
复制
发表时间：
2021-08-09
期刊：
影响因子：
--
通讯作者：
Gentner TQ
中科院分区：
文献类型：
--
作者：
Arneodo EM;Chen S;Brown DE 2nd;Gilja V;Gentner TQ
脑机接口（BMIs）有望恢复受损的运动功能，并作为研究学习运动技能的有力工具。虽然基于肢体的运动假体系统利用非人类灵长类动物作为重要的动物模型，但语音假体缺乏类似的动物模型，并且在神经接口技术，大脑覆盖范围和行为研究设计方面更加有限。鸣禽是一个有吸引力的模型学习复杂的声音行为。鸟鸣声与人类言语有许多独特的相似之处，它的研究使人们对发声运动技能的学习、执行和维持背后的多种机制和回路有了全面的了解。此外，鸣唱产生的生物力学与人类和一些非人类灵长类动物相似。在这里，我们展示了一个声乐合成器鸟鸣，实现映射的神经群体活动记录从电极阵列植入在运动前核HVC到低维压缩表示的歌曲，使用简单的计算方法，可实现在真实的时间。使用发声器官（鸣管）的生成生物力学模型作为这些映射的低维目标，可以合成与鸟自己的歌曲相匹配的发声。这些结果提供了概念证明，高维，复杂的自然行为可以直接从正在进行的神经活动合成。这可能会激发类似的方法，以假肢在其他物种利用知识的外围系统和时间结构的输出。鸣禽，像人类一样，需要控制一个复杂的发声器官来产生丰富的声音序列。阿内奥多等利用发声器官的生物力学知识和发声序列的结构，从记录的运动前神经活动中合成鸟鸣。
Brain Machine Interfaces (BMIs) hold promise to restore impaired motor function and serve as powerful tools to study learned motor skill. While limb-based motor prosthetic systems have leveraged nonhuman primates as an important animal model, speech prostheses lack a similar animal model and are more limited in terms of neural interface technology, brain coverage, and behavioral study design. Songbirds are an attractive model for learned complex vocal behavior. Birdsong shares a number of unique similarities with human speech, and its study has yielded general insight into multiple mechanisms and circuits behind learning, execution, and maintenance of vocal motor skill. In addition, the biomechanics of song production bear similarity to those of humans and some nonhuman primates. Here, we demonstrate a vocal synthesizer for birdsong, realized by mapping neural population activity recorded from electrode arrays implanted in the premotor nucleus HVC onto low-dimensional compressed representations of song, using simple computational methods that are implementable in real time. Using a generative biomechanical model of the vocal organ (syrinx) as the low-dimensional target for these mappings allows for the synthesis of vocalizations that match the bird’s own song. These results provide proof of concept that high-dimensional, complex natural behaviors can be directly synthesized from ongoing neural activity. This may inspire similar approaches to prosthetics in other species by exploiting knowledge of the peripheral systems and the temporal structure of their output. Songbirds, like humans, need to control a sophisticated vocal organ to produce rich vocal sequences. Arneodo et. al. use knowledge of the biomechanics of the vocal organ and the structure of the vocal sequence to synthesize birdsong from recorded premotor neural activity.
登录
查看更多内容
影响因子：
16.6
作者：
Elemans CP;Rasmussen JH;Herbst CT;Düring DN;Zollinger SA;Brumm H;Srivastava K;Svane N;Ding M;Larsen ON;Sober SJ;Švec JG
通讯作者：
Švec JG
DOI：
10.1126/science.aah6837
发表时间：
2016-12-09
期刊：
Science (New York, N.Y.)
影响因子：
--
作者：
Gadagkar V;Puzerey PA;Chen R;Baird-Daniel E;Farhang AR;Goldberg JH
通讯作者：
Goldberg JH
影响因子：
3.7
作者：
Assaneo MF;Trevisan MA;Mindlin GB
通讯作者：
Mindlin GB
DOI：
10.1109/tassp.1984.1164317
发表时间：
1984-01-01
期刊：
IEEE TRANSACTIONS ON ACOUSTICS SPEECH AND SIGNAL PROCESSING
影响因子：
--
作者：
GRIFFIN, DW;LIM, JS
通讯作者：
LIM, JS
影响因子：
2.4
作者：
Anderson, SE;Dave, AS;Margoliash, D
通讯作者：
Margoliash, D