Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses
Hybrid Neural Autoencoders for Stimulus Encoding in Visual and Other Sensory Neuroprostheses
复制标题
DOI：
--
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Jacob Granley;Lucas Relic;M. Beyeler
中科院分区：
文献类型：
--
作者：
Jacob Granley;Lucas Relic;M. Beyeler
感觉神经假体是一种很有前途的技术，可以恢复失去的感觉功能或增强人类的能力。然而，由当前设备引起的感觉常常看起来是人为的和扭曲的。虽然目前的模型可以预测对电刺激的神经或感知反应，但最佳刺激策略解决了相反的问题：产生期望反应所需的刺激是什么？在这里，我们将其视为端到端优化问题，其中训练深度神经网络刺激编码器以反转近似底层生物系统的已知和固定的前向模型。作为概念证明，我们证明了这种混合神经自动编码器（HNA）在视觉神经假体中的有效性。我们发现，HNA产生高保真度的患者特定刺激，代表手写数字和日常物品的分割图像，并显着优于所有模拟患者的传统编码策略。总的来说，这是朝着为患有不可治愈的失明的人恢复高质量视力这一长期挑战迈出的重要一步，并可能成为各种神经假体技术的一个有前途的解决方案。
Sensory neuroprostheses are emerging as a promising technology to restore lost sensory function or augment human capabilities. However, sensations elicited by current devices often appear artificial and distorted. Although current models can predict the neural or perceptual response to an electrical stimulus, an optimal stimulation strategy solves the inverse problem: what is the required stimulus to produce a desired response? Here, we frame this as an end-to-end optimization problem, where a deep neural network stimulus encoder is trained to invert a known and fixed forward model that approximates the underlying biological system. As a proof of concept, we demonstrate the effectiveness of this hybrid neural autoencoder (HNA) in visual neuroprostheses. We find that HNA produces high-fidelity patient-specific stimuli representing handwritten digits and segmented images of everyday objects, and significantly outperforms conventional encoding strategies across all simulated patients. Overall this is an important step towards the long-standing challenge of restoring high-quality vision to people living with incurable blindness and may prove a promising solution for a variety of neuroprosthetic technologies.