Scene interpretation for sensory substitution and intelligent agents: a neurocomputational approach

感觉替代和智能代理的场景解释：一种神经计算方法

• 批准号: RGPIN-2017-06218
• 负责人: ROUAT, Jean
• 金额: $ 1.75万
• 依托单位: Université de Sherbrooke
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=754142
• 关键词: Scene; interpretation; sensory; substitution; intelligent

The long-term goal of my research is the development of assistive technologies with sensory substitution devices. Thanks to the great plasticity of the brain and the densely interconnected cortical areas, it is possible to complement and substitute one sensory modality with another. However current sensory substitution systems are not able to analyze visual or auditory scenes as they are essentially describing the environment without interpretation. Therefore they overload the audition or the vision of the user which might be uncomfortable or unhealthy. Interpretation of auditory, visual and multimodal scenes is also of great need to the development of intelligent agents that interact with their environment and human beings. But actually scene interpretation for intelligent agents is more or less a limited description of scenes and restraints to very specific applications.Through this research program I want to push as much as possible the research in multimodal scene interpretation for the next 5 coming years.To maximize the impact and while doing the research, I will frequently transfer results to applications in sensory substitution (FQRNT équipe) and in artificial intelligence for interacting agents (CHIST-ERA IGLU).To minimize the environmental footprint, the design and realization of the algorithms will favour architectures with the lowest power consumption. To this end, I will focus during the last 2 years on implementations with networks of spiking neurons. I am restricting the research program to sources that are visual (they are moving sources of given shapes, texture 3) Semantic characterization of objects;4) Scene analysis and interpretation.5) Low-power neuromorphic implementations.Sensory substitution has a very strong potential for "exercises" and rewiring of the brain along with development of new brain prosthesis. Intelligent agents that can perceive their environment through scene analysis is a corner stone to the new AI economy and industry. Also, there is a strong need to develop the Canadian industry of spiking neural networks and neuromorphic systems.

我研究的长期目标是开发具有感官替代装置的辅助技术。由于大脑的巨大可塑性和紧密相连的皮层区域，有可能用另一种感觉模式来补充和替代一种感觉模式。然而，目前的感官替代系统无法分析视觉或听觉场景，因为它们本质上是描述环境而没有解释。因此，它们会让用户的听觉或视觉超负荷，这可能会让他们感到不舒服或不健康。对听觉、视觉和多模态场景的解释对于与环境和人类互动的智能体的发展也是非常需要的。但实际上，智能代理的场景解释或多或少是对场景的有限描述和对非常具体应用的限制。通过这个研究项目，我想在未来的5年里尽可能地推动多模态场景解释的研究。为了最大限度地发挥影响，在进行研究时，我将经常将结果转移到感官替代（FQRNT <s:2> quipe）和相互作用代理的人工智能（christ - era IGLU）中的应用中。为了最大限度地减少环境足迹，算法的设计和实现将有利于具有最低功耗的架构。为此，我将在过去的两年里重点研究尖峰神经元网络的实现。我将研究计划限制在视觉源（它们是给定形状、纹理的移动源）；4)场景分析与解读。5)低功耗神经形态实现。随着新脑假体的发展，感觉替代具有很强的“锻炼”和大脑重新布线的潜力。通过场景分析来感知周围环境的智能体是新AI经济和产业的基石。此外，加拿大迫切需要发展脉冲神经网络和神经形态系统产业。