NCS-FO: Algorithmically explicit neural representation of visual memorability

NCS-FO：视觉记忆力的算法明确的神经表示

• 批准号: 1532591
• 负责人: Antonio Torralba
• 金额: $ 80.5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-01 至 2019-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1532591&HistoricalAwards=false
• 关键词: NCS; FO; Algorithmically; explicit; neural

As Lewis Carroll famously wrote in Alice in Wonderland - It's a poor sort of memory that only works backwards-. On this side of the mirror, we cannot remember visual events before they happen; however, our work will help predict what people remember, as they see an image or an event. Our team of investigators in cognitive science, human neuroscience and computer vision bring the synergetic expertise to determine how visual memories are encoded in the human brain at milliseconds and millimeters-resolution. Cognitive-level algorithms of memory would be a game changer for society, ranging from accurate diagnostic tools to human-computer interfaces that will foresee the needs of humans and compensate when cognition fails. The project capitalizes on the spatiotemporal dynamics of encoding memories while providing a computational framework for determining the representations formed from perception to memory along the scale of the whole human brain. A fundamental function of cognition is the encoding of information, a dynamic and complex process underlying much of our successful interaction with the external environment. Here, we propose to combine three technologies to predict what makes an image memorable or forgettable: neuro-imaging technologies recording where encoding happens in the human brain (spatial scale), when it happens (temporal scale), and what types of computation are performed at the different stages of storage (computational scale). Characterizing the spatiotemporal dynamics of visual memorability, and determining the type of computation and representation a successful memorability system performs is a crucial endeavor for both basic and applied sciences.

正如刘易斯·卡罗尔在《爱丽丝梦游仙境》中所写的那样——这是一种只能向后记忆的糟糕记忆。在镜子的这一边，我们无法记住发生之前的视觉事件；然而，我们的工作将有助于预测人们在看到图像或事件时的记忆。我们的研究团队在认知科学，人类神经科学和计算机视觉带来协同的专业知识，以确定视觉记忆如何在人脑中以毫秒和毫米的分辨率编码。认知层面的记忆算法将改变社会的游戏规则，从准确的诊断工具到人机界面，它将预见人类的需求，并在认知失败时进行补偿。该项目利用了编码记忆的时空动态，同时提供了一个计算框架，用于确定从感知到记忆形成的表征，沿着整个人脑的尺度。认知的一个基本功能是信息编码，这是一个动态而复杂的过程，是我们与外部环境成功互动的基础。在这里，我们建议结合三种技术来预测是什么让图像令人难忘或容易忘记：神经成像技术记录编码在人脑中发生的位置（空间尺度），何时发生（时间尺度），以及在存储的不同阶段执行什么类型的计算（计算尺度）。表征视觉记忆的时空动态，并确定一个成功的记忆系统所执行的计算和表征类型，是基础科学和应用科学的重要努力。