CAREER: Sparse Associative Deep Learning using Neural Mimicry in Multimodal Machine Learning

职业：在多模态机器学习中使用神经拟态的稀疏关联深度学习

• 批准号: 1954364
• 负责人: Edward Kim
• 金额: $ 46.57万
• 依托单位: Drexel University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1954364&HistoricalAwards=false
• 关键词: CAREER; Sparse; Associative; Deep; Learning

Artificial intelligence has made incredible progress in the past several years. AI technology is now successfully being used in voice assistants, photo recognition technology, chatbots, search engines, and self-driving cars. While current AI is very good at matching specific patterns for specific tasks, research has shown that it cannot generalize to different tasks and has no real understanding of what it is doing. Thus, radical new directions need to be explored to achieve a truly intelligent machine. This project explores a new kind of AI framework, one that mimics how the human brain senses and understands the world. This new AI system learns much like an infant would, by simply observing the world and learning through exploration. This project also utilizes a new type of computer chip that communicates information in the same way that neurons in the brain communicate. Ultimately, the project will create a new kind of AI by mimicking certain functions of the human brain. This research can inform new methods and approaches to creating an AI that better understands the world in which we live. Furthermore, the project attracts and supports the education of students interested in the interdisciplinary field of human and machine intelligence.This project develops a new multimodal machine learning paradigm that is principally different from the traditional deep learning methods used in the state-of-the-art today. This research is inspired by breakthroughs in computational and theoretical neuroscience that incorporate ideas not explored by current feed-forward deep learning architectures. Rather than using massive labeled datasets, the algorithms learn much like an infant learns, i.e., by unsupervised observation and exploration of the world through different sensory inputs. The project addresses three primary research challenges: (1) the algorithms will robustly learn the structure of the world, (2) the model will learn heterogenous associations from repeated stimuli, and (3) given the same fundamental architecture, the model will learn how to predict the future. Furthermore, the framework described in the project mimics the hierarchical architecture, sparsity, top-down, and feedback functions of the mammalian brain. This model is built upon recent advances in neuromorphic software and hardware that enhance the functionality, energy use, and speed of the underlying algorithms. Given that neuromorphic approaches are under active development, this project has the unique opportunity to provide algorithms and functionality in software and silicon.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能在过去几年里取得了令人难以置信的进步。人工智能技术现在正成功地应用于语音助手、照片识别技术、聊天机器人、搜索引擎和自动驾驶汽车。虽然目前的人工智能非常擅长为特定的任务匹配特定的模式，但研究表明，它不能概括到不同的任务，也没有真正理解它在做什么。因此，需要探索激进的新方向，以实现真正的智能机器。这个项目探索了一种新型的人工智能框架，一种模仿人脑如何感知和理解世界的框架。这个新的人工智能系统的学习方式很像婴儿，只需观察世界，在探索中学习。该项目还利用了一种新型的计算机芯片，它的信息交流方式与大脑中神经元的交流方式相同。最终，该项目将通过模仿人脑的某些功能来创造一种新型的人工智能。这项研究可以为创造更好地理解我们生活的世界的人工智能提供新的方法和途径。此外，该项目还吸引和支持了对人机智能跨学科领域感兴趣的学生的教育。该项目开发了一种新的多模式机器学习范式，与当今最先进的传统深度学习方法主要不同。这项研究的灵感来自于计算和理论神经科学的突破，这些突破融合了当前前馈深度学习体系结构没有探索的想法。这些算法不是使用大量的标记数据集，而是像婴儿学习一样学习，即通过不同的感官输入进行无监督的观察和对世界的探索。该项目解决了三个主要的研究挑战：(1)算法将稳健地学习世界的结构，(2)模型将从重复刺激中学习异质关联，(3)在相同的基本架构下，模型将学习如何预测未来。此外，该项目中描述的框架模拟了哺乳动物大脑的分层结构、稀疏性、自上而下和反馈功能。该模型建立在神经形态软件和硬件的最新进展的基础上，这些软件和硬件增强了基本算法的功能、能量使用和速度。鉴于神经形态方法正在积极开发中，该项目有独特的机会在软件和硅片中提供算法和功能。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Modeling Biological Immunity to Adversarial Examples

• DOI: 10.1109/cvpr42600.2020.00472
• 发表时间: 2020-06
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Edward J. Kim;Jocelyn Rego;Y. Watkins;Garrett T. Kenyon

Spatiotemporal Sequence Memory for Prediction using Deep Sparse Coding

使用深度稀疏编码进行预测的时空序列内存

• DOI: 10.1145/3320288.3320295
• 发表时间: 2019
• 期刊: NICE '19: Proceedings of the 7th Annual Neuro-inspired Computational Elements Workshop
• 作者: Kim, Edward;Lawson, Edgar;Sullivan, Keith;Kenyon, Garrett T.
• 通讯作者: Kenyon, Garrett T.

Information Graphic Summarization using a Collection of Multimodal Deep Neural Networks

使用多模态深度神经网络集合进行信息图形摘要

• DOI: 10.1109/icpr48806.2021.9412146
• 发表时间: 2021
• 期刊: 25th International Conference on Pattern Recognition (ICPR
• 作者: Kim, Edward;Onweller, Connor;McCoy, Kathleen F.
• 通讯作者: McCoy, Kathleen F.

A Neuromorphic Sparse Coding Defense to Adversarial Images

针对对抗性图像的神经形态稀疏编码防御

• DOI: 10.1145/3354265.3354277
• 发表时间: 2019
• 期刊: ICONS '19: Proceedings of the International Conference on Neuromorphic Systems
• 作者: Kim, Edward;Yarnall, Jessica;Shah, Priya;Kenyon, Garrett T.
• 通讯作者: Kenyon, Garrett T.

Distributional Semantics of Line Charts for Trend Classification

趋势分类折线图的分布语义

• 发表时间: 2022
• 期刊: International Symposium on Visual Computing
• 作者: Onweller, Connor;O’Brien, Andrew;Kim, Edward;McCoy, Kathleen. F.
• 通讯作者: McCoy, Kathleen. F.