PFI:BIC - Unobtrusive Neurotechnology and Immersive Human-Computer Interface for Enhanced Learning

PFI:BIC - 用于增强学习的低调神经技术和沉浸式人机界面

• 批准号: 1719130
• 负责人: Gert Cauwenberghs
• 金额: $ 100万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-01 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1719130&HistoricalAwards=false
• 关键词: PFI; BIC; Unobtrusive; Neurotechnology; Immersive

The increasing prevalence of learning disorders, attention deficits, and lackluster appetite for reading across all walks of life, and particularly among school-age children, poses severe problems to humanity and, in the long run, burdens social and economic development. This Partnership for Innovation Building Innovation Capacity (PFI:BIC) collaborative project tackles the impending threats to humanity of illiteracy and faltering education heads-on by creating a new smart-service human-computer interface (HCI) neurotechnology platform as a highly effective, user-friendly, and fun-to-use tool aiding learning and stimulating cognitive development at home and in the classroom. The immersive HCI neurotechnology will allow directly measuring progress at the cognitive level and providing real-time feedback to guide the user in learning to read more effectively. The project is highly Science, Technology, Engineering and Mathematics (STEM) intensive both in its activities and in the targeted benefits of the developed technology, which extends directly to learning science and mathematics by probing cognitive performance of children while they solve puzzles. The development of unobtrusive neurotechnology further addresses a critical need for practical integrated and modular brain-computer interface (BCI) solutions in HCI promoting widespread consumer and clinical use in the marketplace. The partnership provides opportunities for students to gain practical experience in innovation in the marketplace through internships with the industrial partners.The central aim is to develop and leverage new HCI technology as a learning coach and personal cognitive development assistant that facilitates learning to read and acquiring other critical skills in cognitive development. The immersive yet unobtrusive HCI technology testbed will comprise a dry-electrode electroencephalography (EEG) BCI, a tablet with touchscreen and integrated camera, and a suite of signal processing algorithms running in the cloud, for monitoring brain and gaze activity in children learning to read, and providing real-time neurofeedback on progress in cognitive performance to promote enhanced learning. The partnership will transition scientific advances of a previous NSF-sponsored UCSD project (NSF EFRI-M3C, ENG-1137279) in studying the distributed dynamics of human motor control, to development of neurofeedback training paradigms for learning enhancement, and to practical deployment on the unobtrusive immersive testbed implemented using Cognionics dry-electrode EEG wireless BCI neurotechnology and Syntrogi real-time cloud-based signal processing software pipelines. The potential for human empowerment by the technology will be demonstrated by evaluating effectiveness in enhancing learning capabilities and cognitive performance in simulated classroom settings and other targeted learning environments.The lead institution for the project is University of California San Diego (UCSD), with investigators from the Institute for Neural Computation and Department of Bioengineering. The industrial partners in the effort are Syntrogi Inc. (dba Qusp, small business, San Diego CA) and Cognionics, Inc. (small business, San Diego, CA). The project also engages broader context partners Drs. Andrea Chiba and Leanne Chukoskie from the UCSD Temporal Dynamics of Learning Center, Dr. Barbara Moss from San Diego State University Department of Psychology, and Dr. Zewelanji N. Serpell from Virginia Commonwealth University Department of Psychology, in the human case studies and the assessment of the developed HCI technology in diverse learning environments.

学习障碍、注意力缺陷和阅读欲望缺乏在各行各业，特别是学龄儿童中的日益普遍，给人类带来了严重的问题，从长远来看，给社会和经济发展带来了负担。 这个创新能力建设合作伙伴关系（PFI：BIC）合作项目通过创建一个新的智能服务人机界面（HCI）神经技术平台，作为一个高效，用户友好和有趣的工具，帮助学习和刺激认知发展，在家里和教室里，来解决文盲和教育摇摇欲坠对人类的威胁。 沉浸式HCI神经技术将允许直接测量认知水平的进展，并提供实时反馈，以指导用户学习更有效地阅读。 该项目是高度科学，技术，工程和数学（STEM）密集的活动和开发技术的目标效益，通过探索儿童在解决难题时的认知表现直接延伸到学习科学和数学。 不显眼的神经技术的发展进一步解决了HCI中对实用的集成和模块化脑机接口（BCI）解决方案的迫切需求，促进了市场上广泛的消费者和临床使用。 该合作伙伴关系为学生提供了通过与工业合作伙伴实习获得市场创新实践经验的机会，其中心目标是开发和利用新的HCI技术作为学习教练和个人认知发展助手，促进学习阅读和获得认知发展的其他关键技能。 沉浸式但不显眼的HCI技术测试平台将包括干电极脑电图（EEG）BCI，带有触摸屏和集成摄像头的平板电脑，以及一套在云中运行的信号处理算法，用于监测儿童学习阅读的大脑和凝视活动，并提供关于认知表现进展的实时神经反馈，以促进增强学习。 该合作伙伴关系将过渡到以前NSF赞助的UCSD项目的科学进步（NSF EFRI-M3 C，ENG-1137279）在研究人类运动控制的分布式动力学，开发用于学习增强的神经反馈训练范例，以及在使用Cognionics干电极EEG无线BCI神经技术和Syntrogi实时云实现的不引人注目的沉浸式测试平台上的实际部署-基于信号处理软件流水线。 通过评估在模拟课堂环境和其他目标学习环境中增强学习能力和认知表现的有效性，将证明该技术对人类赋权的潜力。该项目的牵头机构是加州圣地亚哥大学（UCSD），研究人员来自神经计算研究所和生物工程系。 这项工作的工业合作伙伴是Syntrogi Inc.。(dba Qusp，small business，San Diego CA）和Cognionics，Inc.（small business，San Diego，CA）。 该项目还吸引了更广泛的背景合作伙伴Andrea千叶博士和Leanne Chukoskie从加州大学圣地亚哥分校学习中心的时间动力学，博士芭芭拉莫斯从圣地亚哥州立大学心理学系，博士Zewelanji N。来自弗吉尼亚联邦大学心理学系的Serpell，在人类案例研究和在不同学习环境中对开发的HCI技术的评估中。

项目成果

Unsupervised learning of brain state dynamics during emotion imagination using high-density EEG

• DOI: 10.1016/j.neuroimage.2022.118873
• 发表时间: 2022-01-21
• 期刊: NEUROIMAGE
• 影响因子: 5.7
• 作者: Hsu, Sheng-Hsiou;Lin, Yayu;Makeig, Scott
• 通讯作者: Makeig, Scott

Hardware-Oriented Memory-Limited Online Artifact Subspace Reconstruction (HMO-ASR) Algorithm

• DOI: 10.1109/tcsii.2021.3124253
• 发表时间: 2021-12-01
• 期刊: IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS
• 影响因子: 4.4
• 作者: Van, Lan-Da;Tu, You-Cheng;Jung, Tzyy-Ping
• 通讯作者: Jung, Tzyy-Ping

Attention State Classification with In-Ear EEG

通过入耳式脑电图进行注意力状态分类

• DOI: 10.1109/biocas49922.2021.9644973
• 发表时间: 2021
• 期刊: Biomedical Circuits and Systems Conference
• 作者: Paul, A.

Integrated In-Ear Device for Auditory Health Assessment

用于听觉健康评估的集成入耳式设备

• DOI: 10.1109/embc.2019.8856455
• 发表时间: 2019
• 期刊: 2019 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society
• 作者: Paul, Akshay;Akinin, Abraham;Lee, Min S.;Kleffner, Matthew;Deiss, Stephen R.;Cauwenberghs, Gert
• 通讯作者: Cauwenberghs, Gert

A Wearable Multi-Modal Bio-Sensing System Towards Real-World Applications

• DOI: 10.1109/tbme.2018.2868759
• 发表时间: 2019-04
• 期刊: IEEE Transactions on Biomedical Engineering
• 影响因子: 4.6
• 作者: .. Siddharth-Siddharth-123964367;Aashish N. Patel;T. Jung;T. Sejnowski