HCC: Assessing Cognitive Function from Interactive Agent Behavior

这是一个开发科学研究和评估人类认知功能的新方法的项目。它将采用复杂的统计多模态数据分析技术，将在完成复杂的认知任务的过程中同时从人类获得的上下文，行为和神经信息融合在一起。这些任务将以定制的计算机游戏的形式呈现，这些游戏旨在展示已建立的认知评估测试的关键方面，同时为受试者与游戏和计算机代理的互动提供激励和参与的环境。这些任务将涉及利用我们现有的监测和控制某些有趣和具有挑战性的电脑游戏的能力，这些游戏涉及从工作记忆和注意力到执行功能的各种认知任务组合。多模态信息融合将利用贝叶斯推理技术和信息论数据分析和降维方法来实现。 这项拨款的目的是发展精密的模式分析技术，以分析长者在进行电脑游戏等复杂认知工作时的细微行为。从拟议的研究中预期的重大科学发现有两个方面：（1）改进EEG处理的统计信号处理和模式识别算法，（2）增强对执行复杂任务期间EEG中多个认知过程及其神经特征的相互作用的理解。 该方法在三个方面具有创新性：（1）将采用先进的自适应交互协议，其修改任务参数以保持对认知状态变化的最大灵敏度，（2）将开发新的信息理论技术并用于从EEG测量中提取最大区分特征，以用于认知状态估计和神经活动可视化，（3）开发的闭环系统将用于研究复杂认知任务中的人-智能体交互，从而产生现实进化环境中微观行为的数学模型，而不是传统的静态重复实验范式。 这项工作的成功完成将为大脑界面设计、闭环协作增强认知人机界面以提高性能以及老年人认知能力下降的早期诊断等方面的进一步协作活动开辟道路。一个跨学科的研究环境将参与研究生在一个多学科的教育环境，并将帮助他们发展技能，进行合作的跨学科研究。

This is a project to develop new methods for scientifically studying and assessing human cognitive function. It will employ sophisticated statistical multimodal data analysis techniques that will fuse contextual, behavioral, and neural information simultaneously obtained from human beings in the process of completing complex batteries of cognitive tasks. The tasks will be presented in the form of customized computer games that are designed to exhibit the crucial aspects of established cognitive assessment tests and at the same time provide a motivating and engaging environment for the subject's interactions with the game and computer agents. The tasks will involve exploiting our existing capabilities of monitoring and controlling certain enjoyable and challenging computer games that involve various combinations of cognitive tasks ranging from working memory and attention to executive functions. Multimodal information fusion will be accomplished by utilizing Bayesian inference techniques and information theoretic data analysis and dimensionality reduction methods. The work to be carried out under this grant aims to develop sophisticated pattern analysis techniques for the purpose of analyzing the fine-grain behaviors of elderly when they are engaged in complex cognitive tasks in the form of computer games. Expected significant scientific findings from the proposed research are two-fold: (1) improved statistical signal processing and pattern recognition algorithms for EEG processing, (2) an enhanced understanding of the interplay of multiple cognitive processes and their neural signatures in EEG during the execution of complex tasks. The approach is innovative in terms of three aspects: (1) an advanced adaptive interaction protocol that modifies the task parameters to maintain maximal sensitivity to cognitive state changes will be employed, (2) novel information theoretic techniques will be developed and utilized for the extraction of maximally discriminative features from EEG measurements for cognitive state estimation and neural activity visualization, (3) the developed closed-loop system will be utilized to study the human-agent interaction in complex cognitive tasks resulting in mathematical models of micro-behavior in realistic evolving environments as opposed to traditional stationary repetitive experimental paradigms. The successful completion of the work will open the way to further collaborative activities in brain interface design, closed-loop collaborative augmented cognition human-agent interfaces for improved performance, and early diagnosis of cognitive decline in elderly. An interdisciplinary research environment will engage the participating graduate students in a multidisciplinary educational setting and will help them develop skills to perform collaborative interdisciplinary research.