ONR-15-FOA-0011 MURI Topic #3 - Closed-Loop Multisensory Brain-Computer Interface for Enhanced Decision Accuracy

点击翻译按钮获取中文摘要

The goals of our interdisciplinary effort are to develop new methodologies for modeling multimodal neural activity underlying multisensory processing and decision making, and to use those methodologies to design closed-loop adaptive algorithms for optimized exploitation of multisensory data for brain-computer communication. We are motivated by the observation that a dismounted soldier or a tank driver routinely makes decisions in time-pressured and stressful conditions based on a multiplicity of multisensory information presented in cluttered and distracting environments. We envision a closed-loop brain-computer interface (BCI) architecture for enhancing decision accuracy. The architecture will collect multimodal neural, physiological, and behavioral data, decode mental states such as attention orientation and situational awareness, and use the decoded states as feedback to adaptively change the multisensory cues provided to the subject, thus closing the loop. To realize such an architecture we will make fundamental advances on four fronts, constituting our research Thrusts: (1) modeling multisensory integration, attention, and decision making, and the associated neural mechanisms; (2) machine-learning algorithms for high-dimensional multimodal data fusion; (3) adaptive tracking of the neural and behavioral models during online operation of the BCI; and (4) adaptive BCI control of multisensory cues for optimized performance. We have assembled a multidisciplinary team with expertise spanning engineering, computer science, and neuroscience. We will take a fully integrated approach to address these challenges by combining rare state-of-the-art experimental capabilities with novel computational modeling. Complementary experiments in rodents, monkeys, and humans will collect multimodal data to study and model multisensory integration, attention, and decision making, and to prototype a BCI for enhanced decision accuracy. Our modeling efforts will span Bayesian inference, stochastic control, adaptive signal processing, and machine learning to develop: novel Bayesian and control-theoretic models of the brain mechanisms; new stochastic models of multimodal data and adaptive inference algorithms for this data; and novel adaptive stochastic controllers of multisensory cues based on the feedback of users' cognitive state.