EAGER: Toward a General Framework for Optimal Experimentation in Computational Cognition

认知科学的目标是对认知任务的潜在机制有详细的了解。为了实现这一点，研究人员在实验中改变影响任务的因素，并观察反应。从这样的实验中学习的一种方法是建立和测试刺激-反应关系的计算模型。然而，模型描述任务的详细程度要求支持数据具有同样的详细程度。由于来自实验的观察通常是昂贵的(例如，儿童受试者)，这是一个相当大的障碍。最优试验的方法可以是一种解决方案。它可以优化刺激的选择，以最大限度地从反应中进行推断。尽管如此，将这种方法应用于每一项新实验的难度一直是一个绊脚石。该项目建议为优化实验的一般框架奠定基础。其目标是使其适用于认知科学中的广泛建模问题。这将有助于认知科学家有效地对认知任务进行量化解释。此外，该方法有可能在社会和行为研究中广泛地加速科学发现。以与受试者的最佳交互为指导的认知科学实验朝着明确的、量化的推理目标是一个强有力的想法。这种方法对于行为实验特别有吸引力，在这些实验中，响应的噪音太大，以至于需要多次重复测量。尽管认知模型研究具有开创性的潜力，但对于该领域的大多数研究人员来说，最优实验方法是遥不可及的。为每一种独特的实验范式实施它的艰巨任务一直是实现该方法论前景光明的力量的障碍。该项目的重点是在任意认知建模环境中建立最优实验的技术可行性。拟议的研究将明确定义该领域方法学的需求，确定合适的计算策略，并在模拟研究中测试替代算法。正在考虑的算法的性能将在建模范例的试验台上进行评估，其成功的处理将转移到广泛的类似问题。该项目旨在为计算认知中的最佳实验的通用方法创建一个切实的蓝图。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Cognitive science aims to gain detailed insight into the underlying mechanisms of cognitive tasks. To achieve this, researchers change factors affecting a task in an experiment and observe responses. One way to learn from such an experiment is to build and test a computational model of stimulus-response relationships. However, the level of detail with which a model describes a task requires as much detail in supporting data. As observations from experiments are often expensive (e.g., child subject), this is a rather significant barrier. The method of optimal experiments can be a solution. It can optimize the selection of stimuli to maximize inference from responses. Nonetheless, the difficulty in applying the method to each new experiment has been a stumbling block. This project proposes to lay the foundation for a general framework for optimal experiments. The goal is to make it applicable to a wide range of modeling problems in cognitive science. This will help cognitive scientists to develop quantitative accounts of cognitive tasks effectively. Further, the method has the potential to accelerate scientific discovery broadly in social and behavioral research.Conducting cognitive science experiments guided by optimal interaction with subjects toward a clear, quantified inference goal is a powerful idea. Such a method is particularly enticing for behavioral experiments in which the amount of noise in response is so great as to require many repeated measurements. Despite its groundbreaking potential for cognitive modeling research, the method of optimal experimentation is out of reach for most researchers in the field. The formidable task of implementing it for each unique experimental paradigm has been an obstacle to the realization of the methodology's promising power. The project focuses on establishing the technical feasibility of optimal experiments in arbitrary cognitive modeling contexts. The proposed research will define the need for the methodology in the field clearly, identify suitable computational strategies, and test alternative algorithms in simulation studies. The performance of algorithms under consideration will be evaluated on a testbed of modeling paradigms whose successful treatment would transfer to a wide range of similar problems. The project aims to create a tangible blueprint for a general-purpose methodology for optimal experimentation in computational cognition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.