EAGER: Toward a General Framework for Optimal Experimentation in Computational Cognition

认知科学旨在深入了解认知任务的潜在机制。为了实现这一目标，研究人员改变实验中影响任务的因素并观察反应。从这样的实验中学习的一种方法是建立和测试刺激-反应关系的计算模型。然而，模型描述任务的详细程度需要同样多的支持数据的细节。由于实验的观察结果通常是昂贵的（例如，孩子的问题，这是一个相当大的障碍。最优试验法可以作为一种解决方案。它可以优化刺激的选择，以最大限度地从反应中推断。尽管如此，将这种方法应用于每一个新实验的困难一直是一个绊脚石。该项目旨在为最佳实验的一般框架奠定基础。我们的目标是使其适用于认知科学中的广泛建模问题。这将有助于认知科学家有效地发展认知任务的定量描述。此外，该方法有可能在社会和行为研究中广泛地加速科学发现。通过与受试者的最佳互动指导进行认知科学实验，以实现明确的量化推理目标是一个强大的想法。这种方法对于行为实验特别有吸引力，因为在这些实验中，响应的噪声量非常大，以至于需要多次重复测量。尽管它在认知建模研究中具有开创性的潜力，但对于该领域的大多数研究人员来说，最佳实验方法是遥不可及的。为每个独特的实验范式实施它的艰巨任务一直是实现该方法的有前途的力量的障碍。该项目的重点是在任意认知建模环境中建立最佳实验的技术可行性。拟议的研究将明确定义该领域的方法的需要，确定合适的计算策略，并在模拟研究中测试替代算法。考虑中的算法的性能将在建模范例的测试平台上进行评估，其成功的治疗将转移到广泛的类似问题。该项目旨在为计算认知的最佳实验的通用方法创造一个切实可行的蓝图。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Cognitive science aims to gain detailed insight into the underlying mechanisms of cognitive tasks. To achieve this, researchers change factors affecting a task in an experiment and observe responses. One way to learn from such an experiment is to build and test a computational model of stimulus-response relationships. However, the level of detail with which a model describes a task requires as much detail in supporting data. As observations from experiments are often expensive (e.g., child subject), this is a rather significant barrier. The method of optimal experiments can be a solution. It can optimize the selection of stimuli to maximize inference from responses. Nonetheless, the difficulty in applying the method to each new experiment has been a stumbling block. This project proposes to lay the foundation for a general framework for optimal experiments. The goal is to make it applicable to a wide range of modeling problems in cognitive science. This will help cognitive scientists to develop quantitative accounts of cognitive tasks effectively. Further, the method has the potential to accelerate scientific discovery broadly in social and behavioral research.Conducting cognitive science experiments guided by optimal interaction with subjects toward a clear, quantified inference goal is a powerful idea. Such a method is particularly enticing for behavioral experiments in which the amount of noise in response is so great as to require many repeated measurements. Despite its groundbreaking potential for cognitive modeling research, the method of optimal experimentation is out of reach for most researchers in the field. The formidable task of implementing it for each unique experimental paradigm has been an obstacle to the realization of the methodology's promising power. The project focuses on establishing the technical feasibility of optimal experiments in arbitrary cognitive modeling contexts. The proposed research will define the need for the methodology in the field clearly, identify suitable computational strategies, and test alternative algorithms in simulation studies. The performance of algorithms under consideration will be evaluated on a testbed of modeling paradigms whose successful treatment would transfer to a wide range of similar problems. The project aims to create a tangible blueprint for a general-purpose methodology for optimal experimentation in computational cognition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.