Collaborative Research: HNDS-I:SweetPea: Automating the Implementation and Documentation of Unbiased Experimental Designs

现代实证研究面临的两个重要问题是透明度（其他人能在多大程度上弄清楚研究是如何完成的）和可复制性（如果其他人做同样的实验，结果是否会相同）。缺乏透明度和重复实验的失败阻碍了科学进步，并导致对科学方法的不信任。该项目开发了一种开源编程语言SweetPea，用于自动化实验设计。它将帮助研究人员理解和复制他人的结果，最大限度地减少错误和偏见，并提高整个实验过程的效率和准确性。因此，这项技术将有助于推进科学发现，使它们更容易获得和可靠，并为实证科学研究人员提供一种有价值的工具来帮助他们的工作。在行为科学中，由于在避免混淆因素的同时实施准确和适当平衡的实验设计所遇到的挑战，出现了许多重复性问题。此外，缺乏清晰和透明的文件降低了透明度和复制实验结果的能力。SweetPea编程语言旨在促进可重复的实验设计。该项目扩展了SweetPea的核心功能，以支持各种设计策略、自动化文档流程以及扩展用户和贡献者社区。SweetPea使用直观的界面进行实验设计的声明式表达，并使用先进的计算算法进行采样和分析。该软件确保实验设计正确实施，而不会引入意想不到的混淆。此外，该项目利用大型语言模型对实验设计进行稳健的文档记录。该项目包括邀请心理学家、神经科学家、行为经济学家和机器学习专家参与的外展活动。总的来说，这个项目提高了实验设计的准确性、透明度和可重复性，为研究人员提供了一个方便和强大的科学研究工具。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Two important issues facing modern empirical research are those of transparency (how well others can figure out how the research was done) and replicability (whether the outcomes will be the same if someone else does the same experiments). Lack of transparency and failures of experiments to replicate stifle scientific progress and lead to a mistrust of the scientific method. This project develops an open-source programming language, SweetPea, that automates experimental design. It will help researchers understand and replicate the results of others, minimize errors and bias, and increase the efficiency and accuracy of the entire experimental process. Consequently, this technology will contribute to advancing scientific discoveries, make them more accessible and reliable, and provide researchers in empirical sciences with a valuable tool to aid their work.Many replication problems in the behavioral sciences arise because of the challenges encountered in implementing accurate and appropriately balanced experimental designs while avoiding confounding factors. Additionally, the lack of clear and transparent documentation reduces transparency and the ability to replicate experimental results. The SweetPea programming language is designed to facilitate reproducible experimental design. This project extends SweetPea's core functionality to support various design strategies, automating the documentation process, and expanding the community of users and contributors. SweetPea uses an intuitive interface for the declarative expression of experimental designs, and advanced computational algorithms for sampling and analysis. The software ensures that experimental designs are properly implemented without introducing unexpected confounds. In addition, the project leverages large language models for robust documentation of experimental designs. The project includes outreach activities to engage psychologists, neuroscientists, behavioral economists, and machine learning experts. Overall, this project improves the accuracy, transparency, and replicability of experimental designs, offering researchers an accessible and powerful tool for scientific investigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.