Collaborative Research: HNDS-I:SweetPea: Automating the Implementation and Documentation of Unbiased Experimental Designs

现代实证研究面临的两个重要问题是透明度(其他人能很好地弄清楚研究是如何完成的)和可复制性(如果其他人做同样的实验，结果是否会相同)。缺乏透明度和复制实验的失败扼杀了科学进步，并导致了对科学方法的不信任。这个项目开发了一种开源编程语言Sweetpea，它可以自动进行实验设计。它将帮助研究人员理解和复制他人的结果，最大限度地减少误差和偏差，提高整个实验过程的效率和准确性。因此，这项技术将有助于推动科学发现，使其更容易获得和可靠，并为经验科学的研究人员提供帮助他们工作的宝贵工具。行为科学中的许多复制问题出现，是因为在实施准确和适当平衡的实验设计同时避免混淆因素方面遇到的挑战。此外，缺乏清晰和透明的文件降低了透明度和复制实验结果的能力。Sweetpea编程语言旨在促进可重复的实验设计。该项目扩展了Sweetpea的核心功能，以支持各种设计策略，自动化文档流程，并扩大用户和贡献者社区。Sweetpea使用直观的界面进行实验设计的声明性表达，并使用先进的计算算法进行采样和分析。该软件确保实验设计得到正确实施，而不会引入意想不到的混乱。此外，该项目利用大型语言模型来健壮地记录实验设计。该项目包括邀请心理学家、神经科学家、行为经济学家和机器学习专家参与的外联活动。总体而言，该项目提高了实验设计的准确性、透明度和可重复性，为研究人员提供了一个可访问的、强大的科学调查工具。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Two important issues facing modern empirical research are those of transparency (how well others can figure out how the research was done) and replicability (whether the outcomes will be the same if someone else does the same experiments). Lack of transparency and failures of experiments to replicate stifle scientific progress and lead to a mistrust of the scientific method. This project develops an open-source programming language, SweetPea, that automates experimental design. It will help researchers understand and replicate the results of others, minimize errors and bias, and increase the efficiency and accuracy of the entire experimental process. Consequently, this technology will contribute to advancing scientific discoveries, make them more accessible and reliable, and provide researchers in empirical sciences with a valuable tool to aid their work.Many replication problems in the behavioral sciences arise because of the challenges encountered in implementing accurate and appropriately balanced experimental designs while avoiding confounding factors. Additionally, the lack of clear and transparent documentation reduces transparency and the ability to replicate experimental results. The SweetPea programming language is designed to facilitate reproducible experimental design. This project extends SweetPea's core functionality to support various design strategies, automating the documentation process, and expanding the community of users and contributors. SweetPea uses an intuitive interface for the declarative expression of experimental designs, and advanced computational algorithms for sampling and analysis. The software ensures that experimental designs are properly implemented without introducing unexpected confounds. In addition, the project leverages large language models for robust documentation of experimental designs. The project includes outreach activities to engage psychologists, neuroscientists, behavioral economists, and machine learning experts. Overall, this project improves the accuracy, transparency, and replicability of experimental designs, offering researchers an accessible and powerful tool for scientific investigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.