Collaborative Research: HNDS-I:SweetPea: Automating the Implementation and Documentation of Unbiased Experimental Designs

现代实证研究面临的两个重要问题是透明度（其他人能在多大程度上弄清楚研究是如何完成的）和可复制性（如果其他人做同样的实验，结果是否会相同）。 缺乏透明度和重复实验的失败扼杀了科学进步，导致对科学方法的不信任。 该项目开发了一种开源编程语言SweetPea，可自动化实验设计。 它将帮助研究人员理解和复制他人的结果，最大限度地减少错误和偏差，并提高整个实验过程的效率和准确性。因此，这项技术将有助于推进科学发现，使它们更容易获得和可靠，并为实证科学的研究人员提供一个有价值的工具，以帮助他们的工作。行为科学中的许多重复问题的出现，是因为在实施准确和适当平衡的实验设计时遇到的挑战，同时避免混淆因素。 此外，缺乏清晰和透明的文件降低了透明度和复制实验结果的能力。SweetPea编程语言旨在促进可重复的实验设计。该项目扩展了SweetPea的核心功能，以支持各种设计策略，自动化文档流程，并扩大用户和贡献者社区。SweetPea使用直观的界面进行实验设计的声明式表达，并使用先进的计算算法进行采样和分析。 该软件可确保正确实施实验设计，而不会引入意外的混淆。此外，该项目利用大型语言模型为实验设计提供强大的文档。 该项目包括外展活动，以吸引心理学家，神经科学家，行为经济学家和机器学习专家。总的来说，该项目提高了实验设计的准确性、透明度和可复制性，为研究人员提供了一个方便和强大的科学调查工具。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Two important issues facing modern empirical research are those of transparency (how well others can figure out how the research was done) and replicability (whether the outcomes will be the same if someone else does the same experiments). Lack of transparency and failures of experiments to replicate stifle scientific progress and lead to a mistrust of the scientific method. This project develops an open-source programming language, SweetPea, that automates experimental design. It will help researchers understand and replicate the results of others, minimize errors and bias, and increase the efficiency and accuracy of the entire experimental process. Consequently, this technology will contribute to advancing scientific discoveries, make them more accessible and reliable, and provide researchers in empirical sciences with a valuable tool to aid their work.Many replication problems in the behavioral sciences arise because of the challenges encountered in implementing accurate and appropriately balanced experimental designs while avoiding confounding factors. Additionally, the lack of clear and transparent documentation reduces transparency and the ability to replicate experimental results. The SweetPea programming language is designed to facilitate reproducible experimental design. This project extends SweetPea's core functionality to support various design strategies, automating the documentation process, and expanding the community of users and contributors. SweetPea uses an intuitive interface for the declarative expression of experimental designs, and advanced computational algorithms for sampling and analysis. The software ensures that experimental designs are properly implemented without introducing unexpected confounds. In addition, the project leverages large language models for robust documentation of experimental designs. The project includes outreach activities to engage psychologists, neuroscientists, behavioral economists, and machine learning experts. Overall, this project improves the accuracy, transparency, and replicability of experimental designs, offering researchers an accessible and powerful tool for scientific investigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.