CAREER: Improving the Reliability of Human-Centered Secure-Development Research

提高软件安全性是美国和世界的一个关键需求。尽管在软件安全方面取得了重大的技术进步，但不安全的软件仍然是一个普遍的问题，有时会带来灾难性的后果。解决这个问题需要理解在安全软件开发过程中，人类的决策是如何与技术相互作用的。然而，研究这些人为因素通常是昂贵的、耗时的和困难的，有几个原因：专业开发人员是一个小而难以接触的研究人群，专业软件开发是一项复杂的任务，很难在研究环境中模仿，而安全性尽管至关重要，但通常是一个次要的目标，很难直接研究。试图进行此类研究的研究人员必须在平衡复杂性、时间和成本限制以及预期结果的有效性或有用性的同时，对实验设计做出许多选择。不幸的是，关于如何最好地做出这些选择，几乎没有基于证据的指导。通过开展各种实验，直接比较不同实验设计选择对以人为本的安全发展研究的影响，本项目将帮助未来的研究人员设计更好的实验，并尽可能有效地部署他们的资源。这个项目将通过经验地建立最佳实践和权衡，以可用的安全性和经验软件工程社区的最佳实践为基础，提高以开发人员为中心的安全性研究的有效性和可靠性。研究人员将在三个关键领域进行一系列方法研究和实验：(a)如何设计适当的方案拟订任务；(b)如何选择一个能有效平衡实验控制与生态效度的研究环境；(c)如何衡量相关结果，如开发人员自我效能和API可用性。研究人员将测试这些研究设计的关键问题跨越多个潜在的研究问题，如api，文档资源和安全工具的比较。研究结果将深入了解在不同类型的研究中，设计决策之间的权衡是如何发挥作用的，从而使研究人员能够做出最适合其背景的明智选择。这些结果将被综合成全面的指导方针，以帮助研究人员进行更好的研究，获得更有力的证据，从而改善安全发展的过程。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Improving software security is a critical need for the U.S. and the world. Despite significant technical advances in software security, insecure software remains a common problem, sometimes with disastrous results. Solving this problem will require understanding how human decision-making interacts with technology in the process of secure software development. However, studying these human factors is typically expensive, time-consuming and difficult, for several reasons: professional developers are a small and hard-to-reach study population, professional software development is a complex task that can be hard to mimic in a study environment, and security is, despite its criticality, often a secondary goal that can be difficult to study directly. Researchers attempting to conduct such studies must make many choices about experimental design while balancing complexity, time and cost constraints, and validity or usefulness of expected results. Unfortunately, there is little evidence-based guidance as to how best to make these choices. By conducting a variety of experiments directly comparing the effects of different experimental design choices on studies of human-centered secure development, this project will help future researchers design better experiments and deploy their resources as effectively as possible.This project will improve the validity and reliability of developer-centered security research by empirically establishing best practices and tradeoffs, building on best practices from the usable security and empirical software engineering communities. Researchers will undertake a series of methodological studies and experiments in three key areas: (a) how to design appropriate programming tasks; (b) how to choose a study environment that effectively balances experimental control with ecological validity; and (c) how to measure relevant outcomes such as developer self-efficacy and API usability. Investigators will test these critical questions of study design across multiple underlying research questions, such as comparisons of APIs, documentation resources, and security tools. The results will provide deep insights into how tradeoffs among design decisions play out in different kinds of studies, allowing researchers to make informed choices that fit best in their context. The results will be synthesized into comprehensive guidelines to help researchers conduct better studies, acquire stronger evidence, and therefore improve the process of secure development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.