CAREER: Improving the Reliability of Human-Centered Secure-Development Research

提高软件安全性是美国和世界的迫切需要。尽管在软件安全方面取得了重大的技术进步，但不安全的软件仍然是一个常见的问题，有时会带来灾难性的后果。要解决这个问题，就需要了解在安全的软件开发过程中，人类的决策如何与技术交互。然而，研究这些人为因素通常是昂贵、耗时和困难的，原因有几个：专业开发人员是一个很小且难以接触到的学习群体，专业软件开发是一项复杂的任务，很难在学习环境中模拟出来，而且尽管安全性很关键，但通常是很难直接学习的次要目标。试图进行这类研究的研究人员在平衡复杂性、时间和成本限制以及预期结果的有效性或有用性的同时，必须在实验设计方面做出许多选择。不幸的是，对于如何最好地做出这些选择，几乎没有循证的指导。通过直接进行各种实验，比较不同的实验设计选择对以人为中心的安全开发研究的影响，该项目将帮助未来的研究人员设计更好的实验，并尽可能有效地配置他们的资源。该项目将通过经验地建立最佳实践和权衡，以可用的安全和经验软件工程社区的最佳实践为基础，提高以开发人员为中心的安全研究的有效性和可靠性。研究人员将在三个关键领域进行一系列方法学研究和实验：(A)如何设计适当的编程任务；(B)如何选择有效平衡实验控制与生态有效性的研究环境；以及(C)如何衡量相关结果，如开发人员自我效能和API可用性。研究人员将通过多个基本研究问题测试这些研究设计的关键问题，例如API、文档资源和安全工具的比较。这些结果将为设计决策之间的权衡如何在不同类型的研究中发挥作用提供深入的见解，使研究人员能够做出最适合他们所处环境的明智选择。结果将被综合成全面的指导方针，以帮助研究人员进行更好的研究，获得更有力的证据，从而改进安全发展的过程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Improving software security is a critical need for the U.S. and the world. Despite significant technical advances in software security, insecure software remains a common problem, sometimes with disastrous results. Solving this problem will require understanding how human decision-making interacts with technology in the process of secure software development. However, studying these human factors is typically expensive, time-consuming and difficult, for several reasons: professional developers are a small and hard-to-reach study population, professional software development is a complex task that can be hard to mimic in a study environment, and security is, despite its criticality, often a secondary goal that can be difficult to study directly. Researchers attempting to conduct such studies must make many choices about experimental design while balancing complexity, time and cost constraints, and validity or usefulness of expected results. Unfortunately, there is little evidence-based guidance as to how best to make these choices. By conducting a variety of experiments directly comparing the effects of different experimental design choices on studies of human-centered secure development, this project will help future researchers design better experiments and deploy their resources as effectively as possible.This project will improve the validity and reliability of developer-centered security research by empirically establishing best practices and tradeoffs, building on best practices from the usable security and empirical software engineering communities. Researchers will undertake a series of methodological studies and experiments in three key areas: (a) how to design appropriate programming tasks; (b) how to choose a study environment that effectively balances experimental control with ecological validity; and (c) how to measure relevant outcomes such as developer self-efficacy and API usability. Investigators will test these critical questions of study design across multiple underlying research questions, such as comparisons of APIs, documentation resources, and security tools. The results will provide deep insights into how tradeoffs among design decisions play out in different kinds of studies, allowing researchers to make informed choices that fit best in their context. The results will be synthesized into comprehensive guidelines to help researchers conduct better studies, acquire stronger evidence, and therefore improve the process of secure development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.