Collaborative Research: CyberTraining: Pilot: Semi-Automatic Assessment of Parallel Programs in Training of Students and Faculty

现代计算机允许将计算机程序分解成可同时执行的多个活动或线程。新兴的大数据科学、健康、社交和工程应用需要这样的并发性才能及时产生结果。将这些应用程序的计算需求与可用的计算资源相匹配的关键是培训员工来开发、维护和配置并发程序。正在进行的开发并发性教学工具包的工作具有挑战性，因为教学是特别劳动密集型的，因此，这些工具包本身不能帮助教师满足此类教学的需求。具体地说，并发程序是出了名的难以编写，需要教师付出大量的努力来评估这些程序的性能和正确性，并确定潜在的问题。该项目将用一个新的软件框架扩展现有的教学工具包，以自动评估并发程序，并使用教学讲习班和大学课程来验证扩展的工具包。该项目的成功实施将改善劳动力发展并促进科学进步。我们正在探索的主要研究问题是：基于规则的软件框架的性质应该是什么，用于评估用多种编程语言编写的并发程序，从而分别提高培训者和受训者的生产率和学习能力？我们正在采取的探索这一问题的关键的新步骤是：(A)开发一种半自动评估模型，其中人工评估与自动规则相结合，减少自动检查的假阳性和假阴性；(B)查明新的协议和相关的体系结构，它们利用以前从未使用过的几个现有强大工具的能力来解决我们的问题；(C)根据对并发程序编制任务的解决办法通常具有规定的代码结构和算法的认识，创造新的技术；(D)支持分层技术，使规则编写者能够在评估质量和较低的规则编写工作之间进行权衡；(E)开发元评估框架，以培训培训者编写规则；(F)分别在教学讲习班和大学课程中使用元评估和评估框架，以及(G)通过不同的机制，包括岗位前调查、课程毕业访谈和焦点小组，对框架的可用性、可编程性、有效性和学习收益进行评估。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Modern computers allow a computer program to be decomposed into multiple activities or threads that can execute concurrently. Emerging big-data scientific, health, social, and engineering applications require such concurrency to give results in a timely manner. The key to matching the computation needs of these applications to the available computing resources is training of a workforce to develop, maintain, and configure concurrent programs. Ongoing work on developing toolkits for teaching concurrency is challenging because instruction is particularly labor-intensive, and thus, these toolkits, on their own, cannot help instructors meet the demands for such instruction. Specifically, concurrent programs are notoriously difficult to write, and substantial instructor effort is required to evaluate the performance and correctness of these programs, and identify potential problems. This project will extend an existing instructional toolkit with a new software framework to automate assessment of concurrent programs, and using instructional workshops and university courses to validate the extended toolkit. Successful execution of the project will improve the workforce development and promote the progress of science.The main research question we are exploring is: What should be the nature of a rule-based software framework for assessing concurrent programs written in multiple programming languages that improves the productivity and learning, respectively, of trainers and trainees? The key novel steps we are taking to explore this question are (a) development of a semi-automatic assessment model in which manual evaluation, integrated with automatic rules, reduces false positives and negatives of the automated checks; (b) identification of new protocols and associated architectures that leverage the capabilities of several existing powerful tools that have not been used before to address our question, (c) creation of new techniques based on the insight that solutions to a concurrent programming assignment often have a prescribed code-structure and algorithm, (d) support for layered techniques that allow rule-writers to tradeoff assessment quality for low rule-writing effort, (e) development of a meta-assessment framework to train the trainers to write rules, (f) use of the meta-assessment and assessment framework in instructional workshops and university course offerings, respectively, and (g) evaluation of the usability, programmability, effectiveness and learning gain of the frameworks through diverse mechanisms including pre-post surveys, course exit interviews, and focus groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.