Collaborative Research: CyberTraining: Pilot: Semi-Automatic Assessment of Parallel Programs in Training of Students and Faculty

现代计算机允许将计算机程序分解为可以并发执行的多个活动或线程。 新兴的大数据科学、健康、社会和工程应用需要这样的并发性来及时给出结果。将这些应用程序的计算需求与可用的计算资源相匹配的关键是培训开发、维护和配置并发程序的劳动力。正在进行的开发并发教学工具包的工作是具有挑战性的，因为教学是特别劳动密集型的，因此，这些工具包，就其本身而言，不能帮助教师满足这种教学的需求。具体来说，并发程序是出了名的难以编写，并且需要大量的教师努力来评估这些程序的性能和正确性，并识别潜在的问题。该项目将扩展现有的教学工具包与一个新的软件框架，以自动评估并发程序，并使用教学研讨会和大学课程，以验证扩展的工具包。该项目的成功实施将促进劳动力的发展，促进科学的进步。我们正在探索的主要研究问题是：什么应该是一个基于规则的软件框架，用于评估并发程序编写的多种编程语言，提高生产力和学习，分别培训师和学员的性质？我们正在采取的探索这一问题的关键新步骤是：（a）开发一个半自动评估模型，其中人工评估与自动规则相结合，减少了自动检查的假阳性和假阴性;（B）识别新的协议和相关联的体系结构，其利用了以前未被用于解决我们的问题的若干现有的强大工具的能力，（c）基于对并行编程任务的解决方案通常具有规定的代码结构和算法的认识，创建新技术，（d）支持分层技术，使规则编写者能够在评估质量与低规则编写工作之间进行权衡，（e）开发元评估框架以培训培训者编写规则，（f）分别在教学讲习班和大学课程中使用元评估和评估框架，（g）通过各种机制，包括职前调查、课程结业面谈，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern computers allow a computer program to be decomposed into multiple activities or threads that can execute concurrently. Emerging big-data scientific, health, social, and engineering applications require such concurrency to give results in a timely manner. The key to matching the computation needs of these applications to the available computing resources is training of a workforce to develop, maintain, and configure concurrent programs. Ongoing work on developing toolkits for teaching concurrency is challenging because instruction is particularly labor-intensive, and thus, these toolkits, on their own, cannot help instructors meet the demands for such instruction. Specifically, concurrent programs are notoriously difficult to write, and substantial instructor effort is required to evaluate the performance and correctness of these programs, and identify potential problems. This project will extend an existing instructional toolkit with a new software framework to automate assessment of concurrent programs, and using instructional workshops and university courses to validate the extended toolkit. Successful execution of the project will improve the workforce development and promote the progress of science.The main research question we are exploring is: What should be the nature of a rule-based software framework for assessing concurrent programs written in multiple programming languages that improves the productivity and learning, respectively, of trainers and trainees? The key novel steps we are taking to explore this question are (a) development of a semi-automatic assessment model in which manual evaluation, integrated with automatic rules, reduces false positives and negatives of the automated checks; (b) identification of new protocols and associated architectures that leverage the capabilities of several existing powerful tools that have not been used before to address our question, (c) creation of new techniques based on the insight that solutions to a concurrent programming assignment often have a prescribed code-structure and algorithm, (d) support for layered techniques that allow rule-writers to tradeoff assessment quality for low rule-writing effort, (e) development of a meta-assessment framework to train the trainers to write rules, (f) use of the meta-assessment and assessment framework in instructional workshops and university course offerings, respectively, and (g) evaluation of the usability, programmability, effectiveness and learning gain of the frameworks through diverse mechanisms including pre-post surveys, course exit interviews, and focus groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.