Collaborative Research: CyberTraining: Pilot: Semi-Automatic Assessment of Parallel Programs in Training of Students and Faculty

现代计算机允许将一个计算机程序分解为多个可以并发执行的活动或线程。新兴的大数据科学、健康、社会和工程应用需要这种并发性，以便及时给出结果。将这些应用程序的计算需求与可用的计算资源相匹配的关键是培训开发、维护和配置并发程序的工作人员。正在进行的开发用于教学并发性的工具包的工作是具有挑战性的，因为教学是特别劳动密集型的，因此，这些工具包本身不能帮助教师满足这种教学的需求。特别地，并发程序的编写是出了名的困难，并且需要教师付出大量的努力来评估这些程序的性能和正确性，并识别潜在的问题。该项目将用新的软件框架扩展现有的教学工具包，以自动评估并发程序，并使用教学研讨会和大学课程来验证扩展的工具包。项目的成功实施将促进劳动力的发展，促进科学的进步。我们正在探索的主要研究问题是：用于评估用多种编程语言编写的并发程序的基于规则的软件框架的本质应该是什么，这些程序分别提高了培训人员和学员的生产力和学习能力？我们正在探索这个问题的关键新颖步骤是：(a)开发半自动评估模型，其中人工评估与自动规则相结合，减少了自动检查的误报和误报；(b)识别新的协议和相关的架构，这些协议和架构利用了以前没有使用过的几个现有的强大工具的功能来解决我们的问题，(c)基于并发编程任务的解决方案通常具有规定的代码结构和算法的洞察力创建新技术，(d)支持分层技术，允许规则编写者以较低的规则编写工作来权衡评估质量。(f)分别在教学研讨会和大学课程中使用元评估和评估框架，以及(g)通过各种机制评估框架的可用性、可编程性、有效性和学习收益，包括前后调查、课程结束访谈和焦点小组。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern computers allow a computer program to be decomposed into multiple activities or threads that can execute concurrently. Emerging big-data scientific, health, social, and engineering applications require such concurrency to give results in a timely manner. The key to matching the computation needs of these applications to the available computing resources is training of a workforce to develop, maintain, and configure concurrent programs. Ongoing work on developing toolkits for teaching concurrency is challenging because instruction is particularly labor-intensive, and thus, these toolkits, on their own, cannot help instructors meet the demands for such instruction. Specifically, concurrent programs are notoriously difficult to write, and substantial instructor effort is required to evaluate the performance and correctness of these programs, and identify potential problems. This project will extend an existing instructional toolkit with a new software framework to automate assessment of concurrent programs, and using instructional workshops and university courses to validate the extended toolkit. Successful execution of the project will improve the workforce development and promote the progress of science.The main research question we are exploring is: What should be the nature of a rule-based software framework for assessing concurrent programs written in multiple programming languages that improves the productivity and learning, respectively, of trainers and trainees? The key novel steps we are taking to explore this question are (a) development of a semi-automatic assessment model in which manual evaluation, integrated with automatic rules, reduces false positives and negatives of the automated checks; (b) identification of new protocols and associated architectures that leverage the capabilities of several existing powerful tools that have not been used before to address our question, (c) creation of new techniques based on the insight that solutions to a concurrent programming assignment often have a prescribed code-structure and algorithm, (d) support for layered techniques that allow rule-writers to tradeoff assessment quality for low rule-writing effort, (e) development of a meta-assessment framework to train the trainers to write rules, (f) use of the meta-assessment and assessment framework in instructional workshops and university course offerings, respectively, and (g) evaluation of the usability, programmability, effectiveness and learning gain of the frameworks through diverse mechanisms including pre-post surveys, course exit interviews, and focus groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.