Elements: HPN-SSH

在计算资源之间传输大型数据集是科学工作流程的一个组成部分。已经开发了多种工具来帮助完成这项任务，但是，尽管存在严重的性能瓶颈，基于安全shell （SSH）的工具，如安全复制协议（SCP）和安全文件传输协议（SFTP），由于无处不在、易于使用和最小的管理负担，仍然很受欢迎。HPN-SSH项目工具最初是在2004年开发的，以满足这一需求。该合同将允许pi扩展HPN-SSH的原始工作，并应对计算机技术进步和用户需求所带来的新挑战。它们将结合硬件加速加密；通过动态调整线程数量，有效地使用现代cpu；通过允许并行工作来加速加密工作流程；研究使默认的开源安全外壳OpenSSH密码使用多核；创建“故障恢复”功能，使用户能够从故障点重新开始传输；并纳入网络指标，以帮助解决HPN-SSH的故障和性能分析。HPN-SSH将通过降低进入大数据和远程计算的成本，在不影响安全性或功能的情况下，支持和加强广泛科学领域的研究工作。这些好处将扩展到商业、工业、教育团体和普通大众。在计算资源之间传输大型数据集是科学工作流程的一个组成部分。已经开发了多种工具来帮助完成这项任务，但是，尽管存在显著的性能瓶颈，基于SSH的工具由于其普遍性、易用性和最小的相关成本而仍然很受欢迎。为了解决这些瓶颈，我们开发了HPN-SSH；为OpenSSH应用程序提供高性能吞吐量的一系列补丁。这些补丁最初于2004年发布，并已在研究、学术、金融和技术社区广泛使用。该奖项使项目经理有机会促进HPN-SSH的创新发展，从而显著提高性能，造福社会。有了这笔拨款，pi将：在其多线程AES计数器密码中集成片上硬件加速加密；通过动态调整线程数量，有效地使用多核cpu；在加密工作流程中引入流水线和并行化；研究OpenSSH默认密码CHACHA20的并行化；创建“故障恢复”功能，使用户能够从故障点重新开始传输；并结合内联网络遥测，以帮助排除故障和性能分析。这项工作还将通过开发和改进并行化方法来提高密码例程的性能，从而推动计算机科学领域的发展。由于大多数广泛使用的加密库和方法本质上是高度串行的，它们无法利用多核处理器的优势。由于处理器速度在过去十年中保持相对稳定，我们必须将加密工作负载分布在多个核心上，以显着提高吞吐量。HPN-SSH将通过在不影响安全性或功能的情况下降低进入成本，使访问民主化并扩大国家网络基础设施的覆盖范围。这些好处将扩展到商业和工业、教育社区和公众。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, secure shell (SSH) based tools like secure copy protocol (SCP) and secure file transfer protocol (SFTP) remain popular due to ubiquity, ease of use, and minimal administrative burdens. HPN-SSH, the project tool, was initially developed in 2004 to address this need. This award will allow the PIs to expand on the original work of HPN-SSH and address new challenges created by advances in computer technology and the needs of users. They will incorporate hardware accelerated encryption; efficiently use modern CPUs by dynamically sizing the number of threads; accelerate the cryptographic workflow by allowing it to work in parallel; investigate making the default open source secure shell OpenSSH cipher use multiple cores; create a ‘resume on failure’ feature enabling users to restart transfers from the point of failure; and incorporate networking metrics to aid in troubleshooting and performance analysis of HPN-SSH. HPN-SSH will support and enhance research efforts across a wide range of scientific domains by lowering the costs of entry to big data and remote computation without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public as well. The transfer of large data sets between computing resources is an integral component of the scientific workflow. Multiple tools have been developed to aid in this task but, despite significant performance bottlenecks, SSH based tools remain popular due to ubiquity, ease of use, and minimal associated costs. To address these bottlenecks we developed HPN-SSH; a series of patches that enable high performance throughput for the OpenSSH application. These patches were initially released in 2004 and have become widely used throughout the research, academic, financial, and technology communities. This award gives the PIs opportunity to foster innovative development in HPN-SSH that will benefit the community by significantly increasing performance. With this grant the PIs will: incorporate on-die hardware accelerated encryption in their multithreaded AES counter cipher; efficiently use multicore CPUs by dynamically sizing the number of threads; introduce pipelining and parallelization into the cryptographic workflow; investigate the parallelization of the default OpenSSH cipher CHACHA20; create a ‘resume on failure’ feature enabling users to restart transfers from the point of failure; and incorporate inline network telemetry to aid in troubleshooting and performance analysis. This work will also advance the field of computer science through the development and improvement of parallelization methods to enhance the performance of cryptographic routines. As most widely used cryptographic libraries and methods are highly serial in nature they are unable to take advantage of multicore processors. As processor speed has remained relatively stable over the past ten years we must distribute the cryptographic workload over multiple cores in order to significantly increase throughput. HPN-SSH will democratize access and extend the reach of the national cyberinfrastructure by lowering the costs of entry without compromising security or functionality. These benefits will extend to business and industry, educational communities, and the general public.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.