RI:Small:Robust Performance Models

算法在现代社会中无处不在，是经济不可或缺的一部分。无论是为飞机、卡车和轮船寻找包裹和操作员的最佳分配，还是在不同语言之间进行翻译，解决的问题每天都变得更大、更具挑战性。使这种发展成为可能的关键因素是人工智能的进步。解决同一类型的问题通常有不同的方法，而且它们通常是协同的--在一个失败的地方，另一个表现得很好。该项目中的人工智能技术允许自动选择解决给定问题的最佳方法。这项研究将允许即使在困难的情况下也能更有力地做出这样的选择，从而提高性能并减少在实际系统中部署人工智能的努力。最终，该项目将使人类更容易开发高性能的人工智能系统。算法选择是将协同算法选择与问题的特定属性自动匹配的过程，以实现最佳性能。相对于可用算法进行这种选择的当前方法通常在适用性上受限于对算法进行基准测试的硬件、对运行施加的资源限制以及随机化算法中的性能波动引起的偏差。在许多情况下，这些问题是由于依赖脆性性能指标而导致的，限制了在学术界和工业中的实际应用。该项目旨在通过三种方式解决这些限制。首先，它将定义一个健壮性的概念来指导算法选择，并确定影响健壮性的算法、实验设置和计算环境的属性。其次，它将根据健壮性的定义开发具体的性能衡量标准，这些衡量标准可以在不同的硬件平台上移植。第三，它将通过基于机器学习建立性能模型的新方法来缓解脆弱性能指标的影响。该项目将向更广泛的人工智能社区传播共享的数据和基准，例如通过算法选择库(ASlib)。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Algorithms are ubiquitous in modern society and integral to the economy. Whether finding optimal assignments of packages and operators to planes, trucks, and ships, or translating between different languages, the problems solved become larger and more challenging every day. Crucially enabling such developments are advances in artificial intelligence. There are often different approaches for solving the same type of problem, and they are often synergistic -- where one fails, another performs well. AI techniques in this project allow the best approach for a given problem to be chosen automatically. This research will allow for such choices to be made more robustly even in difficult circumstances, resulting in improved performance and reduced effort to deploy AI in practical systems. Ultimately, the project will make it easier for humans to develop high-performance AI systems.Algorithm selection is the process of automatically matching synergistic algorithmic choices to the specific properties of a problem in order to achieve optimal performance. Current methods for making such choices over available algorithms are often limited in applicability by the hardware on which the algorithms were benchmarked, the resource limits imposed on runs, and subject to bias caused by performance fluctuations in randomized algorithms. In many cases, these issues are caused by reliance on brittle performance measures, limiting practical application in academia and industry. This project aims to address these limitations in three ways. First, it will define a notion of robustness to guide algorithm selection, and identify properties of algorithms, experimental setups, and computational environments that affect robustness. Second, it will develop specific performance measures informed by this definition of robustness, and which are portable across different hardware platforms. Third, it will mitigate the impact of brittle performance measures through new approaches to building performance models based on machine learning. The project will result in the dissemination of shared data and benchmarks to the broader AI community, for example through the Algorithm Selection Library (ASlib).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.