Uncheatable Benchmarks

不可欺骗的基准

• 批准号: 9319393
• 负责人: Jin-Yi Cai
• 金额: $ 13.42万
• 依托单位: SUNY at Buffalo
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 1993
• 资助国家: 美国
• 起止时间: 1993-08-15 至 1997-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=9319393&HistoricalAwards=false
• 关键词: Uncheatable; Benchmarks

Benchmarks have been used to test everything from the speed of a processor to the access time of a memory system. The computing community relies on them heavily to assess how fast a given hardware or software system operates. They are of fundamental importance in everyday computing. However, up until now, the study of the art of designing a good benchmark has focused on making the benchmark ``realistic'' in predicting how well it will perform for the intended applications; the issue of making benchmark results trustworthy has been relegated to ``trusted'' or third party agents, and little attention has been paid to the question of making benchmarks themselves ``uncheatable.'' The project studies the problem of how to make benchmarks resistant to tampering and hence more trustworthy. The plan is to use modern cryptography and complexity theory to make this possible. Basically the trust in individuals and organizations is replaced by trust in the impossibility of breaking certain computational problems. The advantage of this new approach is clear: the new schemes will be scientifically trustworthy. There is a whole array of problems that need to be formulated and solved, which are important in making uncheatable benchmarks; some are practical issues, some are theoretical.

基准测试已经被用来测试从处理器的速度到内存系统的访问时间的一切。计算社区在很大程度上依赖于它们来评估给定硬件或软件系统的运行速度。它们在日常计算中是至关重要的。然而，到目前为止，对设计一个好的基准的艺术的研究主要集中在使基准“真实”地预测它在预期应用程序中的表现；使基准测试结果可信的问题被降级为“可信”或第三方代理的问题，而使基准测试本身“不可欺骗”的问题却很少受到关注。“该项目研究的问题是如何使基准不受篡改，从而更值得信赖。计划是使用现代密码学和复杂性理论来实现这一目标。基本上，对个人和组织的信任被对不可能打破某些计算问题的信任所取代。这种新方法的优势是显而易见的：新方案在科学上是值得信赖的。有一大堆问题需要制定和解决，这对于制定不可欺骗的基准很重要；有些是实际问题，有些是理论问题。