RoCKIn Benchmarking and Scoring System
RoCKIn Benchmarking and Scoring System
复制标题
RoCKIn 基准测试和评分系统
DOI：
10.5772/intechopen.70013
复制
发表时间：
2017
期刊：
影响因子：
--
通讯作者：
P. Lima
中科院分区：
文献类型：
--
作者：
G. Fontana;M. Matteucci;F. Amigoni;ViolaSchiaffonati;Andrea Bonarini;P. Lima
欧洲 RoCKIn 项目带来的主要创新是通过竞赛对标的新范式的定义、实施和实际机器人竞赛的应用。通过这样做，RoCKIn 启动了一个进化过程，将机器人竞赛从科学影响有限的成功展示转变为对自主机器人系统性能进行一致和客观评估的基准工具。我们的工作首先根据竞赛环境的特点和局限性重新审视科学方法的基础；然后我们在此基础上设计了一种新颖的竞赛，其中基准和客观绩效指标的概念是关键点；最后，我们以现实世界的机器人竞赛的形式实现了这些概念。本章描述了上述过程，解释了 RoCKIn 如何解决其各个方面（理论、技术、程序）。将特别关注定义性能指标和捕获可靠评估机器人感知和动作所需的基本事实的问题。
The main innovation brought forth by the European Project RoCKIn is the definition, implementation and application to an actual robot competition of the novel paradigm of benchmarking through competitions . By doing so, RoCKIn set in motion an evolution ary process to transform robot competitions from successful showcases with limited scientific impact into benchmarking tools for the consistent and objective evaluation of the performance of autonomous robot systems. Our work began by revisiting, in the light of the features and limitations of a competition setting, the very foundations of the scientific method; then we built on these by designing a novel type of competitions where the concepts of benchmark and objective performance metrics are the key points; finally, we arrived to the implementation of such concepts in the form of a real-world robot competition. This chapter describes the above process, explaining how each of its several aspects (theoretical, technical, procedural) has been tackled by RoCKIn. Special attention will be devoted to the problems of defining performance metrics and of capturing the ground truth needed to reliably assess robot perceptions and actions.