Evaluating simulation software components with player rating systems
Evaluating simulation software components with player rating systems
复制标题
使用玩家评级系统评估模拟软件组件
DOI：
--
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Roland Ewald
中科院分区：
文献类型：
--
作者：
Jonathan Wienß;Michael Stein;Roland Ewald
在基于组件的仿真系统中，仿真运行通常由不同组件的组合执行，每个组件解决一个特定的子任务。如果一个给定的子任务有多个组件可用（例如，不同的事件队列实现），仿真系统可能依赖于自动选择机制、用户决策，或者——如果两者都不可用——依赖于预定义的默认组件。然而，为每种子任务确定默认组件是困难的：这样的组件应该在各种应用程序域和与其他组件的各种组合中良好地工作。此外，单个组件的性能不容易评估，因为性能通常是作为一个整体来衡量组件组合的（例如，模拟运行的执行时间）。最后，默认组件的选择应该是动态的，因为随着时间的推移，可能会部署新的和潜在更好的组件到系统中。我们说明了团队游戏的玩家评分系统如何解决上述问题，并通过TrueSkill™评分系统[14]的实现来评估我们的方法，该系统应用于开源建模和仿真框架JAMES II。我们还展示了如何使用这样的系统来引导组件排名的性能分析实验。
In component-based simulation systems, simulation runs are usually executed by combinations of distinct components, each solving a particular sub-task. If multiple components are available for a given sub-task (e.g., different event queue implementations), a simulation system may rely on an automatic selection mechanism, on a user decision, or --- if neither is available --- on a predefined default component. However, deciding upon a default component for each kind of sub-task is difficult: such a component should work well across various application domains and various combinations with other components. Furthermore, the performance of individual components cannot be evaluated easily, since performance is typically measured for component combinations as a whole (e.g., the execution time of a simulation run). Finally, the selection of default components should be dynamic, as new and potentially superior components may be deployed to the system over time. We illustrate how player rating systems for team-based games can solve the above problems and evaluate our approach with an implementation of the TrueSkill™ rating system [14], applied in the context of the open-source modeling and simulation framework JAMES II. We also show how such systems can be used to steer performance analysis experiments for component ranking.