Game-based Assessment: An Interdisciplinary Workshop Across Assessment, Business, and Education

基于游戏的评估（gba）被定义为通过游戏设计方法创造的互动体验，其中人类的知识、技能、能力和其他特征（KSAOs）可以被测量，有时也被称为“不测试的评估”，在公共和私人组织、教育和军队中变得越来越普遍。不幸的是，大湾区研究分为三个领域——组织科学、游戏科学和教育，这些领域之间没有任何系统的协调，甚至没有太多的互动。与此同时，GBA供应商在几乎没有研究人员指导的情况下开发了自己的GBA方法，有时会产生偏见和其他有问题的评估，这些评估目前在美国教育系统，美国企业的招聘和晋升流程以及美国军队中使用。因此，本次研讨会的目的是将跨应用领域的研究人员和供应商聚集在一起，为未来的研究创建路线图，整合跨学科的知识，并将研究人员和供应商直接联系起来，以促进使用实际gba的基础研究。因此，我们认为GBAs是反映NSF两个“大想法”的主要领域：在人类技术前沿工作的未来背景下不断增长的融合研究。通过将这些领域整合在一起，我们将创建一个研究路线图，以便随着工作的不断发展，更好地评估所需的复杂ksao。我们目前缺乏一个综合的理论理解或经验证据的各种成本效益权衡固有的gba与传统的测量方法，跨相关学科。通常有两种主要的权衡：1)以牺牲集中的度量为代价进行更广泛和更响应的KSAO度量；2)尽管增加了开发成本，但改进了总体评估效用。开发具有吸引力和心理测量学意义的GBA来克服这些问题比调查开发更复杂和多维度，因为原型开发需要具有多种多学科专业知识的大型团队，然而研究人员通常仍处于严格定义的学科竖井中，开发关于GBA的平行研究文献，尽管人们普遍认识到这会减缓理论的发展。我们的目标是通过这次研讨会打破这些学科障碍，向综合研究文献迈出第一步。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Game-based assessments (GBAs), defined as interactive experiences created through game design methods in which human knowledge, skills, abilities, and other characteristics (KSAOs) can be measured, and sometimes referred to as "assessment without tests", are becoming increasingly common across public and private organizations, education, and the military. Unfortunately, GBA research has been split across three fields -- organizational science, game science, and education -- without any systematic coordination or even much interaction between these silos. Meanwhile, GBA vendors have developed their own approaches to GBA with little guidance from researchers, sometimes creating biased and otherwise problematic assessments that are currently in use within the US education system, the hiring and promotion processes for US businesses, and the US military. Thus, the purpose of this workshop is to bring both researchers and vendors across applications domains together to create a roadmap for future research, to integrate knowledge across disciplines, and to connect researchers and vendors directly to facilitate basic research using real-world GBAs. We therefore recognize GBAs as a prime area reflecting two of NSF's "Big Ideas": Growing Convergence research in the context of the Future of Work at the Human-Technology Frontier. By bringing these fields together, we will create a research roadmap to better assess the complex KSAOs needed as work continues to evolve.We currently lack an integrative theoretical understanding or empirical evidence of the various cost-benefit trade-offs inherent to GBAs versus traditional measurement approaches, across relevant disciplines. There are two primary trade-offs commonly cited: 1) broader and more responsive KSAO measurement at the expense of focused measurement and 2) improved overall assessment utility despite increased development cost. Development of engaging and psychometrically meaningful GBAs to overcome these problems is more complex and multidimensional than survey development because prototype development requires large teams with diverse multidisciplinary expertise, yet researchers generally remain in tightly-defined disciplinary silos, developing parallel research literatures on GBA despite a general recognition of how this slows theory development. We aim to break across these disciplinary barriers with this workshop, to take the first steps towards an integrated research literature.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.