Predicting Generated Story Quality with Quantitative Measures
Predicting Generated Story Quality with Quantitative Measures
复制标题
通过定量措施预测生成的故事质量
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Mark O. Riedl
中科院分区：
文献类型：
--
作者：
Chris Purdy;Xinyu Wang;Larry He;Mark O. Riedl
数字讲故事代理评估其输出的能力对于确保高质量的人类-代理交互非常重要。然而，评估故事仍然是一个开放的问题。过去的评估技术要么是特定于模型的--测量模型的特征，但不评估生成的故事--要么需要直接的人工反馈，这是资源密集型的。我们介绍了一些故事的功能，与人类的判断的故事和目前的算法，可以衡量这些功能。我们发现这种方法的结果在人类受试者研究的研究人员评估故事生成系统的代理。
The ability of digital storytelling agents to evaluate their output is important for ensuring high-quality human-agent interactions. However, evaluating stories remains an open problem. Past evaluative techniques are either model-specific--- which measure features of the model but do not evaluate the generated stories ---or require direct human feedback, which is resource-intensive. We introduce a number of story features that correlate with human judgments of stories and present algorithms that can measure these features. We find this approach results in a proxy for human-subject studies for researchers evaluating story generation systems.