Collaborative Research: Developing a Computational Model of "Quality" for Educational Digital Libraries

Prop ID：IIS-0534581，IIS-0534515PI：Sumner，Tamara R.，Marlino，Marlino，Marlino。机构：科罗拉多大学博尔德分校/UCARTITLE：合作研究：为教育数字图书馆开发“质量”的计算模型这个项目将开发和评估机器学习派生的数字图书馆内容质量统计模型，用于科学和工程教育应用。开发接近专家人类判断的质量计算模型是开发界面和工具的基本要求，这些界面和工具可以优化和支撑人类对质量的判断。这项研究将调查：a)数字学习资源和图书馆藏书的特征，作为从事资源选择和馆藏管理的专家的关键质量标志；b)机器学习和自然语言处理技术，具有足够的辨别力，接近人类的决策；以及c)如何对质量标志进行计算建模和必要的设计考虑。这项研究将由科罗拉多大学博尔德分校和UCAR地球系统教育数字图书馆(DLESE)项目中心的研究人员合作进行。在国家科学数字图书馆(NSDL)和DLESE等国家教育数字图书馆工作以及其他大规模努力中，质量已成为一个占主导地位但鲜为人知的问题。评估质量涉及做出复杂、耗时和多变的人类判断，目前还不清楚人力密集型实践是否会扩大规模，以满足预期的未来图书馆增长。因此，支持人类质量判断的计算模型可以在建立未来多样化的数据仓库和网络方面发挥关键作用。这项工作的主要成果将包括专家质量评价过程的概念模型和相应的计算模型，该模型将概念模型形式化并可进行经验验证。次要成果包括记录收集整理的最佳做法，以及为围绕质量的整理过程设计工具的初步准则。这项研究在设计和优化有关质量的信息流方面扩展了协作系统中的现有理论，以支持分布式和不断发展的数字图书馆环境中的人类判断。

Prop ID: IIS-0534581, IIS-0534515PI: Sumner, Tamara R., Marlino, Mary.Institution: University of Colorado at Boulder/UCARTitle: Collaborative Research: Developing a Computational Model of "Quality" for Educational Digital LibrariesThis project will develop and evaluate machine-learning derived statistical models of quality for digital library contents for science and engineering education applications. Developing a computational model of quality that approximates expert human judgments is a foundational requirement for developing interfaces and tools that can optimize and scaffold human judgments on quality. The research will investigate: a) characteristics of digital learning resources and library collections that serve as key markers of quality for experts engaged in resource selection and collection curation; B) machine learning and natural language processing techniques with sufficient discrimination to approximate human-decision making; and c) how quality markers might be modeled computationally and requisite design considerations.The research will be conducted by a collaborative partnership between investigators at the University of Colorado at Boulder and the Digital Library for Earth System Education (DLESE) Program Center at UCAR. Quality has emerged as a dominant yet poorly understood concern within national educational digital library efforts such as the National Science Digital Library (NSDL) and DLESE and other large-scale efforts. Evaluating quality involves making complex, time-consuming, and variable human judgments and it is not clear that human-intensive practices will scale to meet anticipated future library growth. Computational models for supporting human quality judgments can thus play a critical role in building future diverse data repositories and networks of these. Primary outcomes from this work will include a conceptual model of expert quality evaluation processes and a corresponding computational model that formalizes the conceptual model and can be empirically validated. Secondary outcomes include the documentation of best practices for collection curation and preliminary guidelines for the design of tools to scaffold curation processes around quality. This research extends current theories in collaborative systems in the design and optimization of the flow of information about quality to support human judgments in distributed and evolving digital library environments.