OPENING THE BLACK BOX OF ‘RELEVANCE’
OPENING THE BLACK BOX OF ‘RELEVANCE’
复制标题
打开“相关性”黑匣子
DOI：
10.1108/eb026436
复制
发表时间：
1967
影响因子：
2.1
通讯作者：
R. Katter
中科院分区：
文献类型：
--
作者：
C. A. Cuadra;R. Katter
本研究的目的是找出影响关联判断的变量，并进行一系列的实验室研究，以确定这些变量对关联判断的影响。本文讨论了“隐性使用倾向”这一变量，即主体（法官）对文件预期使用所持的特定态度。140名评委对9份摘要中的每一份进行了与几份简短的信息要求声明的相关性评级。然后，同样的评委重复了评分，每个人都采用（假设）向他们描述的14种使用取向之一。研究发现，判断者的特定使用倾向对关联判断有显著影响。研究还发现，内隐使用取向可以根据其潜在结构进行分析，并且他们的研究提供了发现一个人可以准确模拟和使用另一个人的内隐使用取向的条件的可能性。相关性判断已被用来作为一个基础的措施，旨在评估信息检索系统的有效性。这些判断通常被接受的表面价值，并没有受到严格审查。然而，有理由相信，通常获得的这些数据可能不可靠，而且对以前评价研究中没有仔细控制的一些测量条件敏感。
The purpose of this project was to identify variables thought to affect relevance judgments and conduct a series of laboratory studies to determine the effects of these variables on relevance judgments. This paper discusses the variable of ‘implicit use orientations’—the particular attitude taken by a subject (judge) about the intended use of a document. One hundred and forty judges rated each of nine abstracts for relevance to several short information requirement statements. The same judges then repeated the ratings, each adopting (assuming) one of fourteen use orientations described to them. It was found that the particular use orientation assumed by the judge has a marked effect on relevance judgments. It was also found that implicit use orientations can be analysed in terms of their underlying structure and that their study offers the possibility of discovering the conditions under which one person can accurately simulate and use the implicit use orientations of another. Relevance judgments have been used as a basis for measures designed to evaluate the effectiveness of information retrieval systems. These judgments have usually been accepted at face value and have not been subjected to critical scrutiny. There is reason to believe, however, that as ordinarily obtained, they may be unreliable and sensitive to a number of conditions of measurement that have not been carefully controlled in previous evaluation studies.