Optimizing the Number of Points on Rating Scales

优化评分量表的点数

• 批准号: 0352372
• 负责人: Jon Krosnick
• 金额: --
• 依托单位: Ohio State University Research Foundation -DO NOT USE
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-06-15 至 2007-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0352372&HistoricalAwards=false
• 关键词: Optimizing; Number; Points; Rating; Scales

Social scientists routinely design questionnaires using rating scales to measure social and psychological constructs. There is no consensus, however, among researchers about the optimal number of points to offer on such scales so as to maximize reliability and validity. This project will compare the reliabilities and validities of rating scales ranging from 2 points long to 11 points long presented visually. This project also will explore the moderating roles of three aspects of the task (biopolar vs. unipolar constructs; verbal labeling on all scale points vs. the end points only; topic familiarity) and two respondent characteristics (cognitive skills and motivation to provide accurate reports, as gauged by need for cognition). Longer scales may be optimal for bipolar constructs, scales that are fully verbally labeled, and highly familiar topics. To address these issues, a three-wave survey will be conducted with a nationally representative sample of adults by Knowledge Networks, a survey research firm that collects data weekly via the Internet from an established panel of respondents who have all been equivalently equipped with Internet access. The questionnaire will manipulate number of points on target rating scales addressing a range of topics and will include criterion measures for assessing concurrent validity of the target ratings. Also manipulated will be polarity of the construct and the extent of verbal labeling of the scale points. Issue familiarity, respondents' cognitive skills, and their motivation to provide accurate reports will be measured. Multiple groups longitudinal structural equation modeling will be conducted to assess the effects of these factors on the reliability and validity of responses, explicitly estimating random and systematic measurement error and measurement validity.Rating scales are ubiquitous in social science research because they are convenient tools for measuring attitudes, beliefs, behavioral intentions, behavioral experiences, and a range of other phenomenon of great interest to scholars, policy-makers, and business planners. Researchers use a wide variety of different scale lengths in different investigations, however, suggesting that there is no consensus as to the optimal length of a rating scale to maximize the accuracy of the measurements obtained. But it is easy to imagine that rating scales may be too long, making it difficult for respondents to use them, because the meanings of the scale points will be ambiguous. And rating scales may also be too short and therefore suboptimal because they clump people into a few crude categories (e.g., separating the people who support a law from those who oppose it, rather than permitting people to express degrees of support or opposition). This project will help to improve the accuracy of social science measurements by comparing the effectiveness of rating scales of various lengths to identify which are optimal. The study will investigate whether the optimal scale length is greater for more familiar topics, for scales that have all points labeled with words (instead of only the end points being labeled as such), and for scales with a zero-point in the middle (e.g., from "dislike a great deal" through "neither like nor dislike" to "like a great deal"), as compared to those with a zero point at the end (e.g., from "not at all important" to "extremely important"). And the project will test whether people with greater cognitive skills and more motivation to report precisely are able to manage longer rating scales more effectively. The findings will contribute to the psychological literature on how ability and motivation moderate task performance, and the findings will provide practical guidance for questionnaire designers in government, in the commercial world, and in academia (in psychology, sociology, political science, economics, education, communication, anthropology, medicine, and many other sciences that make measurements via questionnaires).

社会科学家通常使用评分量表设计问卷来衡量社会和心理结构。然而，研究人员对于在这些量表上提供的最佳点数，以最大限度地提高信度和效度，并没有达成共识。这个项目将比较从2分到11分的评分量表的信度和效度。本项目还将探讨任务的三个方面（生物极性与单极结构；所有量表点上的语言标记与终点的语言标记；话题熟悉度）和两个被调查者特征（认知技能和提供准确报告的动机，由认知需求来衡量）的调节作用。较长的量表可能是最佳的双相结构，量表是完全口头标记，和高度熟悉的话题。为了解决这些问题，Knowledge Networks将对具有全国代表性的成年人样本进行三波调查。Knowledge Networks是一家调查研究公司，每周通过互联网从已建立的受访者小组中收集数据，这些受访者都具有同等的互联网接入能力。该问卷将处理一系列主题的目标评定量表上的点数，并将包括评估目标评定的并发有效性的标准措施。同样被操纵的还有构念的极性和量表点的言语标注程度。问题熟悉度，受访者的认知技能，以及他们提供准确报告的动机将被衡量。采用多组纵向结构方程模型来评估这些因素对应答信度和效度的影响，明确估计随机和系统测量误差和测量效度。评分量表在社会科学研究中无处不在，因为它们是衡量态度、信念、行为意图、行为经验以及学者、政策制定者和商业规划者非常感兴趣的一系列其他现象的方便工具。然而，研究人员在不同的调查中使用了各种不同的量表长度，这表明，对于评定量表的最佳长度，以最大限度地提高所获得的测量结果的准确性，并没有达成共识。但是很容易想象，评分量表可能太长，使受访者难以使用它们，因为量表点的含义将是模糊的。评级量表也可能太短，因此不是最优的，因为它们将人们分为几个粗略的类别（例如，将支持法律的人与反对法律的人分开，而不是允许人们表达支持或反对的程度）。这个项目将有助于提高社会科学测量的准确性，通过比较不同长度的评分量表的有效性，以确定哪些是最佳的。该研究将调查对于更熟悉的主题，对于所有点都标有单词的量表（而不是只有终点被标记为这样），以及对于中间有零点的量表（例如，从“非常不喜欢”到“既不喜欢也不喜欢”到“非常喜欢”），与那些在末尾有零点的量表（例如，从“一点也不重要”到“非常重要”）相比，最佳量表长度是否更大。该项目将测试认知能力更强、更有动力准确报告的人是否能够更有效地管理较长的评分量表。这些发现将有助于心理学文献关于能力和动机如何调节任务表现，并且这些发现将为政府，商业世界和学术界（心理学，社会学，政治学，经济学，教育学，传播学，人类学，医学和许多其他通过问卷进行测量的科学）的问卷设计者提供实用指导。