Optimizing the Number of Points on Rating Scales

优化评分量表的点数

• 批准号: 0645931
• 负责人: Jon Krosnick
• 金额: --
• 依托单位: University of Illinois at Chicago
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-07-01 至 2011-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0645931&HistoricalAwards=false
• 关键词: Optimizing; Number; Points; Rating; Scales

Social scientists routinely design questionnaires using rating scales to measure social and psychological constructs. There is no consensus, however, among researchers about the optimal number of points to offer on such scales so as to maximize reliability and validity. This project will compare the reliabilities and validities of rating scales ranging from 2 points long to 11 points long presented visually. This project also will explore the moderating roles of three aspects of the task (biopolar vs. unipolar constructs; verbal labeling on all scale points vs. the end points only; topic familiarity) and two respondent characteristics (cognitive skills and motivation to provide accurate reports, as gauged by need for cognition). Longer scales may be optimal for bipolar constructs, scales that are fully verbally labeled, and highly familiar topics. To address these issues, a three-wave survey will be conducted with a nationally representative sample of adults by Knowledge Networks, a survey research firm that collects data weekly via the Internet from an established panel of respondents who have all been equivalently equipped with Internet access. The questionnaire will manipulate number of points on target rating scales addressing a range of topics and will include criterion measures for assessing concurrent validity of the target ratings. Also manipulated will be polarity of the construct and the extent of verbal labeling of the scale points. Issue familiarity, respondents' cognitive skills, and their motivation to provide accurate reports will be measured. Multiple groups longitudinal structural equation modeling will be conducted to assess the effects of these factors on the reliability and validity of responses, explicitly estimating random and systematic measurement error and measurement validity.Rating scales are ubiquitous in social science research because they are convenient tools for measuring attitudes, beliefs, behavioral intentions, behavioral experiences, and a range of other phenomenon of great interest to scholars, policy-makers, and business planners. Researchers use a wide variety of different scale lengths in different investigations, however, suggesting that there is no consensus as to the optimal length of a rating scale to maximize the accuracy of the measurements obtained. But it is easy to imagine that rating scales may be too long, making it difficult for respondents to use them, because the meanings of the scale points will be ambiguous. And rating scales may also be too short and therefore suboptimal because they clump people into a few crude categories (e.g., separating the people who support a law from those who oppose it, rather than permitting people to express degrees of support or opposition). This project will help to improve the accuracy of social science measurements by comparing the effectiveness of rating scales of various lengths to identify which are optimal. The study will investigate whether the optimal scale length is greater for more familiar topics, for scales that have all points labeled with words (instead of only the end points being labeled as such), and for scales with a zero-point in the middle (e.g., from "dislike a great deal" through "neither like nor dislike" to "like a great deal"), as compared to those with a zero point at the end (e.g., from "not at all important" to "extremely important"). And the project will test whether people with greater cognitive skills and more motivation to report precisely are able to manage longer rating scales more effectively. The findings will contribute to the psychological literature on how ability and motivation moderate task performance, and the findings will provide practical guidance for questionnaire designers in government, in the commercial world, and in academia (in psychology, sociology, political science, economics, education, communication, anthropology, medicine, and many other sciences that make measurements via questionnaires).

社会科学家通常使用评级量表设计问卷，以衡量社会和心理结构。 然而，研究人员对于在这种量表上提供的最佳点数并没有达成共识，以便最大限度地提高可靠性和有效性。 本专题将比较视觉呈现的2分至11分等级量表的信度和效度。 该项目还将探讨任务的三个方面（双极与单极结构;所有量表点与终点的口头标记;主题熟悉度）和两个受访者特征（认知技能和提供准确报告的动机，以认知需求衡量）的调节作用。 对于双极结构、完全口头标记的量表和高度熟悉的主题，较长的量表可能是最佳的。 为了解决这些问题，调查研究公司“知识网络”将对具有全国代表性的成年人进行三波调查，该公司每周通过互联网从一个已确定的、具备同等互联网接入能力的调查对象小组收集数据。 调查问卷将操纵目标评级量表上的点数，涉及一系列主题，并将包括评估目标评级同时有效性的标准措施。 也将操纵的极性的建设和程度的口头标记的规模点。 问题的熟悉程度，受访者的认知技能，以及他们提供准确报告的动机将被衡量。 多组纵向结构方程模型将被用来评估这些因素对回答的信度和效度的影响，明确估计随机和系统测量误差和测量效度。评定量表在社会科学研究中无处不在，因为它们是测量态度，信念，行为意图，行为经验，以及学者、政策制定者和商业规划者非常感兴趣的一系列其他现象。 然而，研究人员在不同的调查中使用了各种不同的量表长度，这表明对于最佳的量表长度没有达成共识，以最大限度地提高所获得的测量结果的准确性。 但很容易想象，评级量表可能太长，使受访者难以使用它们，因为量表点的含义将是模糊的。 评级量表也可能太短，因此是次优的，因为它们把人分成了几个粗略的类别（例如，将支持法律的人与反对法律的人分开，而不是允许人们表达支持或反对的程度）。 该项目将通过比较不同长度的评级量表的有效性来确定哪些是最佳的，从而有助于提高社会科学测量的准确性。 该研究将调查最佳尺度长度是否对于更熟悉的主题更大，对于所有点都用单词标记的尺度（而不是只有端点被标记为这样），以及对于中间有零点的尺度（例如，从“非常不喜欢”到“既不喜欢也不喜欢”到“非常喜欢”），与那些在结尾处具有零点的（例如，从“根本不重要”到“极其重要”）。 该项目将测试具有更好的认知技能和更有动机准确报告的人是否能够更有效地管理更长的评级量表。 研究结果将有助于心理学文献中的能力和动机如何调节任务绩效，研究结果将为政府，商业世界和学术界（心理学，社会学，政治学，经济学，教育学，传播学，人类学，医学和许多其他通过问卷进行测量的科学）的问卷设计者提供实用指导。