Optimizing the Number of Points on Rating Scales

社会科学家经常使用评分表设计调查问卷，以衡量社会和心理结构。然而，研究人员对于在这些量表上提供的最佳点数以最大限度地提高信度和效度并没有达成共识。这个项目将比较从2分长到11分长的评分标准的可信度和有效性。本项目还将探索任务的三个方面(生物极与单极结构；所有量表点上的言语标记与仅限终点的言语标记；话题熟悉度)和两个受访者特征(提供准确报告的认知技能和动机，以认知需求衡量)的调节作用。较长的标尺可能最适合两极结构、完全口头标记的标尺和非常熟悉的主题。为了解决这些问题，Knowledge Networks将对具有全国代表性的成年人样本进行三波调查。Knowledge Networks是一家调查研究公司，每周通过互联网从所有拥有同等互联网接入能力的受访者中收集数据。调查问卷将处理涉及一系列主题的目标评级量表上的分数，并将包括评估目标评级的同时有效性的标准措施。结构的极性和刻度点的语言标记的程度也将被操纵。将对问题熟悉度、受访者的认知技能以及他们提供准确报告的动机进行衡量。多组纵向结构方程模型将评估这些因素对回答的信度和效度的影响，明确估计随机和系统的测量误差和测量效度。评价量表在社会科学研究中普遍存在，因为它们是测量态度、信念、行为意图、行为体验和学者、政策制定者和商业规划者感兴趣的一系列其他现象的便捷工具。然而，研究人员在不同的调查中使用了各种不同的量表长度，这表明对于最大限度地提高测量结果的准确性，评级量表的最佳长度并没有达成共识。但不难想象，评级等级可能太长，让受访者很难使用，因为等级分数的含义会含糊不清。评级标准也可能太短，因此不是最优的，因为它们将人们集中在几个粗略的类别中(例如，将支持法律的人与反对法律的人分开，而不是允许人们表达支持或反对的程度)。该项目将通过比较不同长度的评级量表的有效性，以确定哪些是最优的，从而帮助提高社会科学衡量的准确性。这项研究将调查，对于更熟悉的主题，对于所有点都标有单词的标尺(而不仅仅是终点被这样标记)，以及对于中间有零点的标尺(例如，从“非常不喜欢”到“既不喜欢也不喜欢”到“非常喜欢”)，与那些结尾有零点的标尺(例如，从“根本不重要”到“极其重要”)相比，最佳音阶长度是否更大。该项目将测试认知能力更强、更有动力准确报道的人是否能够更有效地管理较长的评级等级。这些发现将有助于研究能力和动机如何影响任务绩效的心理学文献，这些发现将为政府、商业世界和学术界(心理学、社会学、政治学、经济学、教育学、传播学、人类学、医学和许多其他通过问卷进行测量的科学)的问卷设计者提供实践指导。

Social scientists routinely design questionnaires using rating scales to measure social and psychological constructs. There is no consensus, however, among researchers about the optimal number of points to offer on such scales so as to maximize reliability and validity. This project will compare the reliabilities and validities of rating scales ranging from 2 points long to 11 points long presented visually. This project also will explore the moderating roles of three aspects of the task (biopolar vs. unipolar constructs; verbal labeling on all scale points vs. the end points only; topic familiarity) and two respondent characteristics (cognitive skills and motivation to provide accurate reports, as gauged by need for cognition). Longer scales may be optimal for bipolar constructs, scales that are fully verbally labeled, and highly familiar topics. To address these issues, a three-wave survey will be conducted with a nationally representative sample of adults by Knowledge Networks, a survey research firm that collects data weekly via the Internet from an established panel of respondents who have all been equivalently equipped with Internet access. The questionnaire will manipulate number of points on target rating scales addressing a range of topics and will include criterion measures for assessing concurrent validity of the target ratings. Also manipulated will be polarity of the construct and the extent of verbal labeling of the scale points. Issue familiarity, respondents' cognitive skills, and their motivation to provide accurate reports will be measured. Multiple groups longitudinal structural equation modeling will be conducted to assess the effects of these factors on the reliability and validity of responses, explicitly estimating random and systematic measurement error and measurement validity.Rating scales are ubiquitous in social science research because they are convenient tools for measuring attitudes, beliefs, behavioral intentions, behavioral experiences, and a range of other phenomenon of great interest to scholars, policy-makers, and business planners. Researchers use a wide variety of different scale lengths in different investigations, however, suggesting that there is no consensus as to the optimal length of a rating scale to maximize the accuracy of the measurements obtained. But it is easy to imagine that rating scales may be too long, making it difficult for respondents to use them, because the meanings of the scale points will be ambiguous. And rating scales may also be too short and therefore suboptimal because they clump people into a few crude categories (e.g., separating the people who support a law from those who oppose it, rather than permitting people to express degrees of support or opposition). This project will help to improve the accuracy of social science measurements by comparing the effectiveness of rating scales of various lengths to identify which are optimal. The study will investigate whether the optimal scale length is greater for more familiar topics, for scales that have all points labeled with words (instead of only the end points being labeled as such), and for scales with a zero-point in the middle (e.g., from "dislike a great deal" through "neither like nor dislike" to "like a great deal"), as compared to those with a zero point at the end (e.g., from "not at all important" to "extremely important"). And the project will test whether people with greater cognitive skills and more motivation to report precisely are able to manage longer rating scales more effectively. The findings will contribute to the psychological literature on how ability and motivation moderate task performance, and the findings will provide practical guidance for questionnaire designers in government, in the commercial world, and in academia (in psychology, sociology, political science, economics, education, communication, anthropology, medicine, and many other sciences that make measurements via questionnaires).