Intraclass Correlation Coefficient (ICC): A Framework for Monitoring and Assessing Performance of Trained Sensory Panels and Panelists
Intraclass Correlation Coefficient (ICC): A Framework for Monitoring and Assessing Performance of Trained Sensory Panels and Panelists
复制标题
DOI：
10.1111/j.1745-459x.2012.00399.x
复制
发表时间：
2012-10-01
影响因子：
2
通讯作者：
Kuesten, Carla
中科院分区：
文献类型：
--
作者：
Bi, Jian;Kuesten, Carla
感官测量是感官科学的基础。感官分析和决策在很大程度上依赖于感官数据的质量，这取决于训练有素的感官小组和小组成员的表现。已经提出了各种方法来监测和评估绩效。目前使用的方法的一个弱点是缺乏针对各种标准的统一框架，以及使用不同类型的数据进行的各种实验。本文建议使用准确性、效度和可靠性作为描述感官测量的通用术语，并使用组内相关系数(ICC)作为监控和评估性能的框架。ICC既可以衡量小组成员之间的相似性，也可以衡量小组成员和小组成员的敏感度。因此，ICC可以同时处理信度和效度的问题。对于不同的实验，可以从不同类型的数据中获得ICC。本文给出了从连续数据(评级)、多元连续数据、有序数据、排序数据、二元选择数据、多项选择数据和强迫选择数据中估计ICC的方程和R函数和S-Plus函数。还提供了多个ICC的可信区间、估计量的方差、与固定值的比较以及差异和相似性检验。讨论了Cronbach系数α与ICC之间的关系。在实际应用中，组内相关系数(ICC)可以在监测和评估训练有素的感官小组和小组成员的表现方面发挥框架作用。它可以作为感官数据质量的一个指标。ICC或Cronbach系数Alpha值越大，专家小组和专家小组成员的表现就越好。当且仅当ICC显著大于指定值时，才能认为该属性的数据质量是可接受的。如果统计检验不能显示ICC或Cronbach系数α显著大于指定的下限，例如0.1，则表明产品是无差别的，或者感官数据，至少就该属性而言，可能不是有效和可靠的。我们应该谨慎使用这些数据。
Sensory measurement underlies sensory science. Sensory analysis and decision-making heavily depend on the quality of sensory data, which is determined by the performance of trained sensory panels and panelists. Various methods have been proposed for monitoring and assessing the performance. A weakness of the currently used methods is lack of a unified framework for various criteria and a variety of experiments with different types of data. This paper proposes to use accuracy, validity and reliability as general terminologies to describe sensory measurement and to apply the intraclass correlation coefficient (ICC) as a framework for monitoring and assessing performance. ICC can measure both similarity among panelists and sensitivity of panels and panelists. Hence, ICC can handle the problems of both reliability and validity. ICC can be obtained from different types of data for diverse experiments. This paper provides the equations and R and S-Plus functions for estimations of ICCs from continuous data (ratings), multivariate continuous data, ordinal data, ranking data, binary-choice data, multiple-choice data and forced-choice data. Confidence intervals, variances of the estimators, comparison with a fixed value and difference and similarity tests for multiple ICCs are also provided. The relationship between Cronbach's coefficient alpha and ICC is discussed. Practical Applications Intraclass correlation coefficient (ICC) may play a framework role in monitoring and assessing performance of trained sensory panels and panelists. It can be used as an index of the quality of sensory data. The larger the ICC or Cronbach's coefficient alpha value, the better the performance of panels and panelists. If and only if an ICC is significantly larger than a specified value, can the quality of the data for that attribute be regarded as acceptable. A statistical test that fails to show that an ICC or Cronbach's coefficient alpha is significantly larger than a specified lowest limit, e.g., 0.1, suggests that the products are undiscriminating, or the sensory data, at least for that attribute, might not be valid and reliable. We should use that data with caution.