Conceptual complexity and the bias/variance tradeoff
Conceptual complexity and the bias/variance tradeoff
复制标题
DOI：
10.1016/j.cognition.2010.10.004
复制
发表时间：
2011-01-01
期刊：
影响因子：
3.4
通讯作者：
Feldman, Jacob
中科院分区：
文献类型：
--
作者：
Briscoe, Erica;Feldman, Jacob
在本文中，我们建议将基于范例和基于原型的概念学习模型之间的传统二分法视为统计学习文献中已知的偏差/方差权衡的一个实例。偏差/方差权衡可以被认为是一个滑动刻度，它调节任何学习过程与其训练数据的紧密程度。在尺度的一端（高方差），模型可以接受非常复杂的假设，允许它们非常接近地拟合各种各样的数据，但结果是泛化效果很差，这种现象被称为过拟合。在尺度的另一端（高偏差），模型做出相对简单和不灵活的假设，结果可能不太适合数据，称为欠拟合。类别形成的范例模型和原型模型在这个尺度的两端：原型模型有高度偏差，因为它们假设一个简单的、标准的概念形式（原型），而范例模型的偏差很小，但方差很大，允许它们适应几乎任何训练数据的组合。我们调查了人类学习者在这个光谱上的位置，通过面对不同内在复杂性水平的类别结构，从简单的原型类类别到更复杂的多模态类别。结果表明，人类学习者在偏差/方差连续统上采用一个中间点，与大多数传统方法所占据的极点不一致。我们提出了一个简单的模型，它调整（正则化）其假设的复杂性以适应训练数据，它比代表性的范例和原型模型更适合实验数据。(C) 2010 Elsevier B.V.版权所有
In this paper we propose that the conventional dichotomy between exemplar-based and prototype-based models of concept learning is helpfully viewed as an instance of what is known in the statistical learning literature as the bias/variance tradeoff. The bias/variance tradeoff can be thought of as a sliding scale that modulates how closely any learning procedure adheres to its training data. At one end of the scale (high variance), models can entertain very complex hypotheses, allowing them to fit a wide variety of data very closely but as a result can generalize poorly, a phenomenon called overfitting. At the other end of the scale (high bias), models make relatively simple and inflexible assumptions, and as a result may fit the data poorly, called underfitting. Exemplar and prototype models of category formation are at opposite ends of this scale: prototype models are highly biased, in that they assume a simple, standard conceptual form (the prototype), while exemplar models have very little bias but high variance, allowing them to fit virtually any combination of training data. We investigated human learners' position on this spectrum by confronting them with category structures at variable levels of intrinsic complexity, ranging from simple prototype-like categories to much more complex multimodal ones. The results show that human learners adopt an intermediate point on the bias/variance continuum, inconsistent with either of the poles occupied by most conventional approaches. We present a simple model that adjusts (regularizes) the complexity of its hypotheses in order to suit the training data, which fits the experimental data better than representative exemplar and prototype models. (C) 2010 Elsevier B.V. All rights reserved.