The Bias-Expressivity Trade-off
The Bias-Expressivity Trade-off
复制标题
偏差与表现力的权衡
DOI：
--
复制
发表时间：
2019
期刊：
影响因子：
--
通讯作者：
George D. Montañez
中科院分区：
文献类型：
--
作者：
Julius Lauw;D. Macias;Akshay Trikha;Julia Vendemiatti;George D. Montañez
学习算法需要偏差来概括并比随机猜测表现得更好。我们检查有偏差的算法的灵活性（表现力）。表达算法可以适应不断变化的训练数据，根据输入的变化改变其结果。我们通过在算法结果分布上使用熵的信息论概念来测量表现力，证明偏差和表现力之间的权衡。算法的偏差程度是指其优于均匀随机采样的程度，但也是算法变得不灵活的程度。我们得出了将偏差与表现力相关的界限，证明了尝试创建性能强大但灵活的算法所固有的必要权衡。
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic notion of entropy on algorithm outcome distributions, demonstrating a trade-off between bias and expressivity. To the degree an algorithm is biased is the degree to which it can outperform uniform random sampling, but is also the degree to which is becomes inflexible. We derive bounds relating bias to expressivity, proving the necessary trade-offs inherent in trying to create strongly performing yet flexible algorithms.