On the S-instability and degeneracy of discrete deep learning models
On the S-instability and degeneracy of discrete deep learning models
复制标题
离散深度学习模型的S-不稳定性和简并性
DOI：
--
复制
发表时间：
2016
期刊：
影响因子：
--
通讯作者：
S. Vardeman
中科院分区：
文献类型：
--
作者：
Andee Kaplan;D. Nordman;S. Vardeman
如果数据结果中的微小变化导致概率的大变化(通常是意想不到的)，则概率模型表现出不稳定性。这种不稳定性是概率模型的一种属性，由分布形式和给定的参数配置给出。对于在几个应用领域中发现的相关数据结构，人们对识别模型概率结构中的这种敏感性越来越感兴趣。我们考虑了定义在观测序列上的一般概率模型的不稳定性量化问题，其中每个长度为$N$的序列在每个点都有有限数量的可能值。以$N$为索引的概率模型序列和相关联的参数序列结果，以适应扩展维度的数据。形式上表明，当这类模型下的某个对数概率比增长快于$N$时，模型不稳定。在这种情况下，数据序列中的一个分量变化可以使概率移动几个数量级。此外，随着不稳定性变得更加极端，由此产生的概率模型显示出趋于退化，将它们的所有概率放在潜在的样本空间的一小部分上。这些关于不稳定性的结果适用于随机图、网络分析和机器学习环境中常用的大类模型。
A probability model exhibits instability if small changes in a data outcome result in large and, often unanticipated, changes in probability. This instability is a property of the probability model, given by a distributional form and a given configuration of parameters. For correlated data structures found in several application areas, there is increasing interest in identifying such sensitivity in model probability structure. We consider the problem of quantifying instability for general probability models defined on sequences of observations, where each sequence of length $N$ has a finite number of possible values that can be taken at each point. A sequence of probability models, indexed by $N$, and an associated parameter sequence result to accommodate data of expanding dimension. Model instability is formally shown to occur when a certain log probability ratio under such models grows faster than $N$. In this case, a one component change in the data sequence can shift probability by orders of magnitude. Also, as instability becomes more extreme, the resulting probability models are shown to tend to degeneracy, placing all their probability on potentially small portions of the sample space. These results on instability apply to large classes of models commonly used in random graphs, network analysis and machine learning contexts.