DEEP NEURAL NETWORKS FOR ESTIMATION AND INFERENCE
DEEP NEURAL NETWORKS FOR ESTIMATION AND INFERENCE
复制标题
用于估计和推理的深度神经网络
DOI：
10.3982/ecta16901
复制
发表时间：
2021-01-01
期刊：
影响因子：
6.1
通讯作者：
Misra, Sanjog
中科院分区：
文献类型：
--
作者：
Farrell, Max H.;Liang, Tengyuan;Misra, Sanjog
我们研究深度神经网络及其在半参数推理中的应用。我们建立了新的深度前馈神经网络的非渐近高概率界。这些提供了足够快的收敛速度（在某些情况下是最小最大最优的），使我们能够在深度学习的第一步估计之后建立有效的第二步推理，这也是文献中的新结果。我们的非渐近高概率界，和随后的半参数推断，治疗目前的标准架构：全连接前馈神经网络（多层感知器），与现在常见的整流线性单位激活函数，无界的权重，和深度显式发散与样本大小。我们还讨论了其他架构，包括固定宽度的深度网络。我们建立了这些深度网络的非渐近界的一般类的非参数回归型损失函数，其中包括作为特殊情况的最小二乘，逻辑回归，和其他广义线性模型。然后，我们应用我们的理论来开发半参数推理，重点关注具体的因果参数，并通过实证应用于直邮营销来证明深度学习的有效性。
We study deep neural networks and their use in semiparametric inference. We establish novel nonasymptotic high probability bounds for deep feedforward neural nets. These deliver rates of convergence that are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second-step inference after first-step estimation with deep learning, a result also new to the literature. Our nonasymptotic high probability bounds, and the subsequent semiparametric inference, treat the current standard architecture: fully connected feedforward neural networks (multilayer perceptrons), with the now-common rectified linear unit activation function, unbounded weights, and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed-width, very deep networks. We establish the nonasymptotic bounds for these deep nets for a general class of nonparametric regression-type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, and demonstrate the effectiveness of deep learning with an empirical application to direct mail marketing.