Collaborative Research: Statistical Properties of Numerical Derivatives and Algorithms

数值微分学广泛应用于计量经济学和其他许多定量经济分析领域。计量经济学分析中需要区分的许多功能需要从数据中估计出来。例如，估计估计器的近似方差通常需要估计定义估计器的矩条件的导数。许多估计量也是通过寻找样本目标函数的一阶条件的零点来获得的，估计的函数可以是不可微的，也可以是难以解析可微的。通常情况下，估计函数是复杂的，甚至在数值上也可能是具有挑战性的计算。经验研究人员经常应用数值微分方法，这些方法依赖于通过使用软件例程显式或隐式地将目标函数在离散点上的有限数目的差值取到样本中的估计函数，以逼近未知真函数的导数。决定数值导数逼近解析导数的一个关键调节参数是有限差分运算中使用的步长。经验研究人员经常发现，不同的步长可能会导致非常不同的数值导数估计。虽然数值导数在计量经济学、统计学和数学中的重要性并没有被忽视，但现有文献中可用的结果范围非常有限。本项目的目标是迈出重要的一步，为理解获得最佳逼近质量所需的数值微分中的步长条件提供一个系统的框架。这些条件涉及需要区分的函数的复杂性和数据样本中可用的信息量与函数预期相对于样本分布的平滑程度之间的微妙权衡。经验过程理论为分析随机抽样数据下函数的复杂性提供了强有力的工具。本课题主要研究如何利用数值导数来估计估计量的渐近方差，以及如何通过基于梯度的优化例程来获得极值估计量。PI的第一个目标是给出一般的充分一致性条件，允许相容方差估计中的不可微矩函数和不连续矩函数。我们得到的数值微分中步长的精确速率条件取决于偏倚和矩条件的非光滑程度之间的权衡。这些一般条件可以专门用于某些连续模型，对于这些模型，选择较小的步长只会有利于减少渐近偏差。然而，一旦渐近偏差降到某一阈值以下，就会被统计噪声所支配。这个项目的第二个目标是分析一类基于有限样本目标函数的数值微分的估计量，并提供基于数值导数的优化方法提供一致和渐近正态参数估计的条件。数值极值估计的条件要求，当样本量增加到无穷大时，数值导数中使用的步长必须以特定的速度收敛到零。渐近方差的一致性和估计器本身收敛所需的条件可以是不同的。PI寻求广泛的结果，涵盖有限维参数模型、无限维半参数模型，以及由涉及采样数据多层求和的U过程定义的模型。拟议的项目涉及与斯坦福大学的Aprajit Mahajan教授共同工作。

Numerical differentiation is widely used in econometrics and many other areas of quantitative economic analysis. Many functions that need to be differentiated in econometric analysis need to be estimated from the data. For example, estimating the approximate variance of an estimator often requires estimating the derivatives of the moment conditions that define the estimator. Many estimators are also obtained by finding the zeros of the first order condition of the sample objective functions.The estimated functions can be either non-differentiable or difficult to differentiate analytically. Oftentimes the estimated functions are complex and can be challenging to compute even numerically. Empirical researchers often apply numerical differentiation methods which depend on taking a finite number of differences of the objective function at discrete points, either explicitly or implicitly through the use of software routines, to the estimated functions from the sample in order to approximate the derivative of the unknown true functions.A key tuning parameter that determines how well the numerical derivatives approximate the analytic derivatives is the step size used in the finite differencing operation. Empirical researchers often find that different step sizes can lead to very different numerical derivative estimates. Whilethe importance of numerical derivatives has not gone unnoticed in econometrics, statistics and mathematics, the results that are available in the existing literature are very limited in scope.The goal of this project is to take an important step to provide a systematic framework for understanding the conditions on the step size in numerical di^erentiation that are needed to obtain the optimal quality of approximation. These conditions involve subtle tradeoffs between the complexity of the function that needs to be differentiated and the amount of information that is available in the sample of data, and the degree of smoothness of the expectation of the function with respect to the sampling distribution. Empirical process theory provides a powerful tool for analyzing the complex of functions in the presence of randomly sampled data.This project focuses on analyzing the use of numerical derivatives in estimating the asymptotic variance of estimators and in obtaining extreme estimators through gradient based optimization routines. The PIs' first goal is to give general sufficient consistency conditions that allow for nondifferentiable and discontinuous moment functions in consistent variance estimation. The precise rate conditions for the step size in numerical differentiation that we obtain depend on the tradeoff between bias and the degree of nonsmoothness of the moment condition. These general conditionscan be specialized for certain continuous models, for which choosing a smaller step size can only be beneficial in reducing the asymptotic bias. However, the asymptotic bias will be dominated by the statistical noise once it falls below a certain threshold. The second goal of this project is to analyze a class of estimators that are based on numerically differentiating a finite sample objective function, and provide conditions under which numerical derivative based optimization methods deliver consistent and asymptotic normal parameter estimates. The conditions for numerical extreme estimators require that the step size used in thenumerical derivative has to converge to zero at specific rates when the sample size increases to infinity. The conditions required for the consistency of the asymptotic variance and for the convergence of the estimator itself can be different. The PIs seek extensive results that cover finite dimensionalparametric models, infinite dimensional semiparametric models, and models that are defined by U-processes involving multiple layers of summation over the sampling data. The proposed project involves joint work with Professor Aprajit Mahajan from Stanford University.