Collaborative Research: Statistical Properties of Numerical Derivatives and Algorithms

数值微分法广泛应用于计量经济学和许多其他定量经济分析领域。计量经济学分析中许多需要区分的函数都需要从数据中进行估计。例如，估计估计量的近似方差通常需要估计定义估计量的矩条件的导数。通过寻找样本目标函数的一阶条件的零点也得到了许多估计量。估计的函数要么是不可微的，要么是难以解析微分的。通常估计的函数是复杂的，甚至在数值上计算也是具有挑战性的。经验研究人员经常应用数值微分方法，该方法依赖于通过使用软件例程在离散点上显式或隐式地对样本的估计函数取有限数量的差值，以近似未知真函数的导数。在有限差分运算中使用的步长是决定数值导数近似解析导数程度的关键调优参数。实证研究人员经常发现，不同的步长可能导致非常不同的数值导数估计。虽然数值导数在计量经济学、统计学和数学中的重要性并没有被忽视，但现有文献中可用的结果在范围上非常有限。该项目的目标是迈出重要的一步，为理解数值微分中获得最佳近似质量所需的步长条件提供一个系统框架。这些条件涉及到需要微分的函数的复杂性与数据样本中可用的信息量之间的微妙权衡，以及函数期望相对于抽样分布的平滑程度。经验过程理论为分析随机抽样数据下的函数复合体提供了有力的工具。这个项目的重点是分析数值导数在估计估计量的渐近方差和通过基于梯度的优化例程获得极值估计量中的使用。pi的第一个目标是给出在一致方差估计中允许不可微和不连续矩函数的一般充分一致性条件。我们得到的数值微分中步长的精确速率条件取决于力矩条件的偏差和非光滑程度之间的权衡。这些一般条件可以专门用于某些连续模型，对于这些模型，选择较小的步长只能有利于减小渐近偏差。然而，当渐近偏差低于某一阈值时，统计噪声将主导渐近偏差。该项目的第二个目标是分析一类基于有限样本目标函数的数值微分估计器，并提供基于数值导数的优化方法提供一致和渐近正态参数估计的条件。数值极值估计的条件要求当样本量增加到无穷大时，用于数值导数的步长必须以特定的速率收敛于零。渐近方差的一致性和估计量本身的收敛性所需要的条件可以是不同的。pi寻求广泛的结果，涵盖有限维参数模型，无限维半参数模型，以及由涉及采样数据的多层求和的u过程定义的模型。拟议的项目包括与斯坦福大学的Aprajit Mahajan教授合作。

Numerical differentiation is widely used in econometrics and many other areas of quantitative economic analysis. Many functions that need to be differentiated in econometric analysis need to be estimated from the data. For example, estimating the approximate variance of an estimator often requires estimating the derivatives of the moment conditions that define the estimator. Many estimators are also obtained by finding the zeros of the first order condition of the sample objective functions.The estimated functions can be either non-differentiable or difficult to differentiate analytically. Oftentimes the estimated functions are complex and can be challenging to compute even numerically. Empirical researchers often apply numerical differentiation methods which depend on taking a finite number of differences of the objective function at discrete points, either explicitly or implicitly through the use of software routines, to the estimated functions from the sample in order to approximate the derivative of the unknown true functions.A key tuning parameter that determines how well the numerical derivatives approximate the analytic derivatives is the step size used in the finite differencing operation. Empirical researchers often find that different step sizes can lead to very different numerical derivative estimates. Whilethe importance of numerical derivatives has not gone unnoticed in econometrics, statistics and mathematics, the results that are available in the existing literature are very limited in scope.The goal of this project is to take an important step to provide a systematic framework for understanding the conditions on the step size in numerical di^erentiation that are needed to obtain the optimal quality of approximation. These conditions involve subtle tradeoffs between the complexity of the function that needs to be differentiated and the amount of information that is available in the sample of data, and the degree of smoothness of the expectation of the function with respect to the sampling distribution. Empirical process theory provides a powerful tool for analyzing the complex of functions in the presence of randomly sampled data.This project focuses on analyzing the use of numerical derivatives in estimating the asymptotic variance of estimators and in obtaining extreme estimators through gradient based optimization routines. The PIs' first goal is to give general sufficient consistency conditions that allow for nondifferentiable and discontinuous moment functions in consistent variance estimation. The precise rate conditions for the step size in numerical differentiation that we obtain depend on the tradeoff between bias and the degree of nonsmoothness of the moment condition. These general conditionscan be specialized for certain continuous models, for which choosing a smaller step size can only be beneficial in reducing the asymptotic bias. However, the asymptotic bias will be dominated by the statistical noise once it falls below a certain threshold. The second goal of this project is to analyze a class of estimators that are based on numerically differentiating a finite sample objective function, and provide conditions under which numerical derivative based optimization methods deliver consistent and asymptotic normal parameter estimates. The conditions for numerical extreme estimators require that the step size used in thenumerical derivative has to converge to zero at specific rates when the sample size increases to infinity. The conditions required for the consistency of the asymptotic variance and for the convergence of the estimator itself can be different. The PIs seek extensive results that cover finite dimensionalparametric models, infinite dimensional semiparametric models, and models that are defined by U-processes involving multiple layers of summation over the sampling data. The proposed project involves joint work with Professor Aprajit Mahajan from Stanford University.