Variance Reduction Techniques for the Identification of Noisy Systems

识别具有高噪声水平的系统是非常具有挑战性的，因为这类系统的模型通常受到高模型方差的困扰。这导致了更高的预期预测误差。这个项目将研究两种有希望的新方法来减少模型方差，从而减少预测误差的方差：(1)用于前馈和递归网络的新的平滑正则化，用于在施加期望的模型偏差的同时减少模型方差。PI预计，他的新的平滑正则化在许多感兴趣的情况下将优于标准的二次权重衰减和特别方法。(2)用新的委员会Bootstrap方法减小模型方差引起的预测误差。这些措施包括委员会内部培训和验证集的独立自举、相互培训和模型选择方法以及强大的适应性委员会。PI预计，他的新的委员会自举方法将实现比单个网络或传统委员会平均方法更好的训练、更好的模型选择和更大的方差减少。这项研究将涉及宏观经济学、生理学和工程学中关于噪声时间序列预测问题的新的分析工作、算法开发和广泛的实证测试。

The identification of systems with high noise levels is very challenging, since models of such systems are typically plagued by high model variance. This lead to higher expected prediction errors. This project will investigate two promising new approaches for reducing model variance and thus the variance of prediction errors: (1) New classes of smoothing regularizes for both feedforward and recurrent networks for reducing model variance while imposing desirable model biases. The PI expect, that his new smoothing regularizes will outperform standard quadratic weight decay, and ad hoc methods, in many cases of interest. (2) New committee bootstrap methods for reducing the prediction errors due to model variance. These include independent bootstrapping of training and validation sets within the committee, mutual training and model selection methods, and robust adaptive committees. The PI expects that his new committee bootstrap methods will achieve better training, better model selection, and greater variance reduction than is attainable be individual networks or by conventional committee averaging methods. The research will involve new analytical work, algorithm development, and extensive empirical testing of the algorithms on noisy time series prediction problems n macroeconomics, physiology, and engineering.