On-line methods in machine learning

建议的研究集中在随机环境中的在线学习和顺序决策。顺序决策问题的例子比比皆是：控制工业厂房，进行临床试验，管理投资，监测管道系统等在线学习算法提高了系统的性能在其正常运行。相比之下，离线学习算法使用固定量的数据进行训练，然后在关闭学习的情况下使用。在离线学习中，训练数据必须完全代表系统的行为，否则性能可能很差。在线学习系统克服了这个潜在的弱点，因为他们永远不会停止适应。在线学习是具有挑战性的，因为学习和控制是交织在一起的。目前的在线学习解决方案仅限于小的有限域或作出强有力的假设（如不确定性是参数），严重限制了他们的适用性，因为复杂的环境不满足这个条件，在这里，我建议开发和研究在线学习算法，工作在大型，复杂的环境。我计划集中在以下几个方面：（一）理解什么使有效的在线学习成为可能，（二）表征在线学习算法的行为，（三）开发在线学习算法，是有效的数据和计算。为了实现这一目标，我从理论上分析了算法的性能，推导出的界限，显示他们的优势，揭示他们的弱点，其次是努力改善他们的弱点points.Although提供良好的在线性能在大规模的，现实的环境是一个雄心勃勃的目标，在这方面的进展很可能会发现重大的未来，现实世界的应用。

The proposed research is focused on on-line learning and sequential decision making in stochastic environments. Examples of sequential decision making problems abound: controlling industrial plants, performing clinical trials, managing investments, monitoring pipeline-systems, etc. An on-line learning algorithm improves the performance of a system during its normal operation. In contrast, off-line learning algorithms are trained with a fixed amount of data and then used with learning turned off. In off-line learning the training data has to be fully representative of the system's behaviour or performance may be poor. On-line learning systems overcome this potential weakness because they never stop adapting.On-line learning is challenging since learning and control is interleaved. Current on-line learning solutions are limited to small finite domains or make strong assumptions (such as that the uncertainty is parametric) that seriously limit their applicability since complex environments do not meet this conditions.Here I propose to develop and study on-line learning algorithms that work in large, complex environments. I plan to concentrate on the following aspects: (i) Understanding what makes efficient on-line learning possible, (ii) characterizing the behaviour of on-line learning algorithms, (iii) developing on-line learning algorithms that are efficient in terms of both data and computation. In order to accomplish this goal, I analyse the performance of the algorithms theoretically, deriving bounds that show their strengths and reveal their weaknesses, followed by an effort to improve their weakest points.Although delivering good on-line performance in large-scale, realistic environments is an ambitious goal, progress on this area is likely to find significant future, real-world applications.