Efficient Algorithms for Sequential Decision-making

点击翻译按钮获取中文摘要

Machine learning allows computers to automatically detect patterns in data, and leverage it to make predictions or decisions in the real-world. In the last decade, we have witnessed an increasing number of technological and scientific fields gather large amounts of data, and use machine learning techniques for making data-driven decisions. Reinforcement learning (RL) is a subfield of machine learning that focuses on problems that involve making repeated, sequential decisions to interact with the world, collect data and reason about it, all with incomplete information about the world. Applications of such problems include clinical trials in medicine, monitoring industrial plants, robotics, and computational marketing and advertising. Though RL has found great practical success recently, typical practical algorithms are often (i) brittle, meaning that their performance is sensitive to hyper-parameters and minor design decisions, (ii) inefficient in that they require a large number of interactions to learn to make good decisions and, (iii) do not have theoretical guarantees on their performance and can fail on simple, synthetic problems. To alleviate these problems, we propose to develop statistically efficient, computationally tractable algorithms that can easily scale to large sequential decision-making problems. Throughout, our aim will be to develop algorithms that (i) are either parameter-free or robust to hyper-parameter tuning, (ii) can effectively exploit the underlying problem structure and be sample-efficient and (iii) have tight bounds on their worst-case statistical and computational performance for representative problem classes The research program will especially focus on RL problems that need to incorporate constraints while making decisions, trade off multiple conflicting objectives and reason in the presence of other cooperating or competing agents.  The proposed research will help bridge the gap between the theory and practice of RL, and also contribute to the adjacent fields of machine learning and numerical optimization. Via collaborations with experts in industry and academia, we aim to use the developed techniques in healthcare, recommendation systems and computational advertising. By making progress towards the research program's objectives, we hope to expand the scope of data-driven decision-making in the real-world. Furthermore, we will provide key training to graduate students equipping them with strong mathematical foundations and programming skills necessary to solve important problems. We expect that the research program will augment Canada's existing strengths in machine learning, and help develop highly-skilled professionals valuable to Canadian institutions and companies.