Efficient Algorithms for Sequential Decision-making

机器学习允许计算机自动检测数据中的模式，并利用它在现实世界中做出预测或决策。在过去的十年中，我们目睹了越来越多的技术和科学领域收集大量数据，并使用机器学习技术来做出数据驱动的决策。强化学习（RL）是机器学习的一个子领域，专注于涉及做出重复的、顺序的决策以与世界互动的问题，收集数据并对其进行推理，所有这些都是关于世界的不完整信息。这些问题的应用包括医学临床试验、工业工厂监控、机器人技术、计算机营销和广告。尽管RL最近在实践中取得了巨大的成功，但典型的实用算法通常是(i)脆弱的，这意味着它们的性能对超参数和次要的设计决策很敏感；（ii）低效的，因为它们需要大量的交互来学习做出好的决策；（iii）对它们的性能没有理论保证，在简单的综合问题上可能会失败。为了缓解这些问题，我们建议开发统计上高效、计算上易于处理的算法，这些算法可以很容易地扩展到大型顺序决策问题。在整个过程中，我们的目标将是开发(i)无参数或对超参数调优具有鲁棒性的算法，（ii）可以有效地利用潜在的问题结构并具有样本效率，以及（iii）对于代表性问题类别的最坏情况统计和计算性能具有严格的界限。研究计划将特别关注需要在决策时纳入约束的强化学习问题。在存在其他合作或竞争代理的情况下权衡多个相互冲突的目标和原因。所提出的研究将有助于弥合强化学习理论与实践之间的差距，并为机器学习和数值优化的邻近领域做出贡献。通过与工业界和学术界专家的合作，我们的目标是将开发的技术应用于医疗保健、推荐系统和计算广告。通过朝着研究计划的目标取得进展，我们希望在现实世界中扩大数据驱动决策的范围。此外，我们将为研究生提供关键培训，使他们具备强大的数学基础和解决重要问题所需的编程技能。我们期望该研究项目将增强加拿大在机器学习方面的现有优势，并帮助培养对加拿大机构和公司有价值的高技能专业人员。

Machine learning allows computers to automatically detect patterns in data, and leverage it to make predictions or decisions in the real-world. In the last decade, we have witnessed an increasing number of technological and scientific fields gather large amounts of data, and use machine learning techniques for making data-driven decisions. Reinforcement learning (RL) is a subfield of machine learning that focuses on problems that involve making repeated, sequential decisions to interact with the world, collect data and reason about it, all with incomplete information about the world. Applications of such problems include clinical trials in medicine, monitoring industrial plants, robotics, and computational marketing and advertising. Though RL has found great practical success recently, typical practical algorithms are often (i) brittle, meaning that their performance is sensitive to hyper-parameters and minor design decisions, (ii) inefficient in that they require a large number of interactions to learn to make good decisions and, (iii) do not have theoretical guarantees on their performance and can fail on simple, synthetic problems. To alleviate these problems, we propose to develop statistically efficient, computationally tractable algorithms that can easily scale to large sequential decision-making problems. Throughout, our aim will be to develop algorithms that (i) are either parameter-free or robust to hyper-parameter tuning, (ii) can effectively exploit the underlying problem structure and be sample-efficient and (iii) have tight bounds on their worst-case statistical and computational performance for representative problem classes The research program will especially focus on RL problems that need to incorporate constraints while making decisions, trade off multiple conflicting objectives and reason in the presence of other cooperating or competing agents.  The proposed research will help bridge the gap between the theory and practice of RL, and also contribute to the adjacent fields of machine learning and numerical optimization. Via collaborations with experts in industry and academia, we aim to use the developed techniques in healthcare, recommendation systems and computational advertising. By making progress towards the research program's objectives, we hope to expand the scope of data-driven decision-making in the real-world. Furthermore, we will provide key training to graduate students equipping them with strong mathematical foundations and programming skills necessary to solve important problems. We expect that the research program will augment Canada's existing strengths in machine learning, and help develop highly-skilled professionals valuable to Canadian institutions and companies.