ITR: Decision Making, Optimization, Learning, and Adaptation in Uncertain and Dynamic Environments

本项目研究不确定动态环境中的决策、优化、学习和适应，主要目标是：（a）发展有效的学习方法（基于模拟或在线），为不确定环境中的动态决策构造近似最优策略。(b)To开发新的方法，并加强对解决大规模动态决策问题的某些现有方法的理解。(c)To开发计算方法和学习算法，涉及风险敏感的性能标准，以及计算复杂性结果形式的基本限制。(d)To建立了多臂强盗问题和马尔可夫决策过程（MDP）中学习的基本限制，以所需学习量的下界的形式，同时导出了其要求与下界相匹配的最优算法。(e)To为动态环境中的多智能体学习和适应开发新的公式、模型和算法。拟议的研究涉及实际相关和智力深度的问题。在应用方面，相关背景的范围很广。它包括物流（例如，供应链管理、库存控制、EET分配），制造系统（例如，排序和调度），通信（例如，无线系统中的频率分配和切换管理、路由选择和拥塞控制）、通信、机器人技术和经济系统（例如，拍卖和实时市场）。智力价值的产生是因为一些问题提出了重大的数学挑战，需要新的方法和新的问题表述。本项目将导致科学知识基础和决策领域的最新技术的进步。此外，本研究结合了运筹学和控制论的方法（动态规划），应用概率（随机近似和大偏差），阿尔蒂.社会智能（学习）和经济学（博弈论）。因此，这项工作的更广泛影响将是促进这些学科的交叉施肥。更具体地说，这种交叉施肥将不仅通过出版物发生，而且还通过对来自不同社区的观众的演示（例如，最后，在人力资源和培训层面，最直接的影响来自于对博士生的指导，这些博士生有望进入领先的学术机构。1

PROJECT SUMMARYThis project deals with decision making,optimization,learning,and adaptation in uncertainand dynamic environments.The main objectives are:(a)To develop e .cient learning methods (either simulation-based or on-line)for construct-ing near-optimal policies for dynamic decision making in uncertain environments.(b)To develop new methods and to enhance the understanding of certain existing methodsfor addressing large scale dynamic decision-making problems.(c)To develop computational methods and learning algorithms that pertain to risk-sensitiveperformance criteria,as well as fundamental limitations in the form of computationalcomplexity results.(d)To establish the fundamental limitations of learning in multi-armed bandit problemsand Markov decision processes (MDPs),in the form of lower bounds on the amount oflearning that is required,and simultaneously derive optimal algorithms whose require-ments match the lower bounds.(e)To develop new formulations,models,and algorithms for multi-agent learning andadaptation in dynamic environments.The proposed research involves problems that are both practically relevant and intellec-tually deep.On the application side,the range of relevant contexts is vast.It includes logistics(e.g.,supply chain management,inventory control,.eet assignment),manufacturing systems(e.g.,sequencing and scheduling),communications (e.g.,frequency allocation and hando .management in wireless systems,routing and congestion control),.nance,robotics,andeconomic systems (e.g.,auctions and real-time markets).The intellectual merit arises because some of the problems present signi .cant mathe-matical challenges,that require new approaches as well as new problem formulations.Thisproject will lead to advances in the scienti .c knowledge-base and the state of the art inthe .eld of decision making.In addition,this research combines methods from operationsresearch and control theory (dynamic programming),applied probability (stochastic approx-imation and large deviations),arti .cial intelligence (learning),and economics (game theory).As such,a broader impact of this work will be the advancement of the cross-fertilization ofthese disciplines.In more concrete terms,this cross-fertilization will occur not only throughpublications,but also through presentations at audiences from disparate communities (e.g.,at machine learning as well as control theory conferences),and also through the developmentof new courses.Finally,at the human resource and training level,the most direct impactwill occur through the mentoring of doctoral students,who will be expected to populateleading academic institutions.1