ITR: Decision Making, Optimization, Learning, and Adaptation in Uncertain and Dynamic Environments

本项目涉及不确定和动态环境中的决策、优化、学习和适应。主要目标是：(a)发展e。为不确定环境下的动态决策构建接近最优策略的客户学习方法（基于模拟或在线）。(b)发展新方法和增进对某些现有方法的了解，以处理大规模动态决策问题。(c)开发与风险敏感性能标准有关的计算方法和学习算法，以及计算复杂性结果形式的基本限制。(d)建立多臂强盗问题和马尔可夫决策过程（mdp）中学习的基本限制，以所需学习量的下界的形式，同时推导出要求与下界匹配的最优算法。(e)为动态环境中的多智能体学习和适应开发新的公式、模型和算法。所提出的研究涉及的问题既具有实际意义又具有学术深度。在应用方面，相关上下文的范围是巨大的。它包括物流(例如；、供应链管理、库存控制等。Eet分配)，制造系统(例如：例如，排序和调度)，通信（例如，频率分配和handdo）。无线系统管理，路由和拥塞控制)。金融、机器人和经济系统（如拍卖和实时市场）。智力上的价值之所以产生，是因为有些问题提出了重大的数学挑战，需要新的方法和新的问题表述。这个项目将导致科学知识基础的进步和技术水平的提高。决策领域。此外，本研究结合了运筹学和控制论（动态规划）、应用概率论（随机逼近和大偏差）等方法。社交智能（学习）和经济学（博弈论）。因此，这项工作的一个更广泛的影响将是这些学科的交叉受精的进步。更具体地说，这种相互影响不仅会通过出版物，还会通过在不同社区的听众面前发表演讲（例如，在机器学习和控制理论会议上），以及通过开发新课程来实现。最后，在人力资源和培训层面，最直接的影响将通过博士生的指导产生，这些博士生将有望在领先的学术机构中普及

PROJECT SUMMARYThis project deals with decision making,optimization,learning,and adaptation in uncertainand dynamic environments.The main objectives are:(a)To develop e .cient learning methods (either simulation-based or on-line)for construct-ing near-optimal policies for dynamic decision making in uncertain environments.(b)To develop new methods and to enhance the understanding of certain existing methodsfor addressing large scale dynamic decision-making problems.(c)To develop computational methods and learning algorithms that pertain to risk-sensitiveperformance criteria,as well as fundamental limitations in the form of computationalcomplexity results.(d)To establish the fundamental limitations of learning in multi-armed bandit problemsand Markov decision processes (MDPs),in the form of lower bounds on the amount oflearning that is required,and simultaneously derive optimal algorithms whose require-ments match the lower bounds.(e)To develop new formulations,models,and algorithms for multi-agent learning andadaptation in dynamic environments.The proposed research involves problems that are both practically relevant and intellec-tually deep.On the application side,the range of relevant contexts is vast.It includes logistics(e.g.,supply chain management,inventory control,.eet assignment),manufacturing systems(e.g.,sequencing and scheduling),communications (e.g.,frequency allocation and hando .management in wireless systems,routing and congestion control),.nance,robotics,andeconomic systems (e.g.,auctions and real-time markets).The intellectual merit arises because some of the problems present signi .cant mathe-matical challenges,that require new approaches as well as new problem formulations.Thisproject will lead to advances in the scienti .c knowledge-base and the state of the art inthe .eld of decision making.In addition,this research combines methods from operationsresearch and control theory (dynamic programming),applied probability (stochastic approx-imation and large deviations),arti .cial intelligence (learning),and economics (game theory).As such,a broader impact of this work will be the advancement of the cross-fertilization ofthese disciplines.In more concrete terms,this cross-fertilization will occur not only throughpublications,but also through presentations at audiences from disparate communities (e.g.,at machine learning as well as control theory conferences),and also through the developmentof new courses.Finally,at the human resource and training level,the most direct impactwill occur through the mentoring of doctoral students,who will be expected to populateleading academic institutions.1