ITR: Decision Making, Optimization, Learning, and Adaptation in Uncertain and Dynamic Environments

项目摘要这个项目研究不确定和动态环境中的决策、优化、学习和适应。主要目标是：(A)开发有效的学习方法(基于模拟或在线)，为不确定环境中的动态决策构建近最优策略。(B)开发新方法，并增强对某些现有方法的理解，以解决大规模动态决策问题。(C)开发与风险敏感性能标准有关的计算方法和学习算法，(D)建立多臂强盗问题和马尔可夫决策过程(MDP)中学习的基本限制，以所需学习量的下界的形式，并同时导出其要求与下界相匹配的最优算法。(E)为多智能体在动态环境中的学习和适应开发新的公式、模型和算法。所提出的研究涉及实际相关和智力上深入的问题。在应用方面，相关背景的范围很广。它包括物流(例如，供应链管理、库存控制、.eet分配)，制造系统(例如，排序和调度)，通信(例如，无线系统中的频率分配和管理，路由和拥塞控制)，Nance，机器人学，和经济系统(例如，拍卖和实时市场)。之所以出现智力上的优点，是因为一些问题提出了重大的数学挑战，需要新的方法和新的问题公式。这个项目将导致决策领域的科学知识库和最新技术的进步。此外，这项研究结合了运筹学和控制论(动态规划)、应用概率(随机逼近和大偏差)、艺术智力(学习)和经济学(博弈论)的方法。因此，这项工作的更广泛影响将是这些学科之间的交叉影响。更具体地，这种交叉影响不仅将通过出版物，而且将通过向来自不同社区的受众演示(例如，在机器学习和控制理论会议上)，以及通过开发新的课程。最后，在人力资源和培训层面，最直接的影响将通过对博士生的指导来实现，预计博士生将成为领先的学术机构的成员。

PROJECT SUMMARYThis project deals with decision making,optimization,learning,and adaptation in uncertainand dynamic environments.The main objectives are:(a)To develop e .cient learning methods (either simulation-based or on-line)for construct-ing near-optimal policies for dynamic decision making in uncertain environments.(b)To develop new methods and to enhance the understanding of certain existing methodsfor addressing large scale dynamic decision-making problems.(c)To develop computational methods and learning algorithms that pertain to risk-sensitiveperformance criteria,as well as fundamental limitations in the form of computationalcomplexity results.(d)To establish the fundamental limitations of learning in multi-armed bandit problemsand Markov decision processes (MDPs),in the form of lower bounds on the amount oflearning that is required,and simultaneously derive optimal algorithms whose require-ments match the lower bounds.(e)To develop new formulations,models,and algorithms for multi-agent learning andadaptation in dynamic environments.The proposed research involves problems that are both practically relevant and intellec-tually deep.On the application side,the range of relevant contexts is vast.It includes logistics(e.g.,supply chain management,inventory control,.eet assignment),manufacturing systems(e.g.,sequencing and scheduling),communications (e.g.,frequency allocation and hando .management in wireless systems,routing and congestion control),.nance,robotics,andeconomic systems (e.g.,auctions and real-time markets).The intellectual merit arises because some of the problems present signi .cant mathe-matical challenges,that require new approaches as well as new problem formulations.Thisproject will lead to advances in the scienti .c knowledge-base and the state of the art inthe .eld of decision making.In addition,this research combines methods from operationsresearch and control theory (dynamic programming),applied probability (stochastic approx-imation and large deviations),arti .cial intelligence (learning),and economics (game theory).As such,a broader impact of this work will be the advancement of the cross-fertilization ofthese disciplines.In more concrete terms,this cross-fertilization will occur not only throughpublications,but also through presentations at audiences from disparate communities (e.g.,at machine learning as well as control theory conferences),and also through the developmentof new courses.Finally,at the human resource and training level,the most direct impactwill occur through the mentoring of doctoral students,who will be expected to populateleading academic institutions.1