Distributed Resource Allocation and Decision Making under Uncertainty: A Cooperation Perspective

基于蜂窝基础设施的设备到设备（D2D）通信是未来无线网络的关键技术之一。其基本思想是使适当选择的附近设备对能够重用蜂窝频谱进行直接数据传输，同时确保不会对通过基站（BS）进行的传统蜂窝传输产生有害影响。尽管D2D通信具有巨大的性能提升潜力，但它给系统设计人员带来了一些根本性的挑战。这些挑战，包括资源分配和传输模式选择，由于缺乏及时和准确的信道状态信息，直接D2D链路在BSs和无线设备的水平上加剧。因此，为了避免反馈和信号开销的显著增加，强烈需要D2D资源分配解决方案(i)适合分布式实现；（ii）有能力处理不确定性；（iii）能够有益地利用任何可用的附带信息。该项目的核心目标是开发和研究一个新的理论框架，用于网络辅助D2D资源分配，该框架结合了博弈论和强化学习。我们将分布式D2D无线网络建模为一个多智能体系统，其中一组具有有限理性的智能体根据一定的决策策略采取行动，共享有限的资源。每个联合行动都与每个代理的奖励相关，代理的行为随着时间的推移而演变，作为过去结果和（可能）观察到的副信息的函数。在一般的多智能体系统中，智能体的行为要么是竞争的，要么是合作的。在这个项目中，重点研究在不确定和缺乏先验知识的情况下，合作用户的行为。合作要求智能体分担学习过程的成本，例如通过交换信息，这样用户的激励和真实性就发挥了主要作用。此外，由于缺乏对效用函数的先验知识，以及其他代理的行为、激励和类型，代理的策略会因为不断的信息获取和后续学习而发生变化。因此，针对全信息博弈开发的传统解并不适用于不完全信息博弈，寻找新的解概念势在必行。简而言之，目标是(i)将传统的合作博弈论模型和解决方案概念推广到不完全信息的博弈中，（ii）将单智能体学习模型推广到包括多个合作学习智能体的学习场景中，（iii）研究广义模型和开发的解决方案在解决D2D无线网络中资源分配和模式选择问题中的应用。

Device-to-device (D2D) communications underlaying a cellular infrastructure is one of the key technology enablers for future wireless networks. The basic idea consists in enabling suitably-selected nearby device pairs to reuse the cellular spectrum for direct data transfer, while ensuring that there is no detrimental impact on traditional cellular transmissions via base stations (BS). Despite its great potential for performance gains, D2D communications poses some fundamental challenges to system designers. These challenges, which include resource allocation and transmission mode selection, are exacerbated by the lack of timely and accurate channel state information for direct D2D links at the level of BSs and wireless devices. Therefore, in order to avoid a significant increase in the feedback and signaling overhead, there is a strong need for D2D resource allocation solutions that (i) are amenable to distributed implementation; (ii) are capable of dealing with uncertainty; (iii) can beneficially exploit any available side-information. The core objective of this project is to develop and study a novel theoretical framework for network-assisted D2D resource allocation that incorporates game theory and reinforcement learning. We model a distributed D2D wireless network as a multi-agent system, in which a set of smart agents with bounded rationality share limited resources, by taking actions according to some decision making strategy. Every joint action profile is associated with some reward for each agent, and agents' actions evolve over time as a function of past outcomes and (possibly) observed side-information. In a general multi-agent system, agents behave either competitively or cooperatively. In this project, the focus shall be on studying cooperative users' behavior under uncertainty and lack of prior knowledge. Being cooperative requires agents to share the cost of the learning process, for instance by exchanging the information, so that users' incentives and truthfulness play the major roles. Moreover, due to lack of prior knowledge on utility functions, as well as other agents' actions, incentives and types, agents' strategies are subject to change, as a consequence of continuous information acquisition and subsequent learning. Therefore, traditional solutions developed for full-information games are not applicable to games with incomplete-information, and it is imperative to look for new solution concepts. In brief, the objectives are (i) to generalize conventional cooperative game-theoretical models and solution concepts to games with incomplete information, (ii) to generalize single-agent learning models to learning scenarios that include multiple cooperative learning agents, (iii) to investigate the applications of generalized models and developed solutions in solving resource allocation and mode selection problems in D2D wireless networks.