Distributed Resource Allocation and Decision Making under Uncertainty: A Cooperation Perspective

作为蜂窝基础设施基础的设备到设备（D2 D）通信是未来无线网络的关键技术使能器之一。基本思想在于使得适当选择的附近设备对能够重用蜂窝频谱用于直接数据传输，同时确保对经由基站（BS）的传统蜂窝传输没有不利影响。尽管D2 D通信具有很大的性能提升潜力，但它对系统设计人员提出了一些基本挑战。包括资源分配和传输模式选择的这些挑战由于在BS和无线设备级别上缺乏用于直接D2 D链路的及时和准确的信道状态信息而加剧。因此，为了避免反馈和信令开销的显著增加，强烈需要以下D2 D资源分配解决方案：（i）服从分布式实现;（ii）能够处理不确定性;（iii）可以有益地利用任何可用的边信息。该项目的核心目标是开发和研究一种新的理论框架，用于网络辅助的D2 D资源分配，该框架结合了博弈论和强化学习。我们将分布式D2 D无线网络建模为一个多智能体系统，其中一组具有有限理性的智能体共享有限的资源，根据一些决策策略采取行动。每个联合行动配置文件与每个代理的一些奖励相关联，并且代理的行动随着时间的推移而演变为过去结果和（可能）观察到的边信息的函数。在一般的多智能体系统中，智能体的行为要么是竞争性的，要么是合作性的。本研究的重点是在不确定性和缺乏先验知识的情况下研究合作用户的行为。合作要求代理人分担学习过程的成本，例如通过交换信息，因此用户的激励和真实性起主要作用。此外，由于缺乏对效用函数的先验知识，以及其他代理的行为，激励和类型，代理的策略会发生变化，作为连续信息获取和后续学习的结果。因此，为完全信息博弈开发的传统解决方案不适用于不完全信息博弈，寻找新的解决方案势在必行。简而言之，目标是（i）将传统的合作博弈理论模型和解决方案概念推广到具有不完全信息的博弈，（ii）将单代理学习模型推广到包括多个合作学习代理的学习场景，（iii）研究广义模型和开发的解决方案在解决D2 D无线网络中的资源分配和模式选择问题中的应用。

Device-to-device (D2D) communications underlaying a cellular infrastructure is one of the key technology enablers for future wireless networks. The basic idea consists in enabling suitably-selected nearby device pairs to reuse the cellular spectrum for direct data transfer, while ensuring that there is no detrimental impact on traditional cellular transmissions via base stations (BS). Despite its great potential for performance gains, D2D communications poses some fundamental challenges to system designers. These challenges, which include resource allocation and transmission mode selection, are exacerbated by the lack of timely and accurate channel state information for direct D2D links at the level of BSs and wireless devices. Therefore, in order to avoid a significant increase in the feedback and signaling overhead, there is a strong need for D2D resource allocation solutions that (i) are amenable to distributed implementation; (ii) are capable of dealing with uncertainty; (iii) can beneficially exploit any available side-information. The core objective of this project is to develop and study a novel theoretical framework for network-assisted D2D resource allocation that incorporates game theory and reinforcement learning. We model a distributed D2D wireless network as a multi-agent system, in which a set of smart agents with bounded rationality share limited resources, by taking actions according to some decision making strategy. Every joint action profile is associated with some reward for each agent, and agents' actions evolve over time as a function of past outcomes and (possibly) observed side-information. In a general multi-agent system, agents behave either competitively or cooperatively. In this project, the focus shall be on studying cooperative users' behavior under uncertainty and lack of prior knowledge. Being cooperative requires agents to share the cost of the learning process, for instance by exchanging the information, so that users' incentives and truthfulness play the major roles. Moreover, due to lack of prior knowledge on utility functions, as well as other agents' actions, incentives and types, agents' strategies are subject to change, as a consequence of continuous information acquisition and subsequent learning. Therefore, traditional solutions developed for full-information games are not applicable to games with incomplete-information, and it is imperative to look for new solution concepts. In brief, the objectives are (i) to generalize conventional cooperative game-theoretical models and solution concepts to games with incomplete information, (ii) to generalize single-agent learning models to learning scenarios that include multiple cooperative learning agents, (iii) to investigate the applications of generalized models and developed solutions in solving resource allocation and mode selection problems in D2D wireless networks.