DeepMARA - Deep Reinforcement Learning based Massive Random Access Toward Massive Machine-to-Machine Communications

在过去的几十年里，通信技术取得了巨大的成功-今天，我们可以在任何时候从世界上几乎任何地方连接近70亿人，我们可以在旅途中播放YouTube视频，或者在我们的移动设备上进行视频会议。这些成就是不久前科幻文学的一部分。今天，我们需要设计能够实现我们当前梦想的通信技术：我们能否在今天为我们服务的同一网络基础设施上连接超过300亿台智能设备？我们是否可以将所有这些设备连接起来，使世界上任何地方的任何时候都能为所有人提供可靠的医疗服务，或者调节交通流量，协调自动驾驶汽车？我们能不能不仅流预录的视频，而且在移动的设备上提供无缝的AR/VR体验？这些只是大规模机器对机器（M2M）通信（大规模物联网（IoT）的主要推动者之一）的几个应用。为这些未来技术奠定理论和算法基础是该项目的核心目标。M2M设备通常只是偶尔活动并以低数据速率传输。由于不可能协调这些设备的传输，因此需要基于随机访问的解决方案来实现它们的连接。在这些截然不同的要求下，必须设计用于未来M2M通信系统的新型大规模随机接入（MRA）解决方案。机器学习（ML）方法的能力，如深度强化学习（DRL），可以协调多个代理以不协调的方式实现共同目标，使它们成为实现这一目标的正确工具。具体而言，该项目的目的是设计智能传输策略，将基于DRL的解决方案的冲突避免能力与一些MRA算法的冲突解决能力相结合，使未来的大规模M2M通信系统成为现实。

Communication technologies have achieved remarkable success over the last decades - today we can connect almost 7 billionpeople at any time from almost anywhere in the world, we can stream YouTube videos on-the-go, or have video conferences on ourmobile devices. These achievements were part of science fiction literature not long time ago. Today, we need to design thecommunication technologies that can realise our current dreams: Can we connect over 30 billion intelligent devices over the samenetwork infrastructure that serves us today? Can we connect all these devices to enable reliable healthcare services to all the peopleat any time anywhere in the world, or to regulate traffic flow and to coordinate autonomous cars? Can we, not only stream prerecordedvideos, but provide seamless AR/VR experience on mobile devices? These are only a few applications of massive machineto-machine (M2M) communications (one of the main enablers of massive Internet of things (IoT)). Laying the theoretical andalgorithmic foundations of these future technologies is the core ambition of this project.M2M devices are typically only sporadically active and transmit at low data rates. Since it is impossible to coordinate the transmissionof such devices, random access-based solutions are needed to enable their connectivity. With these drastically different requirements,it is imperative to design novel massive random access (MRA) solutions for use in future M2M communication systems. The ability ofmachine learning (ML) approaches such as deep reinforcement learning (DRL) in orchestrating multiple agents to achieve a commongoal in an uncoordinated manner, makes them the right tools to achieve this goal. Specifically, the aim of this project is to devisesmart transmission strategies that combine the collision avoidance capability of DRL-based solutions with collision resolutioncapability of some MRA algorithms to make future massive M2M communication systems realisable.