Optimal Tethered-UAV Deployment in A2G Communication Networks: Multi-Agent Q-Learning Approach

Abstract

An unmanned aerial vehicle-mounted base station (UAV-BS) is a promising technology for the forthcoming sixth-generation wireless networks, owing to its flexibility and cost effectiveness. Besides, the limited network operation time of UAV-BS networks can be overcome with the concept of tethered unmanned aerial vehicles (TUAVs), which are powered from an energy source in the ground. Along with this trend, the optimal deployment (i.e., trajectory control) of TUAVs to maximize throughput in multicell environments has been studied. As the problem is modeled by a Markov decision process, a multiagent Q-learning (QL) algorithm was developed to obtain a solution. When considering the limited inter-UAV link capacity and computing power of each UAV, the proposed multiagent QL algorithm can be a practical approach. Intensive simulations were conducted to evaluate the performance of the proposed algorithm with respect to various metrics, such as sum or individual rates, fairness, and computational complexity in multicell air-to-ground (A2G) networks. Our proposed algorithm achieves superior performance compared to conventional algorithms, such as random action, QL-based altitude control algorithms (QAC), and centralized QL algorithm.