DRL-Aided Joint Resource Block and Beamforming Management for Cellular-Connected UAVs

Yuanjian Li, Mathini Sellathurai, Zheng Chu, Pei Xiao, A. Hamid Aghvami

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review


In this paper, we investigate a cellular-connected unmanned aerial vehicle (UAV) network, where multiple UAVs receive messages from base stations (BSs) in the down-link, and in the meantime, BSs serve their paired ground user equipments (UEs). To effectively manage inter-cell interferences (ICIs) among UEs due to intense reuse of time-frequency resource block (RB) resource, a first p-tier based RB coordination criterion is adopted. Then, to enhance wireless transmission quality for UAVs while protecting terrestrial UEs from being interfered by ground-to-air (G2A) transmissions, a radio resource management (RRM) problem of joint dynamic RB coordination and time-varying beamforming design is formulated to minimize UAV's ergodic outage duration (EOD). To cope with conventional optimization techniques' inefficiency in solving the formulated RRM problem, a deep reinforcement learning (DRL)-aided solution is proposed, where deep double duelling Q network (D3QN) and twin delayed deep deterministic policy gradient (TD3) are invoked to deal with RB coordination in the discrete action domain and beamforming design in the continuous action regime, respectively. Numerical results illustrate the effectiveness of the proposed hybrid D3QNTD3 algorithm, compared to representative baselines.

Original languageEnglish
Title of host publicationGLOBECOM 2023 - 2023 IEEE Global Communications Conference
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9798350310900
Publication statusPublished - 2023
Externally publishedYes
Event2023 IEEE Global Communications Conference, GLOBECOM 2023 - Kuala Lumpur, Malaysia
Duration: 4 Dec 20238 Dec 2023

Publication series

NameProceedings - IEEE Global Communications Conference, GLOBECOM
ISSN (Print)2334-0983
ISSN (Electronic)2576-6813


Conference2023 IEEE Global Communications Conference, GLOBECOM 2023
CityKuala Lumpur

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing


Dive into the research topics of 'DRL-Aided Joint Resource Block and Beamforming Management for Cellular-Connected UAVs'. Together they form a unique fingerprint.

Cite this