Deep reinforcement learning agents for dynamic spectrum access in television whitespace cognitive radio networks

Abstract

Businesses, security agencies, institutions, and individuals depend on wireless communication to run their day-to-day activities successfully. The ever-increasing demand for wireless communication services, coupled with the scarcity of available radio frequency spectrum, necessitates innovative approaches to spectrum management. Cognitive Radio (CR) technology has emerged as a pivotal solution, enabling dynamic spectrum sharing among secondary users while respecting the rights of primary users. However, the basic setup of CR technology is insufficient to manage spectrum congestion, as it lacks the ability to predict future spectrum holes, leading to interferences. With predictive intelligence and Dynamic Spectrum Access (DSA), a CR can anticipate when and where other users will be using the radio frequency spectrum, allowing it to overcome this limitation. Reinforcement Learning (RL) in CRs helps predict spectral changes and identify optimal transmission frequencies. This work presents the development of Deep RL (DRL) models for enhanced DSA in TV Whitespace (TVWS) cognitive radio networks using Deep Q-Networks (DQN) and Quantile-Regression (QR-DQN) algorithms. The implementation was done in the Radio Frequency Reinforcement Learning (RFRL) Gym, a training environment of the RF spectrum designed to provide comprehensive functionality. Evaluations show that the DQN model achieves a 96.34 % interference avoidance rate compared to 95.97 % of QRDQN. Average latency was estimated at 1 millisecond and 3.33 milliseconds per packet, respectively. Therefore DRL proves to be a more flexible, scalable, and adaptive approach to dynamic spectrum access, making it particularly effective in the complex and constantly evolving wireless spectrum environment.