Learning-based Spectrum Sensing and Access in Cognitive Radios via Approximate POMDPs

Bharath Keshavamurthy, Nicolo Michelusi

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


A novel LEarning-based Spectrum Sensing and Access (LESSA) framework is proposed, wherein a cognitive radio (CR) learns a time-frequency correlation model underlying spectrum occupancy of licensed users (LUs) in a radio ecosystem; concurrently, it devises an approximately optimal spectrum sensing and access policy under sensing constraints. A Baum-Welch algorithm is proposed to learn a parametric Markov transition model of LUs’ spectrum occupancy based on noisy spectrum measurements. Spectrum sensing and access are cast as a Partially-Observable Markov Decision Process, approximately optimized via randomized point-based value iteration. Fragmentation, Hamming-distance state filters and Monte-Carlo methods are proposed to alleviate the inherent computational complexity, and a weighted reward metric to regulate the trade-off between CR’s throughput and interference to the LUs. Numerical evaluations demonstrate that LESSA performs within 5% of a genie-aided upper bound with foreknowledge of LUs’ spectrum occupancy, and outperforms state-of-the-art algorithms across the entire trade-off region: 71% over correlation-based clustering, 26% over Neyman-Pearson-based spectrum sensing, 6% over the Viterbi algorithm, and 9% over adaptive Deep Q-Network. LESSA is then extended to a distributed Multi-Agent setting (MA-LESSA), by proposing novel neighbor discovery and channel access rank allocation. MA-LESSA improves CRs’ throughputs by 43% over cooperative TD-SARSA, 84% over cooperative greedy distributed learning, and 3× over non-cooperative learning via g-statistics and ACKs. Finally, MA-LESSA is implemented on the DARPA SC2 platform, manifesting superior performance over competitors in a real-world TDWR-UNII WLAN emulation; its implementation feasibility is further validated on an ad-hoc distributed wireless testbed of ESP32 radios, exhibiting 96% success probability.

Original languageEnglish (US)
JournalIEEE Transactions on Cognitive Communications and Networking
StateAccepted/In press - 2021


  • Adaptation models
  • Cognitive Radio
  • Correlation
  • Hidden Markov Model
  • Hidden Markov models
  • Noise measurement
  • POMDP.
  • Resource management
  • Sensors
  • Spectrum Sensing
  • Time-frequency analysis

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence


Dive into the research topics of 'Learning-based Spectrum Sensing and Access in Cognitive Radios via Approximate POMDPs'. Together they form a unique fingerprint.

Cite this