TY - JOUR
T1 - Learning-Based Spectrum Sensing and Access in Cognitive Radios via Approximate POMDPs
AU - Keshavamurthy, Bharath
AU - Michelusi, Nicolo
N1 - Funding Information:
Funded in part by NSF under grants CNS-1642982 and CNS-2129015.
Publisher Copyright:
© 2015 IEEE.
PY - 2022/6/1
Y1 - 2022/6/1
N2 - A novel LEarning-based Spectrum Sensing and Access (LESSA) framework is proposed, wherein a cognitive radio (CR) learns a time-frequency correlation model underlying spectrum occupancy of licensed users (LUs) in a radio ecosystem; concurrently, it devises an approximately optimal spectrum sensing and access policy under sensing constraints. A Baum-Welch algorithm is proposed to learn a parametric Markov transition model of LUs' spectrum occupancy based on noisy spectrum measurements. Spectrum sensing and access are cast as a Partially-Observable Markov Decision Process, approximately optimized via randomized point-based value iteration. Fragmentation, Hamming-distance state filters and Monte-Carlo methods are proposed to alleviate the inherent computational complexity, and a weighted reward metric to regulate the trade-off between CR's throughput and interference to the LUs. Numerical evaluations demonstrate that LESSA performs within 5% of a genie-aided upper bound with foreknowledge of LUs' spectrum occupancy, and outperforms state-of-the-art algorithms across the entire trade-off region: 71% over correlation-based clustering, 26% over Neyman-Pearson-based spectrum sensing, 6% over the Viterbi algorithm, and 9% over adaptive Deep Q-Network. LESSA is then extended to a distributed Multi-Agent setting (MA-LESSA), by proposing novel neighbor discovery and channel access rank allocation. MA-LESSA improves CRs' throughputs by 43% over cooperative TD-SARSA, 84% over cooperative greedy distributed learning, and 3× over non-cooperative learning via g-statistics and ACKs. Finally, MA-LESSA is implemented on the DARPA SC2 platform, manifesting superior performance over competitors in a real-world TDWR-UNII WLAN emulation; its implementation feasibility is further validated on an ad-hoc distributed wireless testbed of ESP32 radios, exhibiting 96% success probability.
AB - A novel LEarning-based Spectrum Sensing and Access (LESSA) framework is proposed, wherein a cognitive radio (CR) learns a time-frequency correlation model underlying spectrum occupancy of licensed users (LUs) in a radio ecosystem; concurrently, it devises an approximately optimal spectrum sensing and access policy under sensing constraints. A Baum-Welch algorithm is proposed to learn a parametric Markov transition model of LUs' spectrum occupancy based on noisy spectrum measurements. Spectrum sensing and access are cast as a Partially-Observable Markov Decision Process, approximately optimized via randomized point-based value iteration. Fragmentation, Hamming-distance state filters and Monte-Carlo methods are proposed to alleviate the inherent computational complexity, and a weighted reward metric to regulate the trade-off between CR's throughput and interference to the LUs. Numerical evaluations demonstrate that LESSA performs within 5% of a genie-aided upper bound with foreknowledge of LUs' spectrum occupancy, and outperforms state-of-the-art algorithms across the entire trade-off region: 71% over correlation-based clustering, 26% over Neyman-Pearson-based spectrum sensing, 6% over the Viterbi algorithm, and 9% over adaptive Deep Q-Network. LESSA is then extended to a distributed Multi-Agent setting (MA-LESSA), by proposing novel neighbor discovery and channel access rank allocation. MA-LESSA improves CRs' throughputs by 43% over cooperative TD-SARSA, 84% over cooperative greedy distributed learning, and 3× over non-cooperative learning via g-statistics and ACKs. Finally, MA-LESSA is implemented on the DARPA SC2 platform, manifesting superior performance over competitors in a real-world TDWR-UNII WLAN emulation; its implementation feasibility is further validated on an ad-hoc distributed wireless testbed of ESP32 radios, exhibiting 96% success probability.
KW - Cognitive radio
KW - Hidden Markov Model
KW - POMDP
KW - Spectrum sensing
UR - http://www.scopus.com/inward/record.url?scp=85115692492&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85115692492&partnerID=8YFLogxK
U2 - 10.1109/TCCN.2021.3129802
DO - 10.1109/TCCN.2021.3129802
M3 - Article
AN - SCOPUS:85115692492
VL - 8
SP - 514
EP - 528
JO - IEEE Transactions on Cognitive Communications and Networking
JF - IEEE Transactions on Cognitive Communications and Networking
SN - 2332-7731
IS - 2
ER -