Learning-based Cognitive Radio Access via Randomized Point-Based Approximate POMDPs

In this paper, a novel spectrum sensing and access strategy based on approximate Partially Observable Markov Decision Processes (POMDPs) is proposed, wherein a cognitive radio learns the time-frequency correlation model defining the occupancy behavior of incumbents, via the Baum-Welch algorithm, and concurrently devises an optimal strategy to perform spectrum sensing and access that exploits this learned correlation model. To ameliorate the complexity of the POMDP optimization, the PERSEUS algorithm, a randomized point-based value iteration method, is designed, with fragmentation and Hamming distance state filters. Evaluating the cognitive radio throughput against incumbent interference, we demonstrate that, with sensing restrictions, our framework achieves a 6% performance gain over that attained by a maximum a-posteriori (MAP) state estimator with prior model knowledge, and outperforms correlation-coefficient based clustering algorithms by an average of 60%; additionally, it surpasses a Neyman-Pearson Detector that assumes independence among channels with no sensing restrictions, by an average of 25%. Furthermore, unlike state-of-the-art algorithms, the proposed design facilitates the regulation of the trade-off between cognitive radio throughput and incumbent interference via a penalty parameter in the underlying MDP.