Real-time scheduling over Markovian channels: When partial observability meets hard deadlines

Lei Yang, Sugumar Murugesan, Junshan Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In this study, downlink scheduling of multiuser traffic with hard deadlines and packet-level priorities is cast as a partially observable Markov decision process. User channels are modeled as Markovian and the base station can learn only the channel condition of the currently scheduled user. The optimization of joint channel learning and scheduling presents the combined challenges incurred by the strict deadline constraint of real-time traffic and the partial observability of multiuser channels. In particular, we show that idling adds a new dimension to the action space; and that, through a case study of heterogeneous multiuser networks, idling is indeed the optimal action under certain system states. This somewhat surprising result reveals the existence of the fundamental tradeoffs between exploitation and exploration/idling, going beyond the classic 'exploitation vs exploration'. We find that, due to hard deadlines and packet priorities, idling is intimately related to the tradeoff between the successful transmission of backlogged packets and that of future arrivals. In contrast, for the special case with a symmetric two-user system, we show that the scheduling problem exhibits unique structures, rendering a non-idling greedy policy optimal.

Original languageEnglish (US)
Title of host publicationGLOBECOM - IEEE Global Telecommunications Conference
DOIs
StatePublished - 2011
Event54th Annual IEEE Global Telecommunications Conference: "Energizing Global Communications", GLOBECOM 2011 - Houston, TX, United States
Duration: Dec 5 2011Dec 9 2011

Other

Other54th Annual IEEE Global Telecommunications Conference: "Energizing Global Communications", GLOBECOM 2011
CountryUnited States
CityHouston, TX
Period12/5/1112/9/11

Fingerprint

Observability
Scheduling
Base stations

ASJC Scopus subject areas

  • Electrical and Electronic Engineering

Cite this

Yang, L., Murugesan, S., & Zhang, J. (2011). Real-time scheduling over Markovian channels: When partial observability meets hard deadlines. In GLOBECOM - IEEE Global Telecommunications Conference [6133855] https://doi.org/10.1109/GLOCOM.2011.6133855

Real-time scheduling over Markovian channels : When partial observability meets hard deadlines. / Yang, Lei; Murugesan, Sugumar; Zhang, Junshan.

GLOBECOM - IEEE Global Telecommunications Conference. 2011. 6133855.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, L, Murugesan, S & Zhang, J 2011, Real-time scheduling over Markovian channels: When partial observability meets hard deadlines. in GLOBECOM - IEEE Global Telecommunications Conference., 6133855, 54th Annual IEEE Global Telecommunications Conference: "Energizing Global Communications", GLOBECOM 2011, Houston, TX, United States, 12/5/11. https://doi.org/10.1109/GLOCOM.2011.6133855
Yang, Lei ; Murugesan, Sugumar ; Zhang, Junshan. / Real-time scheduling over Markovian channels : When partial observability meets hard deadlines. GLOBECOM - IEEE Global Telecommunications Conference. 2011.
@inproceedings{cc3622c029eb4ddbb2ba6ddfdd0b522c,
title = "Real-time scheduling over Markovian channels: When partial observability meets hard deadlines",
abstract = "In this study, downlink scheduling of multiuser traffic with hard deadlines and packet-level priorities is cast as a partially observable Markov decision process. User channels are modeled as Markovian and the base station can learn only the channel condition of the currently scheduled user. The optimization of joint channel learning and scheduling presents the combined challenges incurred by the strict deadline constraint of real-time traffic and the partial observability of multiuser channels. In particular, we show that idling adds a new dimension to the action space; and that, through a case study of heterogeneous multiuser networks, idling is indeed the optimal action under certain system states. This somewhat surprising result reveals the existence of the fundamental tradeoffs between exploitation and exploration/idling, going beyond the classic 'exploitation vs exploration'. We find that, due to hard deadlines and packet priorities, idling is intimately related to the tradeoff between the successful transmission of backlogged packets and that of future arrivals. In contrast, for the special case with a symmetric two-user system, we show that the scheduling problem exhibits unique structures, rendering a non-idling greedy policy optimal.",
author = "Lei Yang and Sugumar Murugesan and Junshan Zhang",
year = "2011",
doi = "10.1109/GLOCOM.2011.6133855",
language = "English (US)",
isbn = "9781424492688",
booktitle = "GLOBECOM - IEEE Global Telecommunications Conference",

}

TY - GEN

T1 - Real-time scheduling over Markovian channels

T2 - When partial observability meets hard deadlines

AU - Yang, Lei

AU - Murugesan, Sugumar

AU - Zhang, Junshan

PY - 2011

Y1 - 2011

N2 - In this study, downlink scheduling of multiuser traffic with hard deadlines and packet-level priorities is cast as a partially observable Markov decision process. User channels are modeled as Markovian and the base station can learn only the channel condition of the currently scheduled user. The optimization of joint channel learning and scheduling presents the combined challenges incurred by the strict deadline constraint of real-time traffic and the partial observability of multiuser channels. In particular, we show that idling adds a new dimension to the action space; and that, through a case study of heterogeneous multiuser networks, idling is indeed the optimal action under certain system states. This somewhat surprising result reveals the existence of the fundamental tradeoffs between exploitation and exploration/idling, going beyond the classic 'exploitation vs exploration'. We find that, due to hard deadlines and packet priorities, idling is intimately related to the tradeoff between the successful transmission of backlogged packets and that of future arrivals. In contrast, for the special case with a symmetric two-user system, we show that the scheduling problem exhibits unique structures, rendering a non-idling greedy policy optimal.

AB - In this study, downlink scheduling of multiuser traffic with hard deadlines and packet-level priorities is cast as a partially observable Markov decision process. User channels are modeled as Markovian and the base station can learn only the channel condition of the currently scheduled user. The optimization of joint channel learning and scheduling presents the combined challenges incurred by the strict deadline constraint of real-time traffic and the partial observability of multiuser channels. In particular, we show that idling adds a new dimension to the action space; and that, through a case study of heterogeneous multiuser networks, idling is indeed the optimal action under certain system states. This somewhat surprising result reveals the existence of the fundamental tradeoffs between exploitation and exploration/idling, going beyond the classic 'exploitation vs exploration'. We find that, due to hard deadlines and packet priorities, idling is intimately related to the tradeoff between the successful transmission of backlogged packets and that of future arrivals. In contrast, for the special case with a symmetric two-user system, we show that the scheduling problem exhibits unique structures, rendering a non-idling greedy policy optimal.

UR - http://www.scopus.com/inward/record.url?scp=84863121481&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863121481&partnerID=8YFLogxK

U2 - 10.1109/GLOCOM.2011.6133855

DO - 10.1109/GLOCOM.2011.6133855

M3 - Conference contribution

AN - SCOPUS:84863121481

SN - 9781424492688

BT - GLOBECOM - IEEE Global Telecommunications Conference

ER -