Fundamental limits of universal variable-to-fixed length coding of parametric sources

Nematollah Iri, Oliver Kosut

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Universal variable-to-fixed (V-F) length coding of d-dimensional exponential family of distributions is considered. An achievable scheme is proposed, which consists of a dictionary to parse the source output stream. The previously-introduced notion of quantized type is employed for the dictionary construction. The quantized type class of a sequence is based on partitioning the space of minimal sufficient statistics into cuboids. The proposed dictionary consists of sequences in the boundaries of transition from low to high quantized type class size. Asymptotics of the -coding rate of the proposed coding scheme for large enough dictionaries is derived. In particular, we show that the third-order coding rate of the proposed scheme is H d/2 log log M/log M, where H is the entropy of the source and M is the dictionary size. We further provide a converse, showing that this rate is optimal up to the third-order term.

Original languageEnglish (US)
Title of host publication55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages31-37
Number of pages7
Volume2018-January
ISBN (Electronic)9781538632666
DOIs
StatePublished - Jan 17 2018
Event55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017 - Monticello, United States
Duration: Oct 3 2017Oct 6 2017

Other

Other55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017
CountryUnited States
CityMonticello
Period10/3/1710/6/17

Fingerprint

Glossaries
Coding
Cuboid
Sufficient Statistics
Exponential Family
Converse
Partitioning
Entropy
Dictionary
Statistics
Output
Term
Class

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing
  • Energy Engineering and Power Technology
  • Control and Optimization

Cite this

Iri, N., & Kosut, O. (2018). Fundamental limits of universal variable-to-fixed length coding of parametric sources. In 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017 (Vol. 2018-January, pp. 31-37). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ALLERTON.2017.8262715

Fundamental limits of universal variable-to-fixed length coding of parametric sources. / Iri, Nematollah; Kosut, Oliver.

55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 31-37.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Iri, N & Kosut, O 2018, Fundamental limits of universal variable-to-fixed length coding of parametric sources. in 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 31-37, 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017, Monticello, United States, 10/3/17. https://doi.org/10.1109/ALLERTON.2017.8262715
Iri N, Kosut O. Fundamental limits of universal variable-to-fixed length coding of parametric sources. In 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 31-37 https://doi.org/10.1109/ALLERTON.2017.8262715
Iri, Nematollah ; Kosut, Oliver. / Fundamental limits of universal variable-to-fixed length coding of parametric sources. 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 31-37
@inproceedings{2407a4196d9d404ca4b9029427cd25fc,
title = "Fundamental limits of universal variable-to-fixed length coding of parametric sources",
abstract = "Universal variable-to-fixed (V-F) length coding of d-dimensional exponential family of distributions is considered. An achievable scheme is proposed, which consists of a dictionary to parse the source output stream. The previously-introduced notion of quantized type is employed for the dictionary construction. The quantized type class of a sequence is based on partitioning the space of minimal sufficient statistics into cuboids. The proposed dictionary consists of sequences in the boundaries of transition from low to high quantized type class size. Asymptotics of the -coding rate of the proposed coding scheme for large enough dictionaries is derived. In particular, we show that the third-order coding rate of the proposed scheme is H d/2 log log M/log M, where H is the entropy of the source and M is the dictionary size. We further provide a converse, showing that this rate is optimal up to the third-order term.",
author = "Nematollah Iri and Oliver Kosut",
year = "2018",
month = "1",
day = "17",
doi = "10.1109/ALLERTON.2017.8262715",
language = "English (US)",
volume = "2018-January",
pages = "31--37",
booktitle = "55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Fundamental limits of universal variable-to-fixed length coding of parametric sources

AU - Iri, Nematollah

AU - Kosut, Oliver

PY - 2018/1/17

Y1 - 2018/1/17

N2 - Universal variable-to-fixed (V-F) length coding of d-dimensional exponential family of distributions is considered. An achievable scheme is proposed, which consists of a dictionary to parse the source output stream. The previously-introduced notion of quantized type is employed for the dictionary construction. The quantized type class of a sequence is based on partitioning the space of minimal sufficient statistics into cuboids. The proposed dictionary consists of sequences in the boundaries of transition from low to high quantized type class size. Asymptotics of the -coding rate of the proposed coding scheme for large enough dictionaries is derived. In particular, we show that the third-order coding rate of the proposed scheme is H d/2 log log M/log M, where H is the entropy of the source and M is the dictionary size. We further provide a converse, showing that this rate is optimal up to the third-order term.

AB - Universal variable-to-fixed (V-F) length coding of d-dimensional exponential family of distributions is considered. An achievable scheme is proposed, which consists of a dictionary to parse the source output stream. The previously-introduced notion of quantized type is employed for the dictionary construction. The quantized type class of a sequence is based on partitioning the space of minimal sufficient statistics into cuboids. The proposed dictionary consists of sequences in the boundaries of transition from low to high quantized type class size. Asymptotics of the -coding rate of the proposed coding scheme for large enough dictionaries is derived. In particular, we show that the third-order coding rate of the proposed scheme is H d/2 log log M/log M, where H is the entropy of the source and M is the dictionary size. We further provide a converse, showing that this rate is optimal up to the third-order term.

UR - http://www.scopus.com/inward/record.url?scp=85047966221&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047966221&partnerID=8YFLogxK

U2 - 10.1109/ALLERTON.2017.8262715

DO - 10.1109/ALLERTON.2017.8262715

M3 - Conference contribution

AN - SCOPUS:85047966221

VL - 2018-January

SP - 31

EP - 37

BT - 55th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -