Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding

Wei Jung Chien, Glen P. Abousleman, Lina Karam

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a software-only, real-time video coder/decoder (codec) with super-resolution-based enhancement for ultra-low-bit-rate compression. The codec incorporates a modified JPEG2000 core and interframe predictive coding, and can operate with network bandwidths as low as 500 bits/second. Highly compressed video exhibits severe coding artifacts that degrade visual quality. To lower the level of noise and retain the sharpness of the video frames, we build on our previous work in super-resolution-based video enhancement and propose a new version that is suitable for real-time video coding systems. The adopted super-resolution-based enhancement uses a constrained set of motion vectors that is computed from the original (uncompressed) video at the encoder. Artificial motion is also added to the difference frame to maximize the enhancement performance. The encoder can transmit either the full set of motion vectors or the constrained set of motion vectors depending upon the available bandwidth. At the decoder, each pixel of the decoded frame is assigned to a motion vector from the constrained motion vector set. L 2-norm minimization super-resolution is then applied to the decoded frame set (previous frame, current frame, and next frame). A selective motion estimation scheme is proposed to prevent ghosting, which otherwise would result from the superresolution enhancement when the motion estimation fails to find appropriate motion vectors. Results using the proposed system demonstrate significant improvements in the quantitative and visual quality of the coded video sequences.

Original languageEnglish (US)
Title of host publicationProceedings of SPIE - The International Society for Optical Engineering
Volume6579
DOIs
StatePublished - 2007
EventMobile Multimedia/Image Processing for Military and Security Applications 2007 - Orlando, FL, United States
Duration: Apr 11 2007Apr 12 2007

Other

OtherMobile Multimedia/Image Processing for Military and Security Applications 2007
CountryUnited States
CityOrlando, FL
Period4/11/074/12/07

Fingerprint

Image coding
coding
augmentation
coders
decoders
Motion estimation
Bandwidth
bandwidth
video compression
Pixels
sharpness
norms
artifacts
pixels
computer programs
optimization

Keywords

  • Motion estimation
  • Super-resolution
  • Ultra-low-bit-rate video coding
  • Video enhancement

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Condensed Matter Physics

Cite this

Chien, W. J., Abousleman, G. P., & Karam, L. (2007). Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding. In Proceedings of SPIE - The International Society for Optical Engineering (Vol. 6579). [657904] https://doi.org/10.1117/12.720065

Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding. / Chien, Wei Jung; Abousleman, Glen P.; Karam, Lina.

Proceedings of SPIE - The International Society for Optical Engineering. Vol. 6579 2007. 657904.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chien, WJ, Abousleman, GP & Karam, L 2007, Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding. in Proceedings of SPIE - The International Society for Optical Engineering. vol. 6579, 657904, Mobile Multimedia/Image Processing for Military and Security Applications 2007, Orlando, FL, United States, 4/11/07. https://doi.org/10.1117/12.720065
Chien WJ, Abousleman GP, Karam L. Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding. In Proceedings of SPIE - The International Society for Optical Engineering. Vol. 6579. 2007. 657904 https://doi.org/10.1117/12.720065
Chien, Wei Jung ; Abousleman, Glen P. ; Karam, Lina. / Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding. Proceedings of SPIE - The International Society for Optical Engineering. Vol. 6579 2007.
@inproceedings{81b13ec5375d4b14a296b4a0a96665fa,
title = "Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding",
abstract = "This paper presents a software-only, real-time video coder/decoder (codec) with super-resolution-based enhancement for ultra-low-bit-rate compression. The codec incorporates a modified JPEG2000 core and interframe predictive coding, and can operate with network bandwidths as low as 500 bits/second. Highly compressed video exhibits severe coding artifacts that degrade visual quality. To lower the level of noise and retain the sharpness of the video frames, we build on our previous work in super-resolution-based video enhancement and propose a new version that is suitable for real-time video coding systems. The adopted super-resolution-based enhancement uses a constrained set of motion vectors that is computed from the original (uncompressed) video at the encoder. Artificial motion is also added to the difference frame to maximize the enhancement performance. The encoder can transmit either the full set of motion vectors or the constrained set of motion vectors depending upon the available bandwidth. At the decoder, each pixel of the decoded frame is assigned to a motion vector from the constrained motion vector set. L 2-norm minimization super-resolution is then applied to the decoded frame set (previous frame, current frame, and next frame). A selective motion estimation scheme is proposed to prevent ghosting, which otherwise would result from the superresolution enhancement when the motion estimation fails to find appropriate motion vectors. Results using the proposed system demonstrate significant improvements in the quantitative and visual quality of the coded video sequences.",
keywords = "Motion estimation, Super-resolution, Ultra-low-bit-rate video coding, Video enhancement",
author = "Chien, {Wei Jung} and Abousleman, {Glen P.} and Lina Karam",
year = "2007",
doi = "10.1117/12.720065",
language = "English (US)",
isbn = "0819467014",
volume = "6579",
booktitle = "Proceedings of SPIE - The International Society for Optical Engineering",

}

TY - GEN

T1 - Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding

AU - Chien, Wei Jung

AU - Abousleman, Glen P.

AU - Karam, Lina

PY - 2007

Y1 - 2007

N2 - This paper presents a software-only, real-time video coder/decoder (codec) with super-resolution-based enhancement for ultra-low-bit-rate compression. The codec incorporates a modified JPEG2000 core and interframe predictive coding, and can operate with network bandwidths as low as 500 bits/second. Highly compressed video exhibits severe coding artifacts that degrade visual quality. To lower the level of noise and retain the sharpness of the video frames, we build on our previous work in super-resolution-based video enhancement and propose a new version that is suitable for real-time video coding systems. The adopted super-resolution-based enhancement uses a constrained set of motion vectors that is computed from the original (uncompressed) video at the encoder. Artificial motion is also added to the difference frame to maximize the enhancement performance. The encoder can transmit either the full set of motion vectors or the constrained set of motion vectors depending upon the available bandwidth. At the decoder, each pixel of the decoded frame is assigned to a motion vector from the constrained motion vector set. L 2-norm minimization super-resolution is then applied to the decoded frame set (previous frame, current frame, and next frame). A selective motion estimation scheme is proposed to prevent ghosting, which otherwise would result from the superresolution enhancement when the motion estimation fails to find appropriate motion vectors. Results using the proposed system demonstrate significant improvements in the quantitative and visual quality of the coded video sequences.

AB - This paper presents a software-only, real-time video coder/decoder (codec) with super-resolution-based enhancement for ultra-low-bit-rate compression. The codec incorporates a modified JPEG2000 core and interframe predictive coding, and can operate with network bandwidths as low as 500 bits/second. Highly compressed video exhibits severe coding artifacts that degrade visual quality. To lower the level of noise and retain the sharpness of the video frames, we build on our previous work in super-resolution-based video enhancement and propose a new version that is suitable for real-time video coding systems. The adopted super-resolution-based enhancement uses a constrained set of motion vectors that is computed from the original (uncompressed) video at the encoder. Artificial motion is also added to the difference frame to maximize the enhancement performance. The encoder can transmit either the full set of motion vectors or the constrained set of motion vectors depending upon the available bandwidth. At the decoder, each pixel of the decoded frame is assigned to a motion vector from the constrained motion vector set. L 2-norm minimization super-resolution is then applied to the decoded frame set (previous frame, current frame, and next frame). A selective motion estimation scheme is proposed to prevent ghosting, which otherwise would result from the superresolution enhancement when the motion estimation fails to find appropriate motion vectors. Results using the proposed system demonstrate significant improvements in the quantitative and visual quality of the coded video sequences.

KW - Motion estimation

KW - Super-resolution

KW - Ultra-low-bit-rate video coding

KW - Video enhancement

UR - http://www.scopus.com/inward/record.url?scp=36049038906&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=36049038906&partnerID=8YFLogxK

U2 - 10.1117/12.720065

DO - 10.1117/12.720065

M3 - Conference contribution

SN - 0819467014

SN - 9780819467010

VL - 6579

BT - Proceedings of SPIE - The International Society for Optical Engineering

ER -