CSVideoNet: A real-time end-to-end learning framework for high-frame-rate video compressive sensing

Kai Xu, Fengbo Ren

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a noniterative model, named 'CSVideoNet', which directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve the trade-o. between compression ratio (CR) and spatialoral resolution of the reconstructed videos. the experiment results demonstrate that CSVideoNet signi.cantly outperforms state-of-the-art approaches. Without any pre/post-processing, we achieve a 25dB Peak signal-to-noise ratio (PSNR) recovery quality at 100x CR, with a frame rate of 125 fps on a Titan X GPU. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share the source code at https://github.com/PSCLab-ASU/CSVideoNet.

Original languageEnglish (US)
Title of host publicationProceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1680-1688
Number of pages9
Volume2018-January
ISBN (Electronic)9781538648865
DOIs
StatePublished - May 3 2018
Event18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018 - Lake Tahoe, United States
Duration: Mar 12 2018Mar 15 2018

Other

Other18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018
CountryUnited States
CityLake Tahoe
Period3/12/183/15/18

Fingerprint

Decoding
Signal to noise ratio
Cameras
Recovery
Processing
Experiments
Graphics processing unit

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Cite this

Xu, K., & Ren, F. (2018). CSVideoNet: A real-time end-to-end learning framework for high-frame-rate video compressive sensing. In Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018 (Vol. 2018-January, pp. 1680-1688). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV.2018.00187

CSVideoNet : A real-time end-to-end learning framework for high-frame-rate video compressive sensing. / Xu, Kai; Ren, Fengbo.

Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. p. 1680-1688.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Xu, K & Ren, F 2018, CSVideoNet: A real-time end-to-end learning framework for high-frame-rate video compressive sensing. in Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 1680-1688, 18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018, Lake Tahoe, United States, 3/12/18. https://doi.org/10.1109/WACV.2018.00187
Xu K, Ren F. CSVideoNet: A real-time end-to-end learning framework for high-frame-rate video compressive sensing. In Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January. Institute of Electrical and Electronics Engineers Inc. 2018. p. 1680-1688 https://doi.org/10.1109/WACV.2018.00187
Xu, Kai ; Ren, Fengbo. / CSVideoNet : A real-time end-to-end learning framework for high-frame-rate video compressive sensing. Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January Institute of Electrical and Electronics Engineers Inc., 2018. pp. 1680-1688
@inproceedings{1da431acfc36436394554d759f3c2d3c,
title = "CSVideoNet: A real-time end-to-end learning framework for high-frame-rate video compressive sensing",
abstract = "This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a noniterative model, named 'CSVideoNet', which directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve the trade-o. between compression ratio (CR) and spatialoral resolution of the reconstructed videos. the experiment results demonstrate that CSVideoNet signi.cantly outperforms state-of-the-art approaches. Without any pre/post-processing, we achieve a 25dB Peak signal-to-noise ratio (PSNR) recovery quality at 100x CR, with a frame rate of 125 fps on a Titan X GPU. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share the source code at https://github.com/PSCLab-ASU/CSVideoNet.",
author = "Kai Xu and Fengbo Ren",
year = "2018",
month = "5",
day = "3",
doi = "10.1109/WACV.2018.00187",
language = "English (US)",
volume = "2018-January",
pages = "1680--1688",
booktitle = "Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - CSVideoNet

T2 - A real-time end-to-end learning framework for high-frame-rate video compressive sensing

AU - Xu, Kai

AU - Ren, Fengbo

PY - 2018/5/3

Y1 - 2018/5/3

N2 - This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a noniterative model, named 'CSVideoNet', which directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve the trade-o. between compression ratio (CR) and spatialoral resolution of the reconstructed videos. the experiment results demonstrate that CSVideoNet signi.cantly outperforms state-of-the-art approaches. Without any pre/post-processing, we achieve a 25dB Peak signal-to-noise ratio (PSNR) recovery quality at 100x CR, with a frame rate of 125 fps on a Titan X GPU. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share the source code at https://github.com/PSCLab-ASU/CSVideoNet.

AB - This paper addresses the real-time encoding-decoding problem for high-frame-rate video compressive sensing (CS). Unlike prior works that perform reconstruction using iterative optimization-based approaches, we propose a noniterative model, named 'CSVideoNet', which directly learns the inverse mapping of CS and reconstructs the original input in a single forward propagation. To overcome the limitations of existing CS cameras, we propose a multi-rate CNN and a synthesizing RNN to improve the trade-o. between compression ratio (CR) and spatialoral resolution of the reconstructed videos. the experiment results demonstrate that CSVideoNet signi.cantly outperforms state-of-the-art approaches. Without any pre/post-processing, we achieve a 25dB Peak signal-to-noise ratio (PSNR) recovery quality at 100x CR, with a frame rate of 125 fps on a Titan X GPU. Due to the feedforward and high-data-concurrency natures of CSVideoNet, it can take advantage of GPU acceleration to achieve three orders of magnitude speed-up over conventional iterative-based approaches. We share the source code at https://github.com/PSCLab-ASU/CSVideoNet.

UR - http://www.scopus.com/inward/record.url?scp=85050989762&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050989762&partnerID=8YFLogxK

U2 - 10.1109/WACV.2018.00187

DO - 10.1109/WACV.2018.00187

M3 - Conference contribution

AN - SCOPUS:85050989762

VL - 2018-January

SP - 1680

EP - 1688

BT - Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -