Reconfiguring the Imaging Pipeline for Computer Vision

Mark Buckler, Suren Jayasuriya, Adrian Sampson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Scopus citations

Abstract

Advancements in deep learning have ignited an explosion of research on efficient hardware for embedded computer vision. Hardware vision acceleration, however, does not address the cost of capturing and processing the image data that feeds these algorithms. We examine the role of the image signal processing (ISP) pipeline in computer vision to identify opportunities to reduce computation and save energy. The key insight is that imaging pipelines should be be configurable: to switch between a traditional photography mode and a low-power vision mode that produces lower-quality image data suitable only for computer vision. We use eight computer vision algorithms and a reversible pipeline simulation tool to study the imaging system's impact on vision performance. For both CNN-based and classical vision algorithms, we observe that only two ISP stages, demosaicing and gamma compression, are critical for task performance. We propose a new image sensor design that can compensate for these stages. The sensor design features an adjustable resolution and tunable analog-to-digital converters (ADCs). Our proposed imaging system's vision mode disables the ISP entirely and configures the sensor to produce subsampled, lower-precision image data. This vision mode can save ~75% of the average energy of a baseline photography mode with only a small impact on vision task accuracy.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages975-984
Number of pages10
Volume2017-October
ISBN (Electronic)9781538610329
DOIs
StatePublished - Dec 22 2017
Externally publishedYes
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: Oct 22 2017Oct 29 2017

Other

Other16th IEEE International Conference on Computer Vision, ICCV 2017
CountryItaly
CityVenice
Period10/22/1710/29/17

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Reconfiguring the Imaging Pipeline for Computer Vision'. Together they form a unique fingerprint.

  • Cite this

    Buckler, M., Jayasuriya, S., & Sampson, A. (2017). Reconfiguring the Imaging Pipeline for Computer Vision. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017 (Vol. 2017-October, pp. 975-984). [8237373] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2017.111