Characterizing the reconfiguration latency of image sensor resolution on android devices

Jinhan Hu, Jianan Yang, Vraj Delhivala, Robert LiKamWa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Advances in vision processing have ignited a proliferation of mobile vision applications, including augmented reality. However, limited by the inability to rapidly reconfigure sensor operation for performance-efficiency tradeoffs, high power consumption causes vision applications to drain the device’s battery. To explore the potential impact of enabling rapid reconfiguration, we use a case study around marker-based pose estimation to understand the relationship between image frame resolution, task accuracy, and energy efficiency. Our case study motivates that to balance energy efficiency and task accuracy, the application needs to dynamically and frequently reconfigure sensor resolution. To explore the latency bottlenecks to sensor resolution reconfiguration, we define and profile the end-to-end reconfiguration latency and frame-to-frame latency of changing capture resolution on a Google LG Nexus 5X device. We identify three major sources of sensor resolution reconfiguration latency in current Android systems: (i) sequential configuration patterns, (ii) expensive system calls, and (iii) imaging pipeline delay. Based on our intuitions, we propose a redesign of the Android camera system to mitigate the sources of latency. Enabling smooth transitions between sensor configurations will unlock new classes of adaptive-resolution vision applications.

Original languageEnglish (US)
Title of host publicationHotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications
PublisherAssociation for Computing Machinery, Inc
Pages81-86
Number of pages6
Volume2018-February
ISBN (Electronic)9781450356305
DOIs
StatePublished - Feb 12 2018
Event19th International Workshop on Mobile Computing Systems and Applications, HotMobile 2018 - Tempe, United States
Duration: Feb 12 2018Feb 13 2018

Other

Other19th International Workshop on Mobile Computing Systems and Applications, HotMobile 2018
CountryUnited States
CityTempe
Period2/12/182/13/18

Fingerprint

Image sensors
Sensors
Energy efficiency
Augmented reality
Electric power utilization
Pipelines
Cameras
Imaging techniques
Processing

Keywords

  • Camera system
  • Image sensor
  • Mobile devices
  • Operating system optimization
  • Reconfiguration

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Computer Science Applications
  • Software
  • Computer Networks and Communications

Cite this

Hu, J., Yang, J., Delhivala, V., & LiKamWa, R. (2018). Characterizing the reconfiguration latency of image sensor resolution on android devices. In HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications (Vol. 2018-February, pp. 81-86). Association for Computing Machinery, Inc. https://doi.org/10.1145/3177102.3177109

Characterizing the reconfiguration latency of image sensor resolution on android devices. / Hu, Jinhan; Yang, Jianan; Delhivala, Vraj; LiKamWa, Robert.

HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications. Vol. 2018-February Association for Computing Machinery, Inc, 2018. p. 81-86.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hu, J, Yang, J, Delhivala, V & LiKamWa, R 2018, Characterizing the reconfiguration latency of image sensor resolution on android devices. in HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications. vol. 2018-February, Association for Computing Machinery, Inc, pp. 81-86, 19th International Workshop on Mobile Computing Systems and Applications, HotMobile 2018, Tempe, United States, 2/12/18. https://doi.org/10.1145/3177102.3177109
Hu J, Yang J, Delhivala V, LiKamWa R. Characterizing the reconfiguration latency of image sensor resolution on android devices. In HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications. Vol. 2018-February. Association for Computing Machinery, Inc. 2018. p. 81-86 https://doi.org/10.1145/3177102.3177109
Hu, Jinhan ; Yang, Jianan ; Delhivala, Vraj ; LiKamWa, Robert. / Characterizing the reconfiguration latency of image sensor resolution on android devices. HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications. Vol. 2018-February Association for Computing Machinery, Inc, 2018. pp. 81-86
@inproceedings{791b0fa6fe5f451089df80afe649ada0,
title = "Characterizing the reconfiguration latency of image sensor resolution on android devices",
abstract = "Advances in vision processing have ignited a proliferation of mobile vision applications, including augmented reality. However, limited by the inability to rapidly reconfigure sensor operation for performance-efficiency tradeoffs, high power consumption causes vision applications to drain the device’s battery. To explore the potential impact of enabling rapid reconfiguration, we use a case study around marker-based pose estimation to understand the relationship between image frame resolution, task accuracy, and energy efficiency. Our case study motivates that to balance energy efficiency and task accuracy, the application needs to dynamically and frequently reconfigure sensor resolution. To explore the latency bottlenecks to sensor resolution reconfiguration, we define and profile the end-to-end reconfiguration latency and frame-to-frame latency of changing capture resolution on a Google LG Nexus 5X device. We identify three major sources of sensor resolution reconfiguration latency in current Android systems: (i) sequential configuration patterns, (ii) expensive system calls, and (iii) imaging pipeline delay. Based on our intuitions, we propose a redesign of the Android camera system to mitigate the sources of latency. Enabling smooth transitions between sensor configurations will unlock new classes of adaptive-resolution vision applications.",
keywords = "Camera system, Image sensor, Mobile devices, Operating system optimization, Reconfiguration",
author = "Jinhan Hu and Jianan Yang and Vraj Delhivala and Robert LiKamWa",
year = "2018",
month = "2",
day = "12",
doi = "10.1145/3177102.3177109",
language = "English (US)",
volume = "2018-February",
pages = "81--86",
booktitle = "HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Characterizing the reconfiguration latency of image sensor resolution on android devices

AU - Hu, Jinhan

AU - Yang, Jianan

AU - Delhivala, Vraj

AU - LiKamWa, Robert

PY - 2018/2/12

Y1 - 2018/2/12

N2 - Advances in vision processing have ignited a proliferation of mobile vision applications, including augmented reality. However, limited by the inability to rapidly reconfigure sensor operation for performance-efficiency tradeoffs, high power consumption causes vision applications to drain the device’s battery. To explore the potential impact of enabling rapid reconfiguration, we use a case study around marker-based pose estimation to understand the relationship between image frame resolution, task accuracy, and energy efficiency. Our case study motivates that to balance energy efficiency and task accuracy, the application needs to dynamically and frequently reconfigure sensor resolution. To explore the latency bottlenecks to sensor resolution reconfiguration, we define and profile the end-to-end reconfiguration latency and frame-to-frame latency of changing capture resolution on a Google LG Nexus 5X device. We identify three major sources of sensor resolution reconfiguration latency in current Android systems: (i) sequential configuration patterns, (ii) expensive system calls, and (iii) imaging pipeline delay. Based on our intuitions, we propose a redesign of the Android camera system to mitigate the sources of latency. Enabling smooth transitions between sensor configurations will unlock new classes of adaptive-resolution vision applications.

AB - Advances in vision processing have ignited a proliferation of mobile vision applications, including augmented reality. However, limited by the inability to rapidly reconfigure sensor operation for performance-efficiency tradeoffs, high power consumption causes vision applications to drain the device’s battery. To explore the potential impact of enabling rapid reconfiguration, we use a case study around marker-based pose estimation to understand the relationship between image frame resolution, task accuracy, and energy efficiency. Our case study motivates that to balance energy efficiency and task accuracy, the application needs to dynamically and frequently reconfigure sensor resolution. To explore the latency bottlenecks to sensor resolution reconfiguration, we define and profile the end-to-end reconfiguration latency and frame-to-frame latency of changing capture resolution on a Google LG Nexus 5X device. We identify three major sources of sensor resolution reconfiguration latency in current Android systems: (i) sequential configuration patterns, (ii) expensive system calls, and (iii) imaging pipeline delay. Based on our intuitions, we propose a redesign of the Android camera system to mitigate the sources of latency. Enabling smooth transitions between sensor configurations will unlock new classes of adaptive-resolution vision applications.

KW - Camera system

KW - Image sensor

KW - Mobile devices

KW - Operating system optimization

KW - Reconfiguration

UR - http://www.scopus.com/inward/record.url?scp=85048544726&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048544726&partnerID=8YFLogxK

U2 - 10.1145/3177102.3177109

DO - 10.1145/3177102.3177109

M3 - Conference contribution

AN - SCOPUS:85048544726

VL - 2018-February

SP - 81

EP - 86

BT - HotMobile 2018 - Proceedings of the 19th International Workshop on Mobile Computing Systems and Applications

PB - Association for Computing Machinery, Inc

ER -