Interactively test driving an object detector: Estimating performance on unlabeled data

Rushil Anirudh, Pavan Turaga

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

In this paper, we study the problem of 'test-driving' a detector, i.e. allowing a human user to get a quick sense of how well the detector generalizes to their specific requirement. To this end, we present the first system that estimates detector performance interactively without extensive ground truthing using a human in the loop. We approach this as a problem of estimating proportions and show that it is possible to make accurate inferences on the proportion of classes or groups within a large data collection by observing only 5 - 10% of samples from the data. In estimating the false detections (for precision), the samples are chosen carefully such that the overall characteristics of the data collection are preserved. Next, inspired by its use in estimating disease propagation we apply pooled testing approaches to estimate missed detections (for recall) from the dataset. The estimates thus obtained are close to the ones obtained using ground truth, thus reducing the need for extensive labeling which is expensive and time consuming.

Original languageEnglish (US)
Title of host publication2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014
PublisherIEEE Computer Society
Pages175-182
Number of pages8
ISBN (Print)9781479949854
DOIs
StatePublished - 2014
Event2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014 - Steamboat Springs, CO, United States
Duration: Mar 24 2014Mar 26 2014

Publication series

Name2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014

Other

Other2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014
CountryUnited States
CitySteamboat Springs, CO
Period3/24/143/26/14

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Interactively test driving an object detector: Estimating performance on unlabeled data'. Together they form a unique fingerprint.

  • Cite this

    Anirudh, R., & Turaga, P. (2014). Interactively test driving an object detector: Estimating performance on unlabeled data. In 2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014 (pp. 175-182). [6836104] (2014 IEEE Winter Conference on Applications of Computer Vision, WACV 2014). IEEE Computer Society. https://doi.org/10.1109/WACV.2014.6836104