System comparison procedures for automatic target recognition systems

Anne E. Catlin, Kenneth W. Bauer, Edward F. Mykytka, Douglas Montgomery

Research output: Contribution to journalArticlepeer-review

1 Scopus citations


Estimating the performance of an automatic target recognition (ATR) system in terms of its probability of successfully identifying a target involves extensive image collection and processing, which can be very time-consuming and expensive. Therefore, we investigate the Wald sequential test for the difference in two proportions as a sample size-reducing alternative to ranking and selection procedures and confidence intervals. An analysis of the test parameters leads to a practical methodology for implementing the Wald test for fairly comparing two systems, based on specific experimental goals. The test is also extended with the modified, sequentially rejective Bonferroni procedure for the multiple pairwise comparison of more than two systems. Two sampling schemes for different experimental goals are also discussed. The test methodology is applied to actual data to compare different configurations of a specific ATR system, with the results demonstrating that the modified Wald sequential procedure is a useful alternative to comparing proportions with confidence intervals, particularly when data are expensive.

Original languageEnglish (US)
Pages (from-to)357-371
Number of pages15
JournalNaval Research Logistics
Issue number4
StatePublished - Jun 1999

ASJC Scopus subject areas

  • Modeling and Simulation
  • Ocean Engineering
  • Management Science and Operations Research


Dive into the research topics of 'System comparison procedures for automatic target recognition systems'. Together they form a unique fingerprint.

Cite this