P values maximized over a confidence set for the nuisance parameter

For testing problems of the form H₀: v = v₀ with unknown nuisance parameter θ, various methods are used to deal with θ. The simplest approach is exemplified by the t test where the unknown variance is replaced by the sample variance and the t distribution accounts for estimation of the variance. In other problems, such as the 2 × 2 contingency table, one conditions on a sufficient statistic for 0 and proceeds as in Fisher’s exact test. Because neither of these standard methods is appropriate for all situations, this article suggests a new method for handling the unknown θ. This new method is a simple modification of the formal definition of a p value that involves taking a maximum over the nuisance parameter space of a p value obtained for the case when θ is known. The suggested modification is to restrict the maximization to a confidence set for the nuisance parameter. After giving a brief justification, we give various examples to show how this new method gives improved results for 2 × 2 tables and solves previously intractable semiparametric problems.