Hierarchical recognition of sparse patterns in large-scale simultaneous inference.

 

Wenguang Sun, and Zhi Wei

 

Summary. We study how to accurately separate signals from noisy data and determine the patterns of the selected signals. Controlling the inflation of false positive errors is an important issue in large-scale simultaneous inference but has not been addressed in the pattern recognition literature. We develop a decision-theoretic framework and formulate the sparse pattern recognition problem as a simultaneous inference problem with multiple decision trees. Oracle and adaptive classifiers are proposed for maximizing the expected number of true positives subject to a constraint on the overall false positive rate. Existing results on multiple testing are extended by allowing more than two states of nature, hierarchical decision-making and new error rate concepts.

 

The paper and supplementary material can be downloaded here.

 

The R code for the restricted Bayes classifier and -adaptive classifier.