Selective Inference: The Silent Killer of Replicability

The replicability problems across varied scientific disciplines have attracted increasing attention in the last two decades. Selective inference, namely the unadjusted inference on the few promising ones, selected as such, is a major source of the problems. Unfortunately, the problem is ignored in many important and highly visible areas of science, and p-values and significance testing are being blamed. I will describe this attack, and the politics surrounding it, in some detail. After presenting this background, the talk will focus on new hierarchical FDR strategies to address selective inference, and on false coverage-statement rate confidence intervals. I shall demonstrate these in the contexts of genomic research, gene-brain associations, NEJM new guidelines as to secondary outcomes, and the Replicability Project in Psychology.