Topic

P-value histograms

What does the shape of my p-value distribution say about my test?

A well-behaved test gives a flat histogram with a spike near zero. A U shape, a hump in the middle, or a conservative pile-up near one each diagnose a specific problem: wrong distribution, unmodeled batch, or filtering after testing. It is the cheapest diagnostic in statistics and almost nobody looks at it.