Nasim Taleb: The probability distribution of p values
Also see Fooled by Randomness
This is a companion to the short paper at
It explains the problem with p-values giving different values across statistically identical samples.
Sorry for the lighting but I am not into these things.
Abstract: We present an explicit and parsimonious probability distribution (meta-distribution) for p-values across ensembles of statistically identical phenomena, having for sole parameter the median “true” p-value. P-values are extremely skewed and volatile, regardless of the sample size n, and vary greatly across repetitions of exactly same protocols under identical stochastic copies of the phenomenon. The convenience of formula allows the investigation of scientific results, particularly meta-analyses, and the expression of such biases as “p-hacking”.
Typo: median p is not “true p” (skewness of p) but a proxy.