The piranha problem: Large effects swimming in a small pond.
Christopher Tosh, Philip Greengard, Ben Goodrich, Andrew Gelman, @avehtari, @djhsu
2 Apr 2024
https://arxiv.org/abs/2105.13445
In a lot of social science research, small, random factors are reported as having large effects on social and political attitudes and behavior (social priming, hormonal levels,parental socioeconomic status, weather, ...). Studies have claimed to find large effects from these and other inputs.
The results show that it would be extremely unlikely to have all these large effects coexisting—they would have to almost exactly cancel each other out.