Post

Image duplication has been a powerful signal for detecting scientific fraud, but is irrelevant in many fields. I've been working a bit on finding new signals like it that work across fields. I've found one using LLMs that can predict retractions, weakly, for $1 per paper. 1/4

From a few independent studies, 15-25% of papers show signs of faked results. There are many caveats here, but we do know that reproducibility of papers is below 50% and may be related to this. LLMs offer some chance of automating this analysis. 2/4

This is still an early topic and my work is very preliminary, but I think we may be able to start auditing scientific literature at scale. I’ve written up a lot of thoughts, background, and analysis in a blog post: diffuse.one/p/d1-008 4/4