Financial markets for binary prediction options which pay upon successful replication would provide insight into chances of a finding being true. Professors could be paid with these options.
Nit: Philopsophy is *supposed* to reflect the real world (but I agree it doesn't; one of my blogs: https://mugwumpery.com/?p=587).
I've worked in research biology for the last 15 years, this is my field nowadays.
The replication crisis is real; we see less than 20% of published papers replicate when we try (and since I'm pseudonymous - we're a really good lab).
Almost 10 years ago I proposed "Project Popper", a web-based mechanism for assigning scientists reputation scores based on replication of their work (modeled on eBay and Stack Exchange reputations; I ran it past Robin Hanson, who didn't seem impressed).
A prediction market is a very plausible solution, but there are difficulties.
Salaries based solely on replication would incent "trivially true" papers - work that doesn't present anything really new or useful, but is pretty sure to replicate. Maybe journals would push back against publishing such.
A larger problem is how to incentivize replication attempts. This seems to be the core of the problem - good journals want to publish work that's new and surprising and important. Replications don't get published in good journals, but papers in good journals are what makes a career.
Another issue how to judge replication results - when replication fails the original authors are likely to claim the replicators screwed up.
Nit: Philopsophy is *supposed* to reflect the real world (but I agree it doesn't; one of my blogs: https://mugwumpery.com/?p=587).
I've worked in research biology for the last 15 years, this is my field nowadays.
The replication crisis is real; we see less than 20% of published papers replicate when we try (and since I'm pseudonymous - we're a really good lab).
Almost 10 years ago I proposed "Project Popper", a web-based mechanism for assigning scientists reputation scores based on replication of their work (modeled on eBay and Stack Exchange reputations; I ran it past Robin Hanson, who didn't seem impressed).
A prediction market is a very plausible solution, but there are difficulties.
Salaries based solely on replication would incent "trivially true" papers - work that doesn't present anything really new or useful, but is pretty sure to replicate. Maybe journals would push back against publishing such.
A larger problem is how to incentivize replication attempts. This seems to be the core of the problem - good journals want to publish work that's new and surprising and important. Replications don't get published in good journals, but papers in good journals are what makes a career.
Another issue how to judge replication results - when replication fails the original authors are likely to claim the replicators screwed up.