THE FUNDAMENTAL PRINCIPLE OF PROBABILITY: RESOLVING THE REPLICATION CRISIS WITH SKIN IN THE GAME
THE FUNDAMENTAL PRINCIPLE OF PROBABILITY: RESOLVING THE REPLICATION CRISIS WITH SKIN IN THE GAME
复制标题
概率的基本原理：以博弈论解决复制危机
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
Harry Crane
中科院分区：
文献类型：
--
作者：
Harry Crane
如果你不会故意投资一家对冲基金，其经理因产生高额回报而获得的回报与破产时获得的补偿相同，或者你不会把赌注押在一个输了不付清钱的博彩公司身上，或者不相信一个不审查自己消息来源的记者，为什么你会相信科学家的说法，当他们的假设错误、分析有缺陷或发现错误时，他们几乎没有什么可以损失的？投资、收受赌注和新闻报道业务绝不是完美的，但至少它们有内在的机制来协调生产者(基金经理、博彩公司、记者)和消费者(投资者、赌徒、读者)的利益。基金经理、博彩公司和记者必须冒着自己的财富、声誉和信誉的风险，这样他们才能有资金要管理，有书要留，有读者要写。同样的情况也适用于医生、屠夫、厨师和(非工会组织)水管工。但为什么不是科学家呢？职业科学家不是根据客观建立的标准(例如，上面例子中的利润、荣誉和真相)进行评估，而是通过行为准则、同行评议和编辑委员会来监督自己，为自己提供大量的好处--以工作保障、舒适的工资、数百万美元的纳税人资金资助的形式，甚至如果他们的发现之一成为主流，甚至有可能名利双收--下降的可能性主要限于特殊情况，如欺诈或不当行为。这种不对称性与学术界“要么发表，要么灭亡”的文化相结合，导致了一种反常的激励结构，在这种结构中，快速和肮脏的人得到的回报超过了仔细和正确的人，这反过来又导致了所谓的科学“复制危机”。1十多年前，Ioannidis在一篇题为“为什么大多数已发表的研究结果都是错误的”的文章中呼吁人们注意这一系统的不利影响[Ioa05]。十多年后，桌面上出现了许多改革科学调查进行、评估和报告的方式的建议，并随之而来的是提高已发表研究的可靠性。但是，即使人们对改革产生了广泛的兴趣，复制危机也几乎没有消退的迹象，因为最近的一项努力只成功复制了心理学文献[Col15]上发表的结果的37%。这些问题将一直存在，直到科学像投资、博彩和新闻报道一样，实施一种内在的机制，将科学家的职业成功与他们结果的可靠性联系起来。我在这里辩称，这要容易得多
If you wouldn’t knowingly invest in a hedge fund whose manager is compensated the same for generating big returns as for going insolvent, or place a bet with a bookie who doesn’t pay up when he loses, or believe a journalist who doesn’t vet his sources, why would you trust claims made by scientists who have little or nothing to lose when their assumptions are wrong, analyses are flawed, or findings are false? The businesses of investing, bookmaking, and news reporting are by no means perfect, but at least they have built-in mechanisms to align the interests of producer (fund manager, bookie, journalist) and consumer (investor, gambler, reader). Fund managers, bookies, and journalists have to risk their own wealth, reputation, and credibility so that they have funds to manage, books to keep, and audiences to write for. The same goes for doctors, butchers, chefs, and (non-union) plumbers. But why not scientists? Rather than being evaluated against objectively established standards (e.g., profit, honor, and truth in the above examples), professional scientists police themselves with codes of conduct, peer review, and editorial boards, affording themselves a great deal of upside—in the form of job security, a comfortable salary, millions of dollars in taxpayer-funded grants, and even the possibility of fame and fortune if one of their discoveries hits the mainstream—with the potential for downside mostly limited to exceptional circumstances, such as fraud or misconduct. This asymmetry combined with academia’s ‘publish or perish’ culture leads to a perverse incentive structure in which quick-and-dirty is rewarded more than careful-and-correct, which in turn has led to the so-called scientific ‘replication crisis’.1 Ioannidis called attention to the adverse effects of this system more than a decade ago in an essay titled “Why Most Published Research Findings Are False” [Ioa05]. More than a decade later, there are numerous proposals on the table to reform the way scientific inquiries are conducted, evaluated, and reported, and with it improve the reliability of published research. But even with widespread interest in reform, the replication crisis shows little sign of receding, as one recent effort successfully replicated only 37% of results published in the psychology literature [Col15]. These problems will persist until science, like investing, bookmaking, and news reporting, implements a built-in mechanism that aligns the career success of scientists with the reliability of their results. I argue here that this is much easier to achieve