CRCNS: Computational Approach to Assess Replicability of Neurobehavior Phenotypes

项目描述 (as提交给NSF） CRCNS：美国-以色列数据共享提案： 计算方法 评估神经行为表型的可复制性 Molly Bogue博士(PI)Yoav Benjamini博士（PI） 杰克逊实验室统计与运筹学系 美国缅因州巴尔港数学科学学院 萨格尔神经科学学院 以色列特拉维夫大学 Ilan Golani博士（共同投资） 动物学系 智慧生命科学学院 萨戈神经科学学院 以色列特拉维夫大学 Iliana Oozes博士（共同投资） 部人类分子遗传学和生物化学 Saclder医学院 萨戈神经科学学院 以色列特拉维夫大学 Neri Kaflcafi博士（共同投资） 统计和业务研究部 数学科学学院 以色列特拉维夫大学 先前的NSF和/或CRCNS支持：无 35 介绍 我们正在提交一份修改后的计算神经科学提案。我们最初的建议得到了积极的回应。 去年在研究提案机制下收到了评审员的意见。例如：“所有 评论家们对解决问题的变革潜力非常热情， 在提案中得到了解决，也对拟议的基本框架充满热情。 溶液“然而，之前的评审员认为我们的提案更符合数据共享 机制不幸的是，NSF-BSF去年不支持数据共享。今年是这样，所以我们 在数据共享机制下重新申请。我们已经修改和改进了我们的应用程序， 解决先前审查者的关切。他们关注的问题可分为四类：1）难以 获取新数据，2）处理稀疏数据的挑战，3）缺乏关于我们的具体细节 方法，以及4）误解杰克逊实验室是一个营利性组织， 会影响数据发布。JAX是一家非营利研究机构，在提供 公共生物信息学资源的实验室小鼠没有任何限制。 背景和意义 关注神经行为表型的可复制性 科学界和非科学界越来越关注已发表的“发现” 不可复制[10-12]。知名机构和期刊，包括NIH [13]、Science [14]和 自然[15]，最近宣布了关于这个问题的政策立场，但仍然存在混乱， 讨论如何解决这个问题。虽然在所有科学领域都有记载， 在临床前研究中特别注意到一个问题[16]，包括小鼠神经和行为 研究[7]。注意：为了避免混淆，我们使用术语“可复制性”来复制其他 研究和“再现性”，用于在同一研究中再现结论[17，18]。 ANOVA方差可重复 总方差原理 G级 * 是的 0% 100% GXL NS G级 * L * 是 0% 100% GXL NS G级 * L * 是 0'1/o 100% GXL NS G ns L * 否 0% 100% Gxl * g1 g2 图1.主要影响的重要性的说明 基因型（G）、实验室（L）和基因型x实验室 (GxL)互动四种不同行为测量的数据 在三个实验室的左侧面板中进行了描述（彩色 编码）两种基因型（g）。每个的ANOVA结果是 其中 *=显著; ns=不显著。比例 总方差（%）由颜色编码的水平 酒吧.可复制性，基于GXL的重要性， 在右边，每个人都有自己的选择。 定量表型的测定 基因工程小鼠品系 成为一个中心战略， 哺乳动物基因功能， 表征疾病的动物模型， 可以研究哪些假定的治疗方法（对于 评论见[19，20]）。国际 小鼠表型鉴定协会（IMPC） [21[22]协调国际努力 使成千上万的突变株表型化， 最终实现功能注释 大约20,000个蛋白质编码基因中的大部分 在哺乳动物基因组中。一个基本 这项工作的一个方面是使高- 通量表型分析数据， 科学界在公众面前 数据库[23，24]。这个的效用 然而，这项工作的关键取决于 复制表型分析结果的能力 在其他实验室。这个大项目是 但有一个例子表明， 小鼠表型的可复制性， 被反复提出和讨论 (e.g., [27 25、不满意 解决方案尚未通过。任何解决方案， 可能被实验者采用， 36

PROJECT DESCRIPTION (as submitted to NSF) CRCNS: US - Israel Data Sharing Proposal: Computational Approaches to Assess Replicability of Neurobehavioral Phenotypes Molly Bogue, Ph.D. (PI) Yoav Benjamini, Ph.D. (PI) The Jackson Laboratory Department of Statistics and Operations Research Bar Harbor, Maine USA School of Mathematical Sciences Sagol School ofNeuroscience Tel Aviv University ISRAEL Ilan Golani, Ph.D. (Co-Inv) Department of Zoology Wise Faculty of Life Sciences Sago! School of Neuroscience Tel Aviv University ISRAEL Iliana Oozes, Ph.D. (Co-Inv) Dept. of Human Molecular Genetics and Biochemistry Saclder Faculty of Medicine Sago! School of Neuroscience Tel Aviv University ISRAEL Neri Kaflcafi, Ph.D. (Co-Inv) Department of Statistics and Operations Research School of Mathematical Sciences Tel Aviv University ISRAEL Prior NSF and/or CRCNS Support: None 35 Introduction We are submitting a revised computational neuroscience proposal. Our original proposal received positive comments from reviewers last year under the Research Proposal mechanism. For example: "All reviewers were very enthusiastic about the transformative potential of solving the problem being addressed in the proposal, and were also enthusiastic about the basic framework underlying the proposed solution." However, previous reviewers thought our proposal was better aligned with the Data Sharing mechanism. Unfortunately, NSF-BSF did not support Data Sharing last year. This year it does and so we are reapplying under the Data Sharing mechanism. We have revised and improved our application to address previous reviewers' concerns. Their concerns can be grouped in four categories: 1) difficulty in acquiring new data, 2) challenge in dealing with sparse data, 3) lack of specific details about our approach, and 4) misconception that The Jackson Laboratory is a for-profit organization and that this would affect data release. JAX is a not-for-profit research institution and is a world leader in providing public bioinformatics resources for the laboratory mouse without any restrictions. Background and Significance Concernsaboutreplicabilityofneurobehavioralphenotypes The scientific and lay communities have become increasingly concerned with published "discoveries" that are not replicable [10-12]. Prominent institutions and journals, including NIH [13], Science [14] and Nature [15], have recently announced policy positions on the subject, yet there is still confusion and debate regarding how the problem should be addressed. While documented in all scientific fields, the problem was specifically noted in preclinical research [16] including mouse neurological and behavioral studies [7]. Note: To prevent confusion, we use the term "replicability" for replicating results in other studies and "reproducibility" for reproducing conclusions within the same study [17, 18]. ANOVA Variance Replicable Prnportlon of total variance G* L ns Yes 0% 100% Gxl ns G* L * Yes 0% 100% Gxl ns G* L * Yes 0'1/o 100% Gxl ns G ns L * No 0% 100% Gxl * g1 g2 Figure 1. Illustration of the significance of the main effects of Genotype (G), Laboratory (L) and the Genotype x Laboratory (GxL) interaction. Data for four different behavioral measures are depicted in the left panel across three laboratories (color coded) for two genotypes (g). ANOVA results for each are shown where *=significant; ns=not significant. Proportion of total variance (%) is illustrated by color-coded horizontal bars. Replicability, based on the significance of GxL, is indicated on the right for each measure. Measuring quantitative phenotypes of genetically engineered mouse strains has become a central strategy for discovering mammalian gene function, and for characterizing animal models of disease in which putative cures can be researched (for reviews see [19, 20]). The International Mouse Phenotyping Consortium (IMPC) [21, 22] coordinates an international effort to phenotype thousands of mutant strains, eventually achieving a functional annotation of most of the ~20,000 protein-coding genes in the mammalian genome. An essential aspect of this work is making high- throughput phenotyping data accessible to the scientific community in public databases [23, 24]. The utility of this undertaking, however, critically depends on the ability to replicate phenotyping results in other laboratories. This large project is but one example of the general need for replicability of mouse phenotyping, which has been repeatedly raised and discussed (e.g., [27 25 ,7] with no satisfactory solution yet adopted. Any solution that is likely to be adopted by experimentalists for 36