CRCNS: Computational Approach to Assess Replicability of Neurobehavior Phenotypes

项目说明 (提交给NSF) CRCNS：美以数据共享提议： 计算方法可用于 评估神经行为表型的可重复性 莫莉·博格博士(PI)约阿夫·本贾米尼博士(PI) 杰克逊实验室统计和运筹学系 美国缅因州巴尔港数学科学学院 萨戈尔神经科学学院 以色列特拉维夫大学 伊兰·戈拉尼博士(联席投资) 动物学系 WISE生命科学学院 西米！神经科学学院 以色列特拉维夫大学 Iliana Offes，博士(联席投资) 部门人类分子遗传学与生物化学 萨克尔德医学院 西米！神经科学学院 以色列特拉维夫大学 Neri Kaflafi，博士(联席投资) 统计和运筹学系 数学科学学院 以色列特拉维夫大学 之前的NSF和/或CRCNS支持：无 35 引言 我们正在提交一份修订后的计算神经科学提案。我们最初的建议得到了肯定 去年在研究建议机制下的评审员的意见。例如：“所有 评论家们对解决问题的变革性潜力非常热情 ，并对提议的基本框架抱有热情。 然而，之前的评论者认为我们的建议更符合数据共享 机制。不幸的是，NSF-BSF去年不支持数据共享。今年是这样，所以我们 正在数据共享机制下重新申请。我们已经修改和改进了我们的申请 解决之前评审者所关注的问题。他们关心的问题可以分为四类：1)困难 获取新数据，2)处理稀疏数据的挑战，3)缺乏关于我们的 方法，以及误解杰克逊实验室是一个营利性组织，认为这 会影响数据发布。JAX是一家非营利性研究机构，在提供 实验室小鼠的公共生物信息学资源不受任何限制。 背景和意义 Concernsaboutreplicabilityofneurobehavioralphenotypes 科学界和非科学界越来越关注已发表的“发现”。 是不可复制的[10-12]。著名机构和期刊，包括NIH[13]、《科学》[14]和 自然[15]，最近宣布了关于这一主题的政策立场，但仍然存在混乱和 关于应该如何解决这个问题的辩论。虽然在所有科学领域都有记录，但 在临床前研究[16]中特别指出了这个问题，包括小鼠的神经学和行为学 研究[7]。注意：为了防止混淆，我们使用术语“可复制性”来复制其他 研究和在同一研究内复制结论的“再现性”[17，18]。 方差可复制 总方差报告 G* L：是的 0%100% GXL ns G* L*是的 0%100% GXL ns G* L*是的 0‘1/o 100% GXL ns G ns L*不 0%100% GXL* G1和G2 图1.主要影响的重要性图解 基因(G)、实验室(L)和基因x型实验室 (GXL)互动。四种不同行为测量的数据 显示在三个实验室的左侧面板中(颜色 编码)用于两个基因类型(G)。每一项的方差分析结果为 其中*=显著；ns=不显著。的比例 总方差(%)由颜色编码的水平表示 酒吧。基于GXL的重要性，可复制性是 在每一项措施的右侧标明。 数量表型的测定 基因工程小鼠品系已经 成为发现以下问题的中心战略 哺乳动物的基因功能，并为 描述疾病的动物模型 可以研究哪些假定的治疗方法(对于 评论见[19，20])。《国际邮报》 小鼠表型联合体(IMPC) [21，22]协调国际努力 对数以千计的突变菌株进行表型， 最终实现一个功能注解 大约20,000个蛋白质编码基因中的大多数 在哺乳动物的基因组中。必需品 这项工作的一个方面是使高度- 可访问的吞吐量表型数据 科学界在公众面前 数据库[23，24]。它的用处在于 然而，承诺关键取决于 复制表型结果的能力 在其他实验室。这个大型项目是 但一个普遍需要的例子是 小鼠表型的可重复性 已被反复提出和讨论 (例如，[27 25，7]，没有令人满意的 尚未采用的解决方案。任何解决方案都是 可能会被实验者采用为 36

PROJECT DESCRIPTION (as submitted to NSF) CRCNS: US - Israel Data Sharing Proposal: Computational Approaches to Assess Replicability of Neurobehavioral Phenotypes Molly Bogue, Ph.D. (PI) Yoav Benjamini, Ph.D. (PI) The Jackson Laboratory Department of Statistics and Operations Research Bar Harbor, Maine USA School of Mathematical Sciences Sagol School ofNeuroscience Tel Aviv University ISRAEL Ilan Golani, Ph.D. (Co-Inv) Department of Zoology Wise Faculty of Life Sciences Sago! School of Neuroscience Tel Aviv University ISRAEL Iliana Oozes, Ph.D. (Co-Inv) Dept. of Human Molecular Genetics and Biochemistry Saclder Faculty of Medicine Sago! School of Neuroscience Tel Aviv University ISRAEL Neri Kaflcafi, Ph.D. (Co-Inv) Department of Statistics and Operations Research School of Mathematical Sciences Tel Aviv University ISRAEL Prior NSF and/or CRCNS Support: None 35 Introduction We are submitting a revised computational neuroscience proposal. Our original proposal received positive comments from reviewers last year under the Research Proposal mechanism. For example: "All reviewers were very enthusiastic about the transformative potential of solving the problem being addressed in the proposal, and were also enthusiastic about the basic framework underlying the proposed solution." However, previous reviewers thought our proposal was better aligned with the Data Sharing mechanism. Unfortunately, NSF-BSF did not support Data Sharing last year. This year it does and so we are reapplying under the Data Sharing mechanism. We have revised and improved our application to address previous reviewers' concerns. Their concerns can be grouped in four categories: 1) difficulty in acquiring new data, 2) challenge in dealing with sparse data, 3) lack of specific details about our approach, and 4) misconception that The Jackson Laboratory is a for-profit organization and that this would affect data release. JAX is a not-for-profit research institution and is a world leader in providing public bioinformatics resources for the laboratory mouse without any restrictions. Background and Significance Concernsaboutreplicabilityofneurobehavioralphenotypes The scientific and lay communities have become increasingly concerned with published "discoveries" that are not replicable [10-12]. Prominent institutions and journals, including NIH [13], Science [14] and Nature [15], have recently announced policy positions on the subject, yet there is still confusion and debate regarding how the problem should be addressed. While documented in all scientific fields, the problem was specifically noted in preclinical research [16] including mouse neurological and behavioral studies [7]. Note: To prevent confusion, we use the term "replicability" for replicating results in other studies and "reproducibility" for reproducing conclusions within the same study [17, 18]. ANOVA Variance Replicable Prnportlon of total variance G* L ns Yes 0% 100% Gxl ns G* L * Yes 0% 100% Gxl ns G* L * Yes 0'1/o 100% Gxl ns G ns L * No 0% 100% Gxl * g1 g2 Figure 1. Illustration of the significance of the main effects of Genotype (G), Laboratory (L) and the Genotype x Laboratory (GxL) interaction. Data for four different behavioral measures are depicted in the left panel across three laboratories (color coded) for two genotypes (g). ANOVA results for each are shown where *=significant; ns=not significant. Proportion of total variance (%) is illustrated by color-coded horizontal bars. Replicability, based on the significance of GxL, is indicated on the right for each measure. Measuring quantitative phenotypes of genetically engineered mouse strains has become a central strategy for discovering mammalian gene function, and for characterizing animal models of disease in which putative cures can be researched (for reviews see [19, 20]). The International Mouse Phenotyping Consortium (IMPC) [21, 22] coordinates an international effort to phenotype thousands of mutant strains, eventually achieving a functional annotation of most of the ~20,000 protein-coding genes in the mammalian genome. An essential aspect of this work is making high- throughput phenotyping data accessible to the scientific community in public databases [23, 24]. The utility of this undertaking, however, critically depends on the ability to replicate phenotyping results in other laboratories. This large project is but one example of the general need for replicability of mouse phenotyping, which has been repeatedly raised and discussed (e.g., [27 25 ,7] with no satisfactory solution yet adopted. Any solution that is likely to be adopted by experimentalists for 36