Pilot study of the utility of text mining and machine learning tools to accelerate systematic review and meta-analysis of findings of in vivo research

生物医学研究是一个渐进的过程，其中一个实验的发现为未来的实验提供信息，并在未来的实验中受到挑战或证实。如果一个实验的结果不能用于那些计划未来实验的人，研究过程就会失去效率。广泛获取研究成果是开放获取传播的重要驱动力，也是提高研究效率的重要因素。现在发表的研究比以往任何时候都多。生物医学研究的主要书目数据库PubMed每天增加大约3500个新的参考文献。我们对PubMed上2000篇论文的随机抽样表明，2013年有98000篇论文描述了体内实验，其中21000篇是药理学，14500篇是神经科学。没有人能够阅读，更不用说批判性地评价或使用这些新信息中的一小部分，这些信息是研究者数月努力和大量研究资金投入的产物。产生的研究数量与可以有效利用的数量之间的这种不匹配是生物医学研究面临的主要挑战。Cochrane协作在综合临床试验数据的荟萃分析方面取得了巨大的成功，并以易于吸收、广泛认可、易于用于医疗资助决策和日常临床实践的格式提供结果。这种方法也影响了研究质量的重大改进，特别是临床试验的设计、实施和报告。虽然我们希望在临床前领域复制Cochrane的成功，但我们认识到，临床前数据的绝对数量和发表率表明，除了目前大多数临床系统评价采用的大部分手动过程之外，还需要方法创新。例如，在我们最近完成的神经性疼痛的系统综述中，需要提取229个临床试验的数据，而在相应的正在进行的临床前系统综述中，通过检索检索了65,156份出版物，必须筛选33,818份，这些数据是从约6000份中提取的。此外，对于已发表的研究中的偏倚风险（由于次优实验设计）和发表偏倚存在很大的担忧，偏倚可能会夸大观察到的效果。同样，在样本量小的地方（而且样本量的计算很少被报道），重要的生物效应也有被忽视的风险，因为个别研究的能力不足。简而言之，挑战是：1。与科学家潜在相关的信息以如此之多和如此之快的速度产生，以至于“阅读文献”是不可行的。体内研究的偏倚风险是如此之大，以至于需要进行详细的批判性评估，以判断得出的结论是否合理，以及特定的实验设计是否合适。发表偏倚意味着依赖特定来源（如特定期刊）的科学家很可能被误导。传统的系统评价可能会有所帮助，但通常在发表之日就已经过时了一到两年，临床前系统评价中隐含的大量数据进一步加剧了这个问题。我们建议利用文本挖掘和机器学习的最新发展，以确定这些是否还处于可以在体内数据的系统评价中实施的阶段，以协助应对上述挑战。

Biomedical research is an incremental process, in which the findings from one experiment inform, and are challenged or confirmed in, future experiments. Where findings from one experiment are not available to those planning future experiments, the research process loses efficiency. Making results of research widely available is an important driver for open access dissemination, and has been identified as an important factor in increasing research efficiency. There is now more research published than ever before. The primary bibliographic database for biomedical research, PubMed, adds around 3,500 new references every day. Our random sample of 2000 publications in PubMed suggests that in 2013 there were 98,000 publications describing in vivo experiments, of which 21,000 were in pharmacology and 14,500 in neuroscience. . No one individual can read, let alone appraise critically or use, even a small fraction of this new information, information which is the product of months of investigator effort and substantial investment of research funds. This mis-match, between the amount of research produced and the amount that can be effectively used, is a major challenge to biomedical research. The Cochrane Collaboration has been highly successful in synthesising meta-analyses of clinical trial data and providing outcomes in an easily assimilated, widely recognised, format readily useable for healthcare funding decisions and day to day clinical practice. This approach has also influenced major improvements in research quality, especially the design, conduct and reporting of clinical trials. Whilst we wish to replicate the success of Cochrane in the pre-clinical domain, we recognise that the sheer volume and publication rate of pre-clinical data predicate that methodology innovations are required beyond the largely manual processes that are currently adopted for most clinical systematic reviews. For example, in our recently completed systematic reviews of neuropathic pain, data from 229 clinical trials required extraction, whereas for the corresponding on-going pre-clinical systematic review 65,156 publications were retrieved by the search, 33,818 had to be screened and of these data are being extracted from ~6000. Further, there are substantial concerns about the risk of bias (due to sub-optimal experimental design) and publication bias in that work which is published, bias that is likely to overstate observed effects. Also where sample sizes are low (and sample size calculations are seldom reported), there is also a risk that important biological effects are overlooked because individual studies are underpowered.In brief then, the challenges are:1. Information of potential relevance to scientists is produced at such a volume and rate that "reading the literature" is not feasible2. The risk of bias in in vivo research is such that detailed critical appraisal is required to allow judgement of whether the conclusions drawn are justified and whether a particular experimental design is appropriate3. Publication bias means that scientists relying on selected sources (eg particular journals) are likely to be misled4. Conventional systematic review can be helpful, but are usually one to two years out of date on the day of publication, a problem that is further compounded by the sheer volume of data implicit in a pre-clinical systematic reviewHere we propose to exploit recent developments in text mining and machine learning to establish whether these are yet at the stage where they can be implemented in systematic reviews of in vivo data, to assist with the challenges outlined above.