Pilot study of the utility of text mining and machine learning tools to accelerate systematic review and meta-analysis of findings of in vivo research

点击翻译按钮获取中文摘要

Biomedical research is an incremental process, in which the findings from one experiment inform, and are challenged or confirmed in, future experiments. Where findings from one experiment are not available to those planning future experiments, the research process loses efficiency. Making results of research widely available is an important driver for open access dissemination, and has been identified as an important factor in increasing research efficiency. There is now more research published than ever before. The primary bibliographic database for biomedical research, PubMed, adds around 3,500 new references every day. Our random sample of 2000 publications in PubMed suggests that in 2013 there were 98,000 publications describing in vivo experiments, of which 21,000 were in pharmacology and 14,500 in neuroscience. . No one individual can read, let alone appraise critically or use, even a small fraction of this new information, information which is the product of months of investigator effort and substantial investment of research funds. This mis-match, between the amount of research produced and the amount that can be effectively used, is a major challenge to biomedical research. The Cochrane Collaboration has been highly successful in synthesising meta-analyses of clinical trial data and providing outcomes in an easily assimilated, widely recognised, format readily useable for healthcare funding decisions and day to day clinical practice. This approach has also influenced major improvements in research quality, especially the design, conduct and reporting of clinical trials. Whilst we wish to replicate the success of Cochrane in the pre-clinical domain, we recognise that the sheer volume and publication rate of pre-clinical data predicate that methodology innovations are required beyond the largely manual processes that are currently adopted for most clinical systematic reviews. For example, in our recently completed systematic reviews of neuropathic pain, data from 229 clinical trials required extraction, whereas for the corresponding on-going pre-clinical systematic review 65,156 publications were retrieved by the search, 33,818 had to be screened and of these data are being extracted from ~6000. Further, there are substantial concerns about the risk of bias (due to sub-optimal experimental design) and publication bias in that work which is published, bias that is likely to overstate observed effects. Also where sample sizes are low (and sample size calculations are seldom reported), there is also a risk that important biological effects are overlooked because individual studies are underpowered.In brief then, the challenges are:1. Information of potential relevance to scientists is produced at such a volume and rate that "reading the literature" is not feasible2. The risk of bias in in vivo research is such that detailed critical appraisal is required to allow judgement of whether the conclusions drawn are justified and whether a particular experimental design is appropriate3. Publication bias means that scientists relying on selected sources (eg particular journals) are likely to be misled4. Conventional systematic review can be helpful, but are usually one to two years out of date on the day of publication, a problem that is further compounded by the sheer volume of data implicit in a pre-clinical systematic reviewHere we propose to exploit recent developments in text mining and machine learning to establish whether these are yet at the stage where they can be implemented in systematic reviews of in vivo data, to assist with the challenges outlined above.