SHF:Small: Reproducibility and Comprehensive Assessment of Next Generation Sequencing Bioinformatics Software

下一代测序是指大约十年前产生的高通量DNA测序技术的集合，由于其成本效益和普遍性和多功能性，现在已成为所有现代基因组学研究的事实上的设备。该项目正在进行全面的可重复性和评估实验，以确定该领域的最新状况，并使研究结果公开可见和可获得。该项目的结果有望成为从业者、研究人员和下一代测序生物信息学的大量用户的宝贵资源。该项目涉及几名本科生，并提高了年轻研究人员对研究完整性和可重复性问题的认识。该项目正在建立基准数据集，以评估多种下一代测序仪、多种生物有机体类型、多种应用环境和多种问题规模的生物信息学软件。研究范围包括用于读取错误校正的软件产品评估，目标基因组和参考数据库的读取映射，以及基因组和转录组的组装。根据文献发表的结果和数据集，进行重复性实验，独立验证重要软件产品的结果。软件产品也在一系列指标上进行评估-结果质量，对参数值的稳健性和敏感性，运行时性能，内存使用以及处理真实数据集的能力。该项目的工作将为从业者提供全面的建议，并建立最先进的技术，以适当地引导未来的研究工作。

Next-generation sequencing refers to a collection of high throughput DNA sequencing technologies that have originated about a decade ago, and are now the de facto equipment underpinning all modern genomics studies due to their cost-effectiveness and ubiquity and versatility of use. This project is conducting comprehensive reproducibility and assessment experiments to characterize the state of the art in the field, and make the findings publicly visible and accessible. The project results are expected to become a valuable resource for practitioners, researchers, and the significantly large community of users of next generation sequencing bioinformatics. The project is involving several undergraduate students, and raising awareness of research integrity and reproducibility issues among young researchers.The project is establishing benchmark datasets to evaluate bioinformatics software for multiple next generation sequencers, multiple types of biological organisms, in multiple application contexts, and at multiple problem scales. The research spans assessment of software products for read error correction, read mapping to target genomes and reference databases, and assembly of genomes and transcriptomes. Reproducibility experiments are conducted to independently verify results of important software products based on results and datasets published in the literature. The software products are also evaluated on a range of metrics - quality of results, robustness and sensitivity to parameter values, run-time performance, memory usage, and ability to process real-world datasets. The project work will result in comprehensive recommendations available to practitioners as well as establishing state of the art to appropriately channel future research efforts.