SHF:Small: Reproducibility and Comprehensive Assessment of Next Generation Sequencing Bioinformatics Software

下一代测序是指大约十年前起源的高通量DNA测序技术的集合，并且由于其成本效益和普遍性以及使用的多功能性，现在是支撑所有现代基因组学研究的事实上的设备。该项目正在进行全面的再现性和评估实验，以表征该领域的最新技术水平，并使研究结果公开可见和可访问。该项目的结果有望成为从业者，研究人员和下一代测序生物信息学的用户的显着大社区的宝贵资源。该项目涉及多名本科生，并提高年轻研究人员对研究完整性和可重复性问题的认识。该项目正在建立基准数据集，以评估生物信息学软件在多个应用环境和多个问题尺度下的多个下一代测序仪，多种类型的生物有机体。该研究涵盖了用于读取错误校正的软件产品的评估，读取映射到靶基因组和参考数据库，以及基因组和转录组的组装。根据文献中公布的结果和数据集，进行重现性实验，以独立验证重要软件产品的结果。软件产品还根据一系列指标进行评估-结果质量，参数值的鲁棒性和敏感性，运行时性能，内存使用情况以及处理真实世界数据集的能力。项目工作将为从业人员提供全面的建议，并建立最先进的技术，以适当引导未来的研究工作。

Next-generation sequencing refers to a collection of high throughput DNA sequencing technologies that have originated about a decade ago, and are now the de facto equipment underpinning all modern genomics studies due to their cost-effectiveness and ubiquity and versatility of use. This project is conducting comprehensive reproducibility and assessment experiments to characterize the state of the art in the field, and make the findings publicly visible and accessible. The project results are expected to become a valuable resource for practitioners, researchers, and the significantly large community of users of next generation sequencing bioinformatics. The project is involving several undergraduate students, and raising awareness of research integrity and reproducibility issues among young researchers.The project is establishing benchmark datasets to evaluate bioinformatics software for multiple next generation sequencers, multiple types of biological organisms, in multiple application contexts, and at multiple problem scales. The research spans assessment of software products for read error correction, read mapping to target genomes and reference databases, and assembly of genomes and transcriptomes. Reproducibility experiments are conducted to independently verify results of important software products based on results and datasets published in the literature. The software products are also evaluated on a range of metrics - quality of results, robustness and sensitivity to parameter values, run-time performance, memory usage, and ability to process real-world datasets. The project work will result in comprehensive recommendations available to practitioners as well as establishing state of the art to appropriately channel future research efforts.