SHF:Small: Reproducibility and Comprehensive Assessment of Next Generation Sequencing Bioinformatics Software

下一代测序是指大约十年前产生的一系列高通量DNA测序技术，由于其成本效益、普遍性和多功能性，现在已成为支撑所有现代基因组学研究的事实上的设备。该项目正在进行全面的可重复性和评估实验，以表征该领域的最新技术，并使研究结果公开可见和可访问。该项目的成果有望成为从业者、研究人员和下一代测序生物信息学庞大的用户社区的宝贵资源。该项目涉及几名本科生，并在年轻研究人员中提高对研究完整性和重复性问题的认识。该项目正在建立基准数据集，以评估用于多个下一代测序仪、多种类型的生物有机体、多个应用背景和多个问题尺度的生物信息学软件。这项研究包括评估用于读取纠错的软件产品、目标基因组和参考数据库的读取映射以及基因组和转录本的组装。再现性实验是基于文献中公布的结果和数据集独立地验证重要软件产品的结果。软件产品还根据一系列指标进行评估-结果质量、对参数值的稳健性和敏感度、运行时性能、内存使用情况以及处理真实数据集的能力。该项目的工作将导致向从业人员提供全面的建议，并建立最先进的状态，以适当地引导未来的研究工作。

Next-generation sequencing refers to a collection of high throughput DNA sequencing technologies that have originated about a decade ago, and are now the de facto equipment underpinning all modern genomics studies due to their cost-effectiveness and ubiquity and versatility of use. This project is conducting comprehensive reproducibility and assessment experiments to characterize the state of the art in the field, and make the findings publicly visible and accessible. The project results are expected to become a valuable resource for practitioners, researchers, and the significantly large community of users of next generation sequencing bioinformatics. The project is involving several undergraduate students, and raising awareness of research integrity and reproducibility issues among young researchers.The project is establishing benchmark datasets to evaluate bioinformatics software for multiple next generation sequencers, multiple types of biological organisms, in multiple application contexts, and at multiple problem scales. The research spans assessment of software products for read error correction, read mapping to target genomes and reference databases, and assembly of genomes and transcriptomes. Reproducibility experiments are conducted to independently verify results of important software products based on results and datasets published in the literature. The software products are also evaluated on a range of metrics - quality of results, robustness and sensitivity to parameter values, run-time performance, memory usage, and ability to process real-world datasets. The project work will result in comprehensive recommendations available to practitioners as well as establishing state of the art to appropriately channel future research efforts.