CDS&E: Two- and Three-point Correlations for Large Data Sets with TreeCorr

CDS

• 批准号: 1907610
• 负责人: Robert Jarvis
• 金额: $ 26.91万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1907610&HistoricalAwards=false
• 关键词: CDS& Two; Three; point; Correlations

This researcher has developed the leading software for measuring two-point correlation functions on large data sets in astronomy and elsewhere. However, for very large datasets the current version must be split across multiple computer nodes, which can be complicated to do correctly and efficiently. This project will enhance the software to use the Message Passing Interface (MPI) to split a job efficiently over a given number of nodes. Other software changes will further increase efficiency, add capabilities for three-point cross correlations, and include a more accurate shear estimation method. All of these modifications have been explicitly requested by current users of the software, and will make it especially useful for future experiments with much larger data volumes and tighter accuracy requirements. These improvements will help many scientists to carry out their research more efficiently and effectively. Throughout the project, the principal investigator will continue his campaign to improve the coding skills of other astronomers, helping them with code design, algorithm choices, good development workflow, and other software development issues. Some of his lessons are already freely available through YouTube.The software, TreeCorr, uses a very efficient ball-tree algorithm to compute correlations in O(N log N) time, making it significantly faster than other available options. TreeCorr uses OpenMP within a single computer node, and adding MPI will provide optimum overall efficiency. Other improvements include changing internal code for how TreeCorr decides when to stop traversing its tree, allowing arbitrary binning and arbitrarily small bin slop. Three-point auto-correlations are already supported at O(N log N) complexity, but the three-point cross-correlations that enable studies such as count-count-shear have been unavailable due to the complex bookkeeping required: this limitation will be removed. The new shear method, MetaCalibration, requires additional calculations that again can be somewhat complicated to do correctly. Correct and efficient code for MetaCalibration will be built into TreeCorr. This software has been downloaded via pip over 40,000 times and is used by many research teams working on current and forthcoming surveys: they will all benefit greatly from the algorithmic advances to be made by this study.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究人员开发了领先的软件，用于测量天文学和其他地方的大型数据集上的两点相关函数。 然而，对于非常大的数据集，当前版本必须在多个计算机节点上拆分，这可能很复杂，无法正确有效地完成。 该项目将增强软件，使用消息传递接口（MPI）在给定数量的节点上有效地分割作业。 其他软件更改将进一步提高效率，增加三点互相关功能，并包括更准确的剪切估计方法。 所有这些修改都是由软件的当前用户明确要求的，并且将使其对于未来具有更大数据量和更严格精度要求的实验特别有用。 这些改进将帮助许多科学家更有效地开展研究。 在整个项目中，首席研究员将继续提高其他天文学家的编码技能，帮助他们进行代码设计，算法选择，良好的开发工作流程和其他软件开发问题。 他的一些课程已经可以通过YouTube免费获得。该软件TreeCorr使用非常有效的球树算法来计算相关性，时间复杂度为O（N log N），比其他可用选项快得多。 TreeCorr在单个计算机节点中使用OpenMP，添加MPI将提供最佳的整体效率。 其他改进包括更改TreeCorr决定何时停止遍历其树的内部代码，允许任意装箱和任意小的bin slop。 三点自相关已经在O（N log N）复杂度下得到支持，但是由于需要复杂的簿记，使得诸如计数-计数-剪切的研究无法使用三点互相关：这个限制将被删除。 新的剪切方法MetaCalibration需要额外的计算，要正确完成这些计算也会有些复杂。 MetaCalibration的正确和有效代码将内置到TreeCorr中。 该软件已通过pip下载超过40，000次，并被许多研究团队用于当前和即将进行的调查：他们都将从这项研究所取得的算法进步中受益匪浅。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。