Mixed Precision FFTs

这项研究是我的本科同名硕士研究项目的延续。快速傅立叶变换(FFT)是一种在许多科学计算领域中使用非常广泛的算法。随着新一代计算硬件实现并支持使用新的数据类型(例如BFLOAT 16)，了解使用这些新模式对算法的影响是很重要的，FFT就是一个非常重要的例子。到目前为止，该项目已经在现有科学代码Astro-Accelerate的新版本中实现了16个浮点FFT，这是一个用于研究射电天文数据集的GPU(图形处理单元)加速代码。在Astro-Accelerate中，我设法将搜索给定文件所花费的FFT时间减少了50%，将节省的时间扩展到下一代望远镜(如SKA(平方公里阵列))的数据处理器所需的超级计算机的大小，这将转化为价值数百万磅的电力节约。它还可以用于减少数千万磅的前期硬件需求。该项目的进一步目标和目标是继续研究混合精度FFT在更广泛的范围内的影响，更普遍的应用。同样重要的是记录确定混合精度FFT在给定环境中是否有用的过程。这将涉及潜在地开发FFT的新实现，以便在新可用的硬件上使用特性(包括但不限于混合精度)。该项目的预期影响是允许使用FFT的研究人员和其他考虑使用混合精度的人了解改变他们的代码可能产生的影响，而不必自己动手。此外，该项目的发现可能有助于指导下一代计算硬件，因为我们可以找到特别有效/无效的方法来证明硬件改变的合理性。该项目可能采取的另一个方向是使用在数值分析领域发展的技术来研究当用于执行卷积时混合精度对通用FFT的影响。卷积被广泛应用于快速变化的机器学习领域，有趣的是，混合精度实现被宣传为取代现有单精度代码的下降。在某些情况下，它们不仅在计算机上运行得更快，而且由于正则化而产生更健壮的神经网络。这表明，了解如何量化给定应用程序的适当数值精度是极其重要的，而不仅仅是假设精度越高。到目前为止，这项研究涉及到低级编码的混合，使用CUDA C/C++编程和优化GPU代码，以及高级Python+MATLAB脚本来可视化和处理用C编写的软件输出。该项目属于EPSRC数字信号处理研究领域。

This research is a continuation of my undergraduate masters research project of the same title. The Fast Fourier Transform (FFT) is an extremely widely used algorithm in many areas of scientific computing. As new generations of computing hardware implement and enable the use of novel datatypes (for example bfloat16), it is important to understand the impact that using these new modes has on algorithms, of which the FFT is a very important example. So far, the project has involved implementing bfloat16 FFTs ina new version of an existing scientific code, Astro-Accelerate, whichis a GPU (graphics processing unit) accelerated code for studying radio astronomy datasets. In Astro-Accelerate, I managed to reduce the time spent doing FFTs in the search of a given file by 50%, scaling this saving up to the size of supercomputer that would be required inthe data processor of next generation telescopes, such as the SKA(Square Kilometre Array), this would translate to electricitysavings worth millions of pounds.It could also be used to reduce the upfront hardware requirement by tens of millions of pounds.The further aims and objectives of the project are to continue investigating the impact of mixed precision FFTs in wider, more general applications.Also it will be important to document the process of determining whether mixed precision FFTs are useful in a given context.This will involvepotentially developing novel implementations of the FFT to use features (including but not limited to mixed precision) on newly available hardware.The intended impact of the project is to allowresearchers using the FFT, and others considering using mixed precision to understand the impact that changing their code might have, without having to do it themselves. Additionally, the findings of this project may help to guide the next generations of computing hardware, as wemay find either particularly effective/ineffective approaches that could justify hardware changes.Another direction the project could take is using techniques developed in the field of numerical analysis to study the effect of mixed precision on generic FFTs when used to perform convolutions. Convolutions are widely used in the rapidly changing field ofmachine learning, and interestingly, mixed precision implementations are advertised as drop in replacements of existing single precision codes.In some cases they not only runfaster on a computer, but also producemore robust neural networks (due to regularisation). This demonstrates that it is extremely important to understand how to quantify an appropriate amount of numerical precision for a given application, and not just assume that more precision is better.So far, the research has involved a mixture of low level coding, using CUDA C/C++ to programand optimisethe GPUcode, as well as high level Python + MATLAB scripts to visualise and processthe output of the software written in C.This project falls within the EPSRC Digital Signal Processing research area.