Mixed Precision FFTs

本研究是我的本科硕士研究项目的延续。快速傅立叶变换（FFT）是在科学计算的许多领域中极其广泛使用的算法。随着新一代计算硬件实现并启用新数据库（例如bfloat 16），了解使用这些新模式对算法的影响非常重要，FFT就是其中一个非常重要的例子。到目前为止，该项目涉及在现有科学代码Astro-Accelerate的新版本中实现bfloat 16 FFT，Astro-Accelerate是用于研究射电天文学数据集的GPU（图形处理单元）加速代码。在Astro-Accelerate中，我成功地将搜索给定文件所花费的FFT时间减少了50%，将节省的时间扩展到下一代望远镜（如SKA）数据处理器所需的超级计算机的大小（平方公里阵列），这将转化为价值数百万英镑的电费节省。它还可以用于减少数千万美元的前期硬件需求。该项目的进一步目的和目标是继续研究混合精度FFT在更广泛，更一般的应用。同样重要的是记录确定混合精度FFT在给定上下文中是否有用的过程。这将涉及潜在地开发FFT的新颖实现以使用特征（包括但不限于混合精度）。该项目的预期影响是允许研究人员使用FFT，以及其他考虑使用混合精度的人了解更改代码可能产生的影响，而不必自己动手。此外，该项目的发现可能有助于指导下一代计算硬件，因为我们可能会发现特别有效/无效的方法，可以证明硬件的变化。该项目可以采取的另一个方向是使用数值分析领域开发的技术来研究混合精度对用于执行卷积的通用FFT的影响。卷积被广泛应用于快速变化的机器学习领域，有趣的是，混合精度实现被宣传为现有单精度代码的替代品。在某些情况下，它们不仅在计算机上运行得更快，而且还产生更强大的神经网络（由于正则化）。这表明，了解如何量化给定应用程序的适当数值精度是非常重要的，而不仅仅是假设精度越高越好。到目前为止，研究涉及低级编码，使用CUDA C/C++编程和优化GPU代码，以及高层次的Python + MATLAB脚本可视化和processthe输出的软件编写的C.这个项目属于福尔斯的EPSRC数字信号处理研究领域。

This research is a continuation of my undergraduate masters research project of the same title. The Fast Fourier Transform (FFT) is an extremely widely used algorithm in many areas of scientific computing. As new generations of computing hardware implement and enable the use of novel datatypes (for example bfloat16), it is important to understand the impact that using these new modes has on algorithms, of which the FFT is a very important example. So far, the project has involved implementing bfloat16 FFTs ina new version of an existing scientific code, Astro-Accelerate, whichis a GPU (graphics processing unit) accelerated code for studying radio astronomy datasets. In Astro-Accelerate, I managed to reduce the time spent doing FFTs in the search of a given file by 50%, scaling this saving up to the size of supercomputer that would be required inthe data processor of next generation telescopes, such as the SKA(Square Kilometre Array), this would translate to electricitysavings worth millions of pounds.It could also be used to reduce the upfront hardware requirement by tens of millions of pounds.The further aims and objectives of the project are to continue investigating the impact of mixed precision FFTs in wider, more general applications.Also it will be important to document the process of determining whether mixed precision FFTs are useful in a given context.This will involvepotentially developing novel implementations of the FFT to use features (including but not limited to mixed precision) on newly available hardware.The intended impact of the project is to allowresearchers using the FFT, and others considering using mixed precision to understand the impact that changing their code might have, without having to do it themselves. Additionally, the findings of this project may help to guide the next generations of computing hardware, as wemay find either particularly effective/ineffective approaches that could justify hardware changes.Another direction the project could take is using techniques developed in the field of numerical analysis to study the effect of mixed precision on generic FFTs when used to perform convolutions. Convolutions are widely used in the rapidly changing field ofmachine learning, and interestingly, mixed precision implementations are advertised as drop in replacements of existing single precision codes.In some cases they not only runfaster on a computer, but also producemore robust neural networks (due to regularisation). This demonstrates that it is extremely important to understand how to quantify an appropriate amount of numerical precision for a given application, and not just assume that more precision is better.So far, the research has involved a mixture of low level coding, using CUDA C/C++ to programand optimisethe GPUcode, as well as high level Python + MATLAB scripts to visualise and processthe output of the software written in C.This project falls within the EPSRC Digital Signal Processing research area.