Programmable embedded platforms for remote and compute intensive image processing applications

图像处理在我们的生活中发挥着越来越重要的作用，无论是电视等社会用品的众多来源，还是通过机场和城镇中心的安全摄像头的普及而日益依赖安全来保护我们的日常生活。还有一些医疗保健应用对3D(3D)图像的需求增加，例如在查看3D计算机化断层扫描以提供更智能的治疗方面。在汽车应用中，相机用于制造过程中的质量保证和使用中的情景感知。在安全应用方面，组织热衷于拥有更智能的场景视图，以突出安全风险和危险。这增加了我们处理和存储的可视信息量，并日益重视用户在接收数据的地方处理数据的能力，从而推动了更智能的图像处理。虽然已经进行了许多创新工作来推导提供这种智能的算法，但显然需要合适的、高性能、低功率的硬件来提供处理，因为在许多情况下，这些系统可能是远程的，例如具有有限互连的安全摄像机。我们可以像以前一样等待技术进步来提供更高的性能，但关于低于45 nm CMOS技术的工艺可变性的警告表明，这可能不会到来，并意味着需要更多地关注新型处理器架构。虽然已经提出了多核和特定于应用的处理器，如图形处理单元(GPU)，但收益有限。此外，在图像获取和解释以及智能算法开发方面的快速发展，还没有得到健全的软件工程原则的匹配，以开发代码并将其转换为在速度、内存和功率方面高效的硬件实现。在许多情况下，图像传感器包括简单的处理引擎，这些引擎与一些中央资源通信以进行进一步处理。对于许多医疗和安全应用，需要更智能的图像采集、多视点视频处理(将多个视点合并成更有用的、更高级的表示)和更能感知背景的采集设备，这些设备能够意识到其他摄像机的存在，从而有助于创建完整的场景。这需要我们在设计和编程这些系统的方式上进行一步的改变。目前的现场可编程门阵列技术，如Xilinx Virtex-7现场可编程门阵列，提供了巨大的性能(每秒超过6.7千兆乘法累加和高达30万亿比特/S的内存带宽)和比图形处理器更高的能效。目前，FPGA解决方案是通过将强大的知识产权(IP)核和软核聚合在一起来创建的，但由此产生的性能受到整体系统架构的限制，可编程性受到严重限制。因此，显然需要导出一种最匹配算法要求但在应用领域中的一系列算法的软件中可编程的FPGA系统体系结构。通过从一开始就考虑计算模型和编程模型，我们建议为一系列图像处理算法创建一个功能强大的平台。该提案将女王大学(伍兹)的FPGA处理器设计专业知识与赫里奥特-瓦特大学(Heriot-Watt University)的软件语言和编译器研究(Michaelson)和图像处理专业知识(Wallace)相结合。一个关键方面是确保处理器开发与软件语言和表示之间的密切交互，以便确保创建可在软件中编程的处理器体系结构配置。这项研究希望通过提供FPGA解决方案的性能和处理器解决方案的可编程性来从根本上改变前端图像处理系统的设计

Image processing is playing an increasingly important role in our lives whether this is the numerous sources of social provision e.g. TV, or the increased reliance on security to protect our everyday lives through the proliferation of security cameras in airports and town centres. There are also healthcare applications with increased need for 3-dimensional (3D) images such as in viewing 3D computerised tomography scans to provide much more intelligent treatment. In automotive applications, cameras are used for quality assurance in manufacture and situational awareness in use. In security applications, organisations are keen to have more intelligent views of scenes to highlight security risks and dangers. This has increased the amount of visual information that we process and store, and has placed increasing importance on the users' ability to process data where it is received, thus pushing for more intelligent image processing.Whilst a lot of innovative work has been done to derive the algorithms to provide this intelligence, there is a clear need for suitable, high performance, lower power hardware to provide the processing as in many cases, these systems may be remote e.g. security cameras with limited interconnection. We could wait for technology evolutions to provide the increased performance as before, but the warnings on process variability below 45-nm CMOS technology suggest that this might not be forthcoming and implies an increased focus on novel processor architectures is required. Whilst multi-core and application specific processors such as graphical processing units (GPUs) have been proposed, the gains have been limited. In addition, the rapid developments in the acquisition and interpretation of images together with intelligent algorithmic development, have not been matched by sound software engineering principles to develop and transform code into hardware implementations efficient in speed, memory and power. In many cases, image sensors comprise simple processing engines which communicate to some central resource for further processing. For a lot of medical and security applications, there is a need for more intelligent image acquisition, multi-view video processing (merging many views into a more useful, higher-level representation) and more context-aware acquisition devices which are aware of the existence of other cameras which can contribute to the creation of the full scene. This requires a step change in how we design and program these systems. Current FPGA technology such as the Xilinx Virtex-7 FPGA, offers a huge performance capability (over 6.7 Giga Multiply-Accumulate per second and up to 30 Terabits/s of memory bandwidth) and better power efficiency than GPUs. Currently FPGA solutions are created by aggregating powerful intellectual property (IP) cores together with soft cores, but the resulting performance is limited by the overall systems architecture and programmability is severely limited. Hence, there is a clear need to derive a FPGA system architecture that best matches the algorithmic requirements but that is programmable in software for a range of algorithms in the application domain. By considering the model of computation and programming model from the outset, we propose to create a highly powerful platform for a range of image processing algorithms. The proposal combines the FPGA processor design expertise in Queen's University (Woods), with the software language and compiler research (Michaelson) and image processing expertise (Wallace) at Heriot-Watt University. A key aspect is to ensure close interaction between the processor development and software languages and representation, in order to ensure the creation of a processor architecture configuration that is programmable in software. The research looks to radically alter the design of front end image processing systems by offering the performance of FPGA solutions with the programmability of processor solution