Programmable embedded platforms for remote and compute intensive image processing applications

适用于远程和计算密集型图像处理应用的可编程嵌入式平台

• 批准号: EP/K009583/1
• 负责人: Roger Woods
• 金额: $ 80.02万
• 依托单位: Queen's University Belfast
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2013
• 资助国家: 英国
• 起止时间: 2013 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FK009583%2F1
• 关键词: Programmable; embedded; platforms; remote; compute

Image processing is playing an increasingly important role in our lives whether this is the numerous sources of social provision e.g. TV, or the increased reliance on security to protect our everyday lives through the proliferation of security cameras in airports and town centres. There are also healthcare applications with increased need for 3-dimensional (3D) images such as in viewing 3D computerised tomography scans to provide much more intelligent treatment. In automotive applications, cameras are used for quality assurance in manufacture and situational awareness in use. In security applications, organisations are keen to have more intelligent views of scenes to highlight security risks and dangers. This has increased the amount of visual information that we process and store, and has placed increasing importance on the users' ability to process data where it is received, thus pushing for more intelligent image processing.Whilst a lot of innovative work has been done to derive the algorithms to provide this intelligence, there is a clear need for suitable, high performance, lower power hardware to provide the processing as in many cases, these systems may be remote e.g. security cameras with limited interconnection. We could wait for technology evolutions to provide the increased performance as before, but the warnings on process variability below 45-nm CMOS technology suggest that this might not be forthcoming and implies an increased focus on novel processor architectures is required. Whilst multi-core and application specific processors such as graphical processing units (GPUs) have been proposed, the gains have been limited. In addition, the rapid developments in the acquisition and interpretation of images together with intelligent algorithmic development, have not been matched by sound software engineering principles to develop and transform code into hardware implementations efficient in speed, memory and power. In many cases, image sensors comprise simple processing engines which communicate to some central resource for further processing. For a lot of medical and security applications, there is a need for more intelligent image acquisition, multi-view video processing (merging many views into a more useful, higher-level representation) and more context-aware acquisition devices which are aware of the existence of other cameras which can contribute to the creation of the full scene. This requires a step change in how we design and program these systems. Current FPGA technology such as the Xilinx Virtex-7 FPGA, offers a huge performance capability (over 6.7 Giga Multiply-Accumulate per second and up to 30 Terabits/s of memory bandwidth) and better power efficiency than GPUs. Currently FPGA solutions are created by aggregating powerful intellectual property (IP) cores together with soft cores, but the resulting performance is limited by the overall systems architecture and programmability is severely limited. Hence, there is a clear need to derive a FPGA system architecture that best matches the algorithmic requirements but that is programmable in software for a range of algorithms in the application domain. By considering the model of computation and programming model from the outset, we propose to create a highly powerful platform for a range of image processing algorithms. The proposal combines the FPGA processor design expertise in Queen's University (Woods), with the software language and compiler research (Michaelson) and image processing expertise (Wallace) at Heriot-Watt University. A key aspect is to ensure close interaction between the processor development and software languages and representation, in order to ensure the creation of a processor architecture configuration that is programmable in software. The research looks to radically alter the design of front end image processing systems by offering the performance of FPGA solutions with the programmability of processor solution

图像处理在我们的生活中发挥着越来越重要的作用，无论是电视等众多的社会供应来源，还是通过机场和城镇中心安全摄像头的扩散来保护我们的日常生活。还存在对三维（3D）图像的需求增加的医疗保健应用，诸如在观看3D计算机断层扫描时，以提供更智能的治疗。在汽车应用中，摄像头用于制造中的质量保证和使用中的情景感知。在安全应用中，组织渴望拥有更智能的场景视图，以突出安全风险和危险。这增加了我们处理和存储的视觉信息的量，并且越来越重视用户在接收到数据的地方处理数据的能力，从而推动了更智能的图像处理。虽然已经做了很多创新工作来导出提供这种智能的算法，但显然需要合适的高性能，低功率硬件来提供处理，因为在许多情况下，这些系统可以是远程的，例如具有有限互连的安全摄像机。我们可以等待技术的发展，以提供更高的性能，但低于45纳米CMOS技术的工艺可变性的警告表明，这可能不会到来，这意味着需要更多的关注新的处理器架构。虽然已经提出了诸如图形处理单元（GPU）之类的多核和专用处理器，但是收益有限。此外，图像采集和解释的快速发展以及智能算法的发展，还没有与健全的软件工程原理相匹配，以开发和转换代码为速度，内存和功率高效的硬件实现。在许多情况下，图像传感器包括简单的处理引擎，其与某些中央资源进行通信以进行进一步处理。对于许多医疗和安全应用，需要更智能的图像采集，多视图视频处理（将许多视图合并为更有用的更高级别的表示）以及更多的上下文感知采集设备，这些设备知道其他相机的存在，这些相机可以有助于创建完整的场景。这需要我们在设计和编程这些系统的方式上进行一步改变。目前的FPGA技术，如Xilinx Virtex-7 FPGA，提供了巨大的性能能力（每秒超过6.7千兆位的多线程累积和高达30 TB/s的内存带宽）和比GPU更好的能效。目前，FPGA解决方案是通过将强大的知识产权（IP）内核与软核聚合在一起来创建的，但由此产生的性能受到整体系统架构的限制，并且可编程性受到严重限制。因此，有一个明确的需要，以获得最佳匹配的算法要求，但在软件中可编程的一系列算法在应用领域的FPGA系统架构。通过从一开始就考虑计算模型和编程模型，我们建议为一系列图像处理算法创建一个非常强大的平台。该提案结合了女王大学（伍兹）的FPGA处理器设计专业知识，以及赫瑞瓦特大学的软件语言和编译器研究（迈克尔森）和图像处理专业知识（华莱士）。一个关键方面是确保处理器开发与软件语言和表示之间的密切交互，以确保创建可在软件中编程的处理器架构配置。该研究旨在通过提供FPGA解决方案的性能和处理器解决方案的可编程性，从根本上改变前端图像处理系统的设计

项目成果

Handbook of Signal Processing Systems

• DOI: 10.1007/978-1-4614-6859-2
• 发表时间: 2013
• 作者: S. Bhattacharyya;Rainer Leupers;J. Takala;Springer New;York Heidelberg;Dordrecht London;A. Boev;R. Bregović;A. Gotchev;Yu-Han Chen;Liang-Gee Chen;J. Collin;Pavel Davidson;M. Kirkko-Jaakkola;Helena Lepp¨akoski;Raphael Ducasse;M. Schaar;A. Gregerson;Michael J. Schulte;Katherine Compton;Markku Juntti;M. Renfors;M. Valkama;Miroslav Kneˇzevi´c;L. Batina;E. D. Mulder;J. Fan;Benedikt Gierlichs;Yong Ki Lee;Roel Maes;I. Verbauwhede;William S. Levine;M. Mattavelli;Micka¨el Raulet;J¨orn W. Janneck;Naresh Shanbhag;Andrew C. Singer;Hyeon-Min Bae;Raj Shekhar;V. Walimbe;W. Plishker;J. Suhonen;M. Kohvakka;Ville Kaseva;Timo D. H¨am¨al¨ainen;Marko H¨annik¨ainen;A. Veen;S. Wijnholds;Marilyn Wolf;J. Schlessman;C. Banz;Holger Blume;P. Pirsch;Luigi Carro;M. B. Rutzig;B. D. Sutter;Praveen Raghavan;A. Lambrechts;Oscar Gustafsson;L. Wanhammar;Sangjin Hong;Seong-Jun Oh;Dake Liu;Jian Wang;John McAllister;Yang Sun;K. Amiri;Michael Brogioli;Joseph R. Cavallaro;Olli Vainio;Tong Zhang;Yangyang Pan;Yiran Li;Iuliana Bacivarov;Wolfgang Haid;Kai Huang;Lothar Thiele;Ed Deprettere;B. Theelen;F. Brandner;N. Horspool;A. Krall;J. Falk;C. Haubelt;Christian Zebelein;J¨urgen Teich;M. Geilen;T. Basten;Soonhoi Ha;Hyunok Oh;Yu Hen Hu;Sun-Yuan Kung;J. Keinert;Christoph W. Kessler;Weihua Sheng;J. Castrillón;Keshab K. Parhi;Yanni Chen;Sven Verdoolaege;Roger Woods;Daejeon Kaist;South Korea;K. Leuven;Belgium Ibbt;Yong Ki;Bryan E. Olivier;Kapeldreef Heverlee;Belgium

Applied Reconfigurable Computing

应用可重构计算

• DOI: 10.1007/978-3-319-30481-6_7
• 发表时间: 2016
• 作者: Kelly C

FPGA-Based Processor Acceleration for Image Processing Applications.

• DOI: 10.3390/jimaging5010016
• 发表时间: 2019-01-13
• 期刊: Journal of imaging
• 影响因子: 3.2
• 作者: Siddiqui F;Amiri S;Minhas UI;Deng T;Woods R;Rafferty K;Crookes D
• 通讯作者: Crookes D

IPPro: FPGA based image processing processor

IPPro：基于FPGA的图像处理处理器

• DOI: 10.1109/sips.2014.6986057
• 发表时间: 2014
• 作者: Siddiqui F

A Soft Coprocessor Approach for Developing Image and Video Processing Applications on FPGAs.

• DOI: 10.3390/jimaging8020042
• 发表时间: 2022-02-11
• 期刊: Journal of imaging
• 影响因子: 3.2
• 作者: Deng T;Crookes D;Woods R;Siddiqui F
• 通讯作者: Siddiqui F