Real-time image and video processing, classification, and retrieval using hardware acceleration and extremely large datasets

在过去的几年里，我们看到被录制并放在互联网上的视频和图像的数量呈指数级增长。智能手机(黑莓、iPhone等)现在，您可以近乎实时地无缝录制、传输和共享视频。就在20年前，只有几家视频广播公司和出版商，而今天，即使不是数十亿家，也有数百万家视频广播公司和在线出版商。有了所有这些视觉内容，我们如何找到我们想要的东西？我们如何对内容进行分类？我们如何开发与基于文本的搜索引擎(Google、Bing等)一样为视觉内容带来秩序的搜索引擎？给文本网络带来了秩序？ 我们的目标是探索新的硬件和软件解决方案，使之能够使用大型数据库进行实时图像和视频搜索。我们的目标可以描述为以下两个阶段： 1.算法研究和开发(包括寻找利用ELD的方法，以更好地理解图像和视频，并提高视觉分类精度) 2.对开发的算法进行硬件加速，以便能够使用ELD对图像和视频进行准确的实时搜索 总而言之，使用当前可用的已加标签或部分加标签的图像和视频，可以开发高度准确(且计算要求高)的系统，该系统使用该信息来理解和分类大量未加标签的图像和视频。反过来，计算负载可以通过基于FPGA/VLSI的硬件加速来解决，这将使图像或视频的分类能够实时执行。

In the past few years, we have seen an exponential increase in the amount of videos and images that have been recorded and placed on the internet. Smart mobile phones (Blackberry, iPhone, etc.) now enable seamless recording, transmission, and sharing of videos in near-real-time. Whereas just two decades ago there were a few video broadcasters and publishers, today there are millions, if not billions, of video broadcasters and online publishers. With all this visual content, how do we find what we want? How to we categorize the content? How do we develop search engines that bring order to visual content just as text-based search engines (Google, Bing, etc.) brought order to the textual web? We aim to explore new hardware and software solutions that enable real-time image and video searching using large databases. Our goal can be described in the following two phases: 1. Algorithm research and development (including finding ways to utilize ELDs for better image and video understanding and improved visual classification accuracy) 2. Hardware Acceleration of the developed algorithms in order to enable accurate real-time searching of images and videos using ELDs To summarize, using currently available images and videos that are either tagged or partially tagged, it is possible to develop highly accurate (and computationally demanding) systems that use this information for understanding and classification of the vast amount of untagged images and videos. In turn, the computational load can be addressed through FPGA/VLSI-based hardware acceleration which would enable the classification of an image or video to be performed in real-time.