Super Resolution with Deep Learning for Image Recognition

深度学习在自动识别和识别大数据集中的对象方面找到了几个应用。一个特别活跃的研究领域是它在生物医学成像中的超分辨率应用。典型的例子是通过训练机器来校正模糊图像来产生细胞结构的高分辨率图像的能力，这是通过采用图像处理协议来相对于相应的高质量图像来优化恢复的图像来实现的。学习了具有各种细胞图像和光学条件的处理策略后，可以对没有对应的高质量图像的模糊图像进行盲测。该机器能够使用与被测物体相似的训练图像，根据它先前学习到的信息，对处理策略进行分析、调整、优化和应用，以恢复高质量的图像。申请者和莱昂纳多感兴趣的是，如何将同样的方法应用于机载或陆基场景中的目标识别。这个问题的特点是有几个实际的考虑：-有不止一种类型的对象要检测。-背景地形不同。-相机(2D或3D)提供由透镜系统(取决于光学像差)和采样分辨率(由焦平面阵列的像素分辨率和间距确定)确定的有限空间分辨率。这些参数决定了相机的角度分辨率。对于3D成像，应该考虑距离分辨率。那么问题是：图像能否改进到接近衍射极限？-通常需要跟踪对象，因此输出应该是视频率。这也意味着背景在航迹过程中也是可变的。-在远距离大气湍流将以不同的方式扭曲图像。这意味着即使对于静止的物体，模糊的图像也是连续变化的。本研究的目的有两个：1。光学系统的超分辨率。确定用于对象识别的深度学习方法所需的处理协议。一种方法是采用与[1]中描述的类似的计算方案来进行细胞的超分辨率成像，但改变目标集合、标签和训练数据库以适应基于地形的目标识别问题。例如，可以使用包括在近距离拍摄的车辆的高角度分辨率图像的数据库来训练机器以识别其图像是在远距离拍摄的车辆类别中的车辆，其中更精细的细节没有被很好地分辨并且限制了形状和轮廓识别的使用。2.湍流环境下的图像识别与识别。在本应用中，该原理与文献[1]中描述的原理非常相似。训练数据库包括物体的模糊图像(通过湍流拍摄)和相应的无模糊图像(通过非常低的湍流拍摄)，用于训练机器识别模糊图像。然后，将该协议用于未包括在数据库中的对象的模糊图像上，以进行盲测试。这与[1]之间的区别在于，模糊会随着时间的推移而改变，因此几个模糊图像将与单个无模糊图像相关联。关键问题是：可以容忍什么水平的湍流，光线水平对性能的影响程度(例如黄昏和黎明)以及处理速度是多少。参考文献[11]Y.Rivenson，Z.Goroc，H.Gunaydin，Y.Zhang，H.Wang，A.Ozcan，《深度学习显微镜》，Optica，第4卷，第11期，1437-1443,2017。

Deep learning is finding several applications in the automated recognition and identification of objects in large datasets. A particularly active research area is its application to super resolution in biomedical imaging. A typical example is the ability to produce high resolution images of cell structure by training a machine to correct blurred images by adapting image processing protocols to optimise the recovered image against a corresponding high quality image. Having "learned" the processing strategy with a wide variety of cell images and optical conditions, a blind test can then be performed on a blurred image for which there is no corresponding high quality image. The machine is able to "analyse", adapt, optimise and apply a processing strategy to recover a high quality image based on what it has previously learned using training images which are similar to the object under test. The applicants and Leonardo are interested in understanding how the same methodology can be applied to object recognition in an airborne of land-based scenario. The problem is characterised by several practical considerations:- There is more than one type of object to detect.- The background terrain is varied.- The camera (2D or 3D) provides a limited spatial resolution determined by the lens system (dependant on optical aberrations) and the sampling resolution (determined by the pixel resolution and pitch of the focal plane array). These determine the angular resolution of the camera. For 3D imaging the range resolution should be considered. The question then is: can the image be improved to approach the diffraction limit?- The object will usually need to be tracked and therefore the output should be at video rates. This also implies that the background is also variable during the track.- At long range atmospheric turbulence will distort the image in a varying manner. This means that the blurred image is continuously changing even for a stationary object.PROJECT AIMThe aim of the study is two-fold:1. Super Resolution of Optical Systems. Determine the processing protocols necessary for a deep learning approach to object recognition. One approach is to adopt a similar computational scheme as that described in [1] for super resolution imaging of cells but changing the target set, tags and training database to fit a terrain-based, object recognition problem. For example, a database comprising high angular resolution images of vehicles taken at short range might be used to train a machine to recognise a vehicle in a class of vehicles whose images are taken at long range, where the finer detail is not well resolved and limits the use of shape and contour recognition. 2. Image Recognition and Identification Through Turbulence. In this application the principle is very similar to that described in [1]. A training database comprising blurred images of objects (taken through turbulence) and the corresponding blur-free image (taken through very low level turbulence) is used to train a machine to recognise a blurred image . The protocol is then used on a blurred image of an object not included in the database to conduct a blind test. The difference between this and [1] is that the blurring will change over time and therefore several blurred images will be associated with a single blur-free image. The key questions are: what level of turbulence can be tolerated, to what extent does light level affect the performance (e.g. dusk and dawn) and what is the processing speed.REFERENCES [11] Y. Rivenson, Z. Goroc, H. Gunaydin, Y. Zhang, H. Wang, and A. Ozcan, "Deep Learning Microscopy", Optica, vol. 4, no. 11, 1437-1443, 2017.