4D-HSI-4-Free: Integrating Sensors & Vision Process Engineering to Deliver a Tool for revealing 3D Hyperspectral Agri-data from 2D Images

近年来，用于作物表型和胁迫分析的高光谱和多光谱成像在广域扫描方面有了显著增长，尤其是卫星和飞机。这主要是基于归一化差异植被指数、“红边”、比率和/或叶绿素荧光带的变体。尽管在这些远程监测技术中引入了更精细的农业分析，特别是随着欧空局哨兵系列卫星的发射，对阳光成像的依赖和光学分辨率的物理工程限制，以及大气吸光度和折射的变化，为空间和光谱灵敏度设定了严格的界限。近年来，由于上述物理条件的限制，许多研究小组在近距离高光谱成像（CP-HSI）方面取得了进展，以便对远程收集的图像进行地面真实处理，并获得比远程平台获得的数据更精细和区分能力更强的数据。此外，CP-HSI提供了更大数据可用性的潜力，避免了云层覆盖和其他障碍物的问题，以及卫星轨道运行时无法获取图像的问题。这些CP-HSI研究倾向于采用与地理空间成像部门使用的相同的高成本被动HSI仪器，然后利用作物内部和间作冠层产生的植物和土壤数据的变化来识别新的农业特征。近年来，曼彻斯特大学的e-Agri研究团队一直致力于将上述概念转化为低成本和小型化的工程系统，通过使用基于宽带专有硅成像探测器和窄带LED光源的有源系统取代被动HSI仪器。由此产生的主动近距离（ACP） HSI系统不仅比远程被动仪器便宜几个数量级，而且空间分辨率更高，它们还提供的频谱信噪比远远超过被动仪器的可能。令人鼓舞的是，与主动CP-HSI相关的工程挑战，尤其是需要补偿相对于传感器系统的生物标本位置的方向影响，以及防止因照明位置和led光功率的变化而损坏光谱数据。通过集成多光源光学工程以及光度立体（PS）图像重建，这些都是可以调和的。PS技术已经成为结构光的替代技术，以前所未有的空间分辨率提供表面法线和纹理信息，同时使用固有的低成本设备。PS包括在不同但已知的光线下捕获一个物体的许多图像（3+），每张图像都提供了每个像素方向的约束。CMV率先开展的工作，通过放宽先前的假设，如朗伯曲面反射、准直照明，并将该技术扩展到移动应用，使PS技术从实验室环境走向现实世界的应用。该团队将在一个可访问的工具中提供一种新型的4维成像，与2D高光谱前体相比，这只会略微增加组件成本。该项目是可以实现的，因为它将通过e-Agri UoM团队与布里斯托尔机器人实验室的机器视觉中心合作来加速，翻译他们的PS算法，以便他们可以集成到ACP-HSI传感器研究中。这些功能的合并将证明小型四维成像系统的可行性，并提供一个主要的原型系统，用于对植物、动物和水生样本进行表征。

Hyperspectral and multispectral imaging for crop phenotyping and stress analysis has seen significant growth in recent years for wide area scanning, notably from satellites and aircraft. This has been principally based on variants of the Normalised Difference Vegetation Index, 'red edge', ratios and / or chlorophyll fluorescence bands. Though more subtle agri-analysis is being introduced within these remote monitoring technologies, notably with the launch of the ESA Sentinel family of satellites, the reliance on sunlit imaging and the physical engineering limits on resolving power of the optics, alongside atmospheric variations in absorbance and refraction, set hard boundaries on the spatial and spectral sensitivity.In more recent years Close-Proximity Hyperspectral Imaging (CP-HSI) has been progressed by a number of research groups, in order to ground-truth remotely collected images as well as to gain data with greater subtlety and differentiating power than can be obtained from the remote platforms, due to the physical constraints described. Additionally, CP-HSI offers the potential for greater data availability by avoiding the issues of cloud-cover, and other obstructions, as well as the lack of access to images whilst satellites are orbiting. These CP-HSI studies have tended to adopt the same high-cost passive HSI instruments, as used by the geospatial imaging sector, and then take advantage of the variations in plant and soil data arising intra- and inter-crop canopy to identify the new agri-features.The e-Agri research team at the University of Manchester have worked over recent years on translating the above concepts into lower-cost and miniaturised engineered systems through replacing the passive HSI instrumentation with active systems, based on broadband proprietary silicon imaging detectors coupled with narrowband LED sources. The resulting Active Close-Proximity (ACP) HSI systems are not only several orders of magnitude cheaper and higher spatial resolution, than their remote passive equivalents, they also offer spectral signal-to-noise that can far exceed what is possible from the passive instruments.Encouraging as this is, there are engineering challenges associated with active CP-HSI, not least the need to compensate for the orientation affects from position of the biological specimen with respect to the sensor system as well prevent corruption of the spectral data by variations in the lighting location and optical power of the LEDs. These are reconcilable through integrating the optical engineering of multiple sources alongside Photometric Stereo (PS) image reconstruction. The PS technique has arisen as an alternate technique to structured-light for delivering surface normals and texture information at unprecedented spatial resolution, while using inherently low-cost equipment. PS involves capturing a number of images (3+) of an object under varying but known lighting, with each image providing a constraint on the orientation at each pixel. Work pioneered by the CMV has allowed the PS technique to move beyond a laboratory setting towards real-word applications, by relaxing prior assumptions, such as Lambertian surface reflection, collimated illumination and by extending the technique to moving applications.The team will deliver a new class of 4-Dimensional imaging within an accessible tool which will only marginally increases the component cost versus the 2D hyperspectral precursor. The project is achievable as it will be accelerated via the e-Agri UoM team partnering with the Centre for Machine Vision, at the Bristol Robotics Laboratory, to translate their PS algorithms so that they may be integrated within the ACP-HSI sensors research. The merger of these capabilities will prove the viability of a miniaturised 4D imaging system and deliver a proof-of-principal prototype system for characterisation against plant, animal and aquatic samples.