CAREER: From Underground to Space: An AI Infrastructure for Multiscale 3D Crop Modeling and Assessment

作物的性状和三维结构(植物的形状和结构，包括地上和地下部分)是主要影响作物生长和产量的属性，并为植物表型(植物性状的特征评估)提供关键证据。作物产量预测可以通过使用作物传感方法评估3D植物结构来进行。然而，不同尺度上的作物传感结果通常被孤立地分析，这忽略了本质上的联系。此外，虽然根系在植物功能中起着核心作用，但由于难以获得根，目前的方法主要是基于地上作物结构来评估作物。目前的方法使用卫星进行遥感，使用无人机进行本地遥感，从而能够在不同的尺度上对作物进行评估；然而，很难有效地整合这些观测，信息流也是令人敬畏的。该项目的总体目标是开发一种新型的人工智能基础设施，以整合这些观测结果，在多个尺度上对3D作物结构进行建模和评估，并增强地下传感能力。使用这一基础设施，可以在个人、农场和卫星尺度上准确地估计3D作物结构，从而促进作物评估和产量预测。该项目极大地增强和加快了种植者和农学家评估地上和地下部分关键作物田地结构变化的能力，从而实现了大规模作物管理。该项目还通过将地上和地下作物结构的多尺度3D模型应用于沉浸式教育方法(虚拟现实(VR)、增强现实(AR)和在线学习)，使从高中到博士水平的学生受益，这些方法非常适合解决远程学习的挑战，特别是对于需要实地研究的农业等学科。多尺度遥感系统还能够估计3D景观结构和大尺度作物结构，并可用于其他领域，如北极海冰建模、林业和气候变化研究。该项目旨在将植物地上和地下的结构表型联系起来，并将现场测量与卫星遥感数据联系起来，从而实现基于对地上植物生长的非破坏性作物根系传感和根系状况估计，同时使卫星图像能够评估这些因素，以提供更多本地和详细的信息。该项目建立了单株、农田和卫星区域的3D作物遥感方法，为作物评估和产量预测提供多尺度的作物结构证据。该项目还开发了一种新型的人工智能神经网络，基于对地上植物结构的感知来感知根结构和预测特征。该项目研究基于卫星的3D传感和无损地下根部传感的方法。探索了新型人工智能基础设施，以解决计算机视觉和遥感、多尺度传感、3D结构预测和时空4D推理的有效集成中的关键问题。这种办法可以根据有科学依据的多尺度三维评估工作流程，降低卫星和现场图像评估的业务采用上限。除了对农业专业人员的基本和实际影响外，该项目还探索了计算机视觉和遥感领域的新型人工智能解决方案。作物结构是高度多样化、复杂和不断变化的现象。因此，农业为研究新型人工智能方法提供了一个理想的研究领域。这项研究通过1)极大地提高安装在不同设备上的传感器的各种遥感模式的融合效率，2)通过连接以多个尺度表示的传感输出来显著增强学习能力，3)支持跨不同领域的物体的3D结构预测，以及4)提供基于4D时空神经网络的未来状态预测。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

A crop's traits and 3D structure (the shape and architecture of plants, including both above- and below-ground parts) are the attributes that chiefly influence crop growth and yield and provide critical evidence for plant phenotyping (the characterization assessment of plant traits). Crop yield predictions can be made by assessing 3D plant structures using crop sensing methods. However, crop sensing results at different scales are usually analyzed in isolation, which overlooks essential connections. Moreover, while root systems play a central role in plant functions, current methods mainly assess crops based on above-ground crop structure due to the difficulty of accessing roots. Current methods use satellites for remote sensing and drones for local sensing, enabling crop assessment at varying scales; however, it is difficult to integrate these observations effectively, and the information stream is formidable. The overarching objective of this project is to develop a novel AI infrastructure to integrate these observations to model and assess 3D crop structures at multiple scales and enhance below-ground sensing capabilities. Using this infrastructure, 3D crop structures can be estimated accurately at the individual, farm, and satellite scales, facilitating crop assessment and yield prediction. The project dramatically enhances and accelerates the ability of growers and agronomists to assess critical crop field structural variation for both above- and below-ground components, enabling large-scale crop management. This project also benefits students, from the high school to the Ph.D. level, by applying multi-scale 3D models of above- and below-ground crop structures to immersive education methods (Virtual Reality (VR), Augmented Reality (AR), and online learning), which are well-suited to solving the challenges of distance learning, especially for subjects like agriculture requiring field study. The multi-scale sensing system is also capable of estimating 3D landscape structures and large-scale crop structures and can be utilized in other areas, such as Arctic Sea ice modeling, forestry, and climate change studies.This project aims to connect a plant’s structural phenotypes below- and above- ground and link in-situ measurements to satellite sensing data, thus enabling non-destructive crop root sensing and root-system status estimation based on observation of plant growth above-ground while at the same time empowering satellite images to assess these factors to furnish more local and detailed information. This project establishes a method for 3D crop sensing of individual plants, crop fields, and satellite regions to provide multi-scale crop structural evidence for crop assessment and yield prediction. This project also develops a novel AI neural network to sense root structures and predict traits based on sensing above-ground plant structures. This project investigates methods for satellite-based 3D sensing and nondestructive below-ground root sensing. Novel AI infrastructures are explored to address critical issues in computer vision and remote sensing, efficient integration of multi-scale sensing, 3D structure prediction, and spatial-temporal 4D inference. Such an approach can lower the ceiling for operational adoption of satellite and in-situ imagery assessments, based on a scientifically underpinned, multi-scale, 3D assessment workflow. In addition to its essential and practical implications for agriculture professionals, this project also explores novel AI solutions within computer vision and remote sensing. Crop structures are highly diverse, complicated, and changing phenomena. Therefore, agriculture presents an ideal research domain for investigating novel AI methods. This research advances AI by 1) largely improving the fusion effectiveness of various remote sensing modalities from sensors mounted on different devices, 2) significantly enhancing the learning capability by connecting sensing outputs expressed in multiple scales, 3) enabling 3D structure prediction for objects across different domains, and 4) providing future status prediction based on 4D spatial-temporal neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.