EAGER: Collaborative Research: Spatiotemporal transfer learning for enabling cross-country and cross-hemisphere in-season crop mapping

农作物生产是美国的主要产业。2021年，美国粮食出口占国际粮食贸易的份额超过40%。数以百万计的美国农民依靠国际市场生活和繁荣。然而，美国的粮食出口不仅面临着来自其他出口国的激烈竞争，还受到进口国粮食产量的影响。为了获得竞争优势，利益相关者需要尽早知道在世界各地的生长季节，每种作物在哪里种植，种植了多少英亩，这样就可以估计产量，评估生产和需求平衡，预测粮食价格。这需要生成美国和其他国家的当季作物地图。生成应季作物图的经典方法需要大量经过验证的作物信息（即地面事实）来训练对应季卫星遥感图像进行分类的算法。然而，在国外很难甚至不可能获得真实情况，特别是在赛季初期。本研究建议开发一种时空可转移的机器学习算法，该算法将使用美国数据进行训练，并应用于国外的季节性卫星遥感图像，以创建各国的季节性作物地图。该项目的成功将使国外的季节性作物制图成为可能。该项目将显著提高美国农业的竞争力和盈利能力，增加世界粮食安全，并可能为美国农民带来数十亿美元的经济效益。结合地面真值标记的卫星遥感是目前农作物制图的常用方法。然而，它存在着两个问题：1)国外缺乏地面真实资料；2)训练分类器的时空不可转移性。本研究将设计时空可迁移的学习算法和时间学习策略，最大限度地将标签数据和模型从美国迁移到国外。该方法采用对抗训练和对比学习。通过这种二人博弈，特征提取器产生域不变特征。在这种域不变表示上训练的分类器可以将其模型转移到一个新的域，因为目标特征与训练期间看到的特征相匹配，从而弥合了时间和位置之间的差距。美国训练的算法将在加拿大和巴西进行测试，以证明其跨国家和跨半球的可移植性。科学上，该项目将通过提供一种新的迁移学习方法，推进跨空间和时间的无监督域适应中的机器学习，并提供从时间序列遥感图像中获得时空不变特征的新方法，从而推进季节性作物制图中的土地覆盖科学。从社会经济角度来看，该项目将提高美国农业的竞争力和盈利能力，增加世界粮食安全，并可能为美国农民带来数十亿美元的收益。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Crop production is a major industry in the United States (U.S.). In 2021, the U.S. grain export accounted for over 40% share of international grain trade. Millions of U.S. farmers depend on international market for living and prosperity. However, the U.S. grain export is not only facing tough competition from other export countries, but also impacted by grain yield in import countries. In order to gain the competitive edge, stakeholders need to know as early as possible where and how many acres each type of crops that have been planted in a growing season around the world so that yield can be estimated, production and demand balance can be assessed, and grain prices can be predicted. This requires generating in-season crop maps of both U.S. and foreign countries. The classic method to generate in-season crop maps needs a large amount of verified information on crops (i.e., ground truths) to train algorithms for classifying in-season satellite remote sensing images. However, it is difficult or even impossible to obtain ground truths in foreign countries, particularly in early season. This study proposes to develop a spatiotemporally transferable machine-learning algorithm which will be trained with U.S. data and applied to in-season satellite remote sensing images of foreign countries for creating the in-season crop maps of the countries. Success of this project will make the in-season crop mapping of foreign countries possible. The project will significantly enhance the competitiveness and profitability of U.S. agriculture, increase the food security of the world, and potentially bring billions-of-dollars economic benefits to U.S. farmers.Satellite remote sensing with ground truth tagging is the current practice for crop mapping. However, it suffers from two problems: 1) Unavailability of ground truth in foreign countries; 2) Spatiotemporal intransferability of trained classifiers. This study will design spatiotemporally transferable learning algorithm and temporal learning strategy that would maximally transfer label data and models from U.S. to foreign countries. The proposed method utilizes adversarial training and contrastive learning. Through this two-player game, the feature extractor produces domain-invariant features. A classifier trained on this domain-invariant representation can transfer its model to a new domain because the target features match those seen during training, thus bridging the gap between times and locations. The U.S. trained algorithm will be tested in Canada and Brazil to demonstrate its cross-country and cross-hemisphere transferability. Scientifically this project will advance landcover science in in-season crop mapping by offering a novel method of transfer learning, advance machine learning in unsupervised domain adaptation across both space and time, and offer new methods to derive spatiotemporally invariant features from time-series remote sensing images. Socioeconomically this project will enhance competitiveness and profitability of U.S. agriculture, increase food security of the world, and potentially bring billions-of-dollars benefits to U.S. farmers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.