CPS: Medium: Smart Harvesting - Enhancing automated apple harvesting through apple harvesting through collaborative

果树作物常年耕作作业的自动化对于提高耕作效果、效率和作物产量是至关重要的。然而，目前的自动化技术缺乏完全的自主性，在复杂的农场环境中效率低下。为了应对这些挑战，我们的项目旨在开发一种名为智能收获的网络物理系统。该系统集成了人类智能和机器学习，将增强决策和执行能力，提高拣选效率和系统自主性。通过将智能收获集成到作物生产反馈循环中，我们将丰富该系统的曲目，并减少作物生产中的不确定性。此外，研究成果还可以帮助其他劳动密集型果园作业，如疏花和修剪，这些作业也面临劳动力短缺问题。这一多学科研究计划将为研究生和本科生提供宝贵的机会，特别是那些来自西班牙裔和土著服务机构的学生。最终的产品是苹果收获的协作式人机系统，将对农村农业社区产生显著影响。它的广泛应用将大大有助于维持美国果树产业的竞争力。该项目包括三个主要研究领域。第一个领域专注于创建一个实时更新的虚拟现实果园环境。这个环境将使用一个传感器网络和一个名为机器人操作系统的系统，该系统将人与机器连接起来。这将允许控制中心远程接收有关果园的最新3D信息。第二个领域旨在开发一个协作框架，在这个框架中，人类和机器一起有效地合作收获苹果。该框架将利用在第一个领域中创建的虚拟现实环境。人类操作员或机器学习技术将能够从远程位置协助机器人系统。它们可以帮助机器人解决苹果采摘中的挑战，比如找到无法识别的苹果，并确定取回它们的最佳方式。第三个领域涉及创建一个不断更新的曲目，将来自人类专业知识的信息和自己的机器学习经验结合在一起。它将记录来自人类操作员及其机器学习的有价值的信息，并在未来自动使用这些信息来处理类似的案件。这个曲目将改善苹果收获机器人的性能，带来更好的作物产量和质量。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Automating perennial farming operations in tree fruit crops is crucial for improving farming effectiveness, efficiency, and crop yield. However, current automation technologies lack full autonomy and are inefficient in complex farm environments. To address these challenges, our project aims to develop a cyber-physical system called Smart Harvesting. This system, integrating human intelligence and machine learning, will enhance decision-making and actuation, improving picking efficiency and system autonomy. By integrating Smart Harvesting into the crop production feedback loop, we will enrich the system's repertoire and reduce uncertainties in crop production. Additionally, the research outcomes can benefit other labor-intensive orchard operations like flower thinning and pruning, which also face labor shortage issues. This multidisciplinary research initiative will provide valuable opportunities for graduate and undergraduate students, particularly those from Hispanic and Native-serving institutions. The final product, a collaborative human-machine system for apple harvesting, will have a notable impact on rural agricultural communities. Its widespread adoption will contribute significantly to sustaining the competitiveness of the US tree fruit industry.The project consists of three main areas of research. The first area focuses on creating a virtual reality orchard environment that is updated in real-time. This environment will use a network of sensors and a system called the Robotic Operation System that connects humans with machines. This will allow the control center to receive up-to-date 3D information about the orchard remotely. The second area aims to develop a collaborative framework where humans and machines work together effectively to harvest apples. This framework will utilize the virtual reality environment created in the first area. Human operators or machine learning techniques will be able to assist the robot system from a remote location. They can help the robot address challenges in apple picking, such as finding unidentifiable apples and determining the best way to retrieve them. The third area involves creating a constantly updating repertoire that incorporates information from human expertise and its own machine learning experience. It will record valuable information from human operators and its machine learning and use it to handle similar cases in the future autonomously. This repertoire will improve the performance of the apple harvesting robot, leading to better crop yield and quality.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.