Telecommunications and Computer Vision Convergence Tools for Research Infrastructures (CONVERGE)

电信和计算机视觉已经发展成为独立的科学领域。这种情况预计会随着无线通信的出现而改变，无线通信的特征是视线范围，这可以受益于预测无线信道动态的可视数据。如果借助基于无线电的成像技术，计算机视觉应用也将变得更加强大。这一新的联合研究领域依赖于无线通信、计算机视觉、传感和机器学习，它具有很高的创新潜力，因为它支持的创新应用领域很大，而且欧洲有相关的专门知识。然而，只有当有足够的研究基础设施(RI)和工具可用时，才能评估这一新领域的全部潜力。Converge项目的主要目标是开发一套符合“视点对沟通和沟通对视点”座右铭的创新工具包。该工具集是全球首创的，包括视觉辅助大型智能地面、视觉辅助固定和移动基站、视觉无线电模拟器和3D环境建模器，以及用于多模式数据(包括无线电信号、视频流、射频传感和交通跟踪)的机器学习算法。此工具集将部署到7个RI中，主要与ESFRI Slice-RI保持一致，并提高其竞争力。Converge还将向科学界提供使用RIS中的工具集获得的实验和模拟数据的开放数据集，通过解决相关的6G垂直领域来满足科学和工业需求，提高相关公司的竞争力，将欧洲的影响力扩大到世界公认的RIS，使新的RIS得以创建，有助于开发新的环保工具，并帮助欧盟应对其社会挑战。

Telecommunications and computer vision have evolved as separate scientific areas. This is envisioned to change with the advent of wireless communications with radios characterised by line-of-sight ranges which could benefit from visual data to predict the wireless channel dynamics. Computer vision applications will also become more robust if helped by radio-based imaging. This new joint research field relies on wireless communications, computer vision, sensing and machine learning, and it has a high innovation potential because of the large domain of innovative applications it enables and the relevant know-how available in Europe. However, the full potential of this new area can only be evaluated if adequate Research Infrastructures (RI) and tools are available. The main objective of the CONVERGE project is the development of an innovative toolset aligned with the motto “view-to-communicate and communicate-to-view”. This toolset is a world-first and consists of vision-aided large intelligentsurfaces, vision-aided fixed and mobile base stations, a vision-radio simulator and 3D environment modeler, and machine learning algorithms for multimodal data including radio signals, video streams, RF sensing, and traffic traces. This toolset will be deployed into 7 RIs mostly aligned with the ESFRI SLICES-RI and improve their competitiveness. CONVERGE will also provide the scientific community with open datasets of experimental and simulated data obtained with the toolset in the RIs, meet scientific and industrial requirements by addressing relevant 6G verticals, enhance the competitiveness of the involved companies, extend the European influence to world-wide recognised RIs, enable the creation of new RIs, contribute to the development of new environment-friendly tools, and help European Union to address its societal challenges.