Cross-view Transformers for real-time Map-view Semantic Segmentation
Cross-view Transformers for real-time Map-view Semantic Segmentation
复制标题
DOI：
10.1109/cvpr52688.2022.01339
复制
发表时间：
2022-05
期刊：
影响因子：
--
通讯作者：
Brady Zhou;Philipp Krahenbuhl
中科院分区：
文献类型：
--
作者：
Brady Zhou;Philipp Krahenbuhl
我们提出了一种基于注意力的交叉视图转换模型，用于从多个摄像机中进行地图视图语义分割。我们的架构使用相机感知的跨视图注意机制，隐式地学习从单个相机视图映射到规范地图视图表示的映射。每个相机都使用依赖于其内在和外在校准的位置嵌入。这些嵌入允许转换器学习跨不同视图的映射，而无需显式地对其进行几何建模。该架构由每个视图的卷积图像编码器和跨视图转换层组成，用于推断地图-视图语义分割。我们的模型简单，易于并行化，并且是实时运行的。所提出的架构在nuScenes数据集上执行最先进的性能，推理速度提高了4倍。代码可从https://github.com/bradyz/cross_view_transformers获得。
We present cross-view transformers, an efficient attention-based model for map-view semantic segmentation from multiple cameras. Our architecture implicitly learns a mapping from individual camera views into a canonical map-view representation using a camera-aware cross-view attention mechanism. Each camera uses positional embeddings that depend on its intrinsic and extrinsic calibration. These embeddings allow a transformer to learn the mapping across different views without ever explicitly modeling it geometrically. The architecture consists of a convolutional image encoder for each view and cross-view transformer layers to infer a map-view semantic segmentation. Our model is simple, easily parallelizable, and runs in realtime. The presented architecture performs at state-of-the-art on the nuScenes dataset, with 4x faster inference speeds. Code is available at https://github.com/bradyz/cross_view_transformers.