A Camera-based 3D Perception Accelerator for Energy-Efficient Bird’s-Eye-View Processing

Abstract

Multi-camera systems (MCS) increasingly rely on bird’s-eye-view (BEV) representations to perform robust 3D scene understanding. However, the view transformation from multi-camera images into BEV space involves extensive spatial searching and numerous irregular memory access, resulting in considerable computational overhead on existing edge platforms. In particular, the BEV pooling stage alone accounts for 68.3 ms of latency. Moreover, as the transformed BEV features exhibit 69.1% input activation sparsity, redundant computations are conducted during subsequent semantic segmentation. Within this context, a BEV semantic segmentation accelerator is proposed for real-time implementation of BEV perception on edge hardware. The proposed accelerator has two main features: 1) block-based hierarchical decomposition of BEV pooling for parallel BEV pooling on reduced range by partitioning the pooling space; 2) Channel pruning for coarse-grained zero skipping and convolution core architecture for fine-grained zero skipping to leverage the high sparsity of BEV features during segmentation stage. The accelerator is implemented with 28nm technology, and it achieves 23.1 frames- per-second of real-time throughput for BEV semantic segmentation on two representative MCS datasets. Furthermore, 167.4× higher energy efficiency is achieved over the existing edge computing platforms.