Vehicle Pose Estimation using Language Supervision

Abstract

Accurate understanding of traffic scenes is crucial for autonomous driving, requiring precise vehicle pose and shape estimation. This paper introduces an approach that utilizes language-guided supervision to enhance vehicle keypoint detection and extend it to robust 6DoF pose estimation of vehicles in the ApolloCar3D dataset. By leveraging the linguistic capabilities of CLIP-pretrained image and text encoders, our method optimizes keypoint detection through specially crafted prompts tailored to individual keypoints. This approach establishes a connection between keypoint and prompt embeddings, enhancing keypoint detection precision and ensuring better pose estimation. Utilizing precise 2D keypoints alongside predefined 3D models, we employ the EPnP algorithm for 6DoF pose estimation and detailed 3D scene reconstruction. Our approach not only elevates the accuracy of keypoint detection but also significantly enhances 6DoF pose estimation performance, as supported by rigorous evaluations using precision metrics on the ApolloCar3D benchmark. This approach integrates linguistic insights and visual analytics to advance vehicle pose estimation, achieving significant improvements over existing methods.