A real-time vehicular vision system to seamlessly see-through cars

Abstract

Overtaking accidents typically occur when the rear car intends to overtake the front car with limited visibility. This lack of visual information is often attributed to the occlusion caused by the front vehicle. Indeed, in many situations the front car hides the presence of obstacles, such as pedestrians or other cars. Nowadays, the generalization of digital camera embedded automotives represents a great potential to reduce the number of these deadly accidents. Thus, we propose a novel collaborative cars method which allows a driver to literally see through the front vehicle to assist in overtaking manoeuvres. In the studied scenario, both cars are equipped with cameras (stereo and monocular cameras for the front and the rear cars, respectively) and share data through an appropriated wireless communication system. Our method generates a seamless transparency effect from the rear car viewpoint using tri-focal tensor image synthesis where the poses of the cameras are estimated using a marker-based pose estimation. In this article, we present an efficient framework designed to reduce the quantity of information to be transferred between the vehicles and to achieve real-time performances (15 fps). Furthermore, our system is assessed through multiple experiments in controlled and real conditions.