PACMAN: Rapid identification of keypoint patch-based fiducial marker in occluded environments

Abstract

Fiducial marker systems are widely used in image-based localization methods due to their high robustness and low computational latency. However, occlusions caused by dynamic environmental factors, such as shadows and unexpected objects, significantly hinder the detection of fiducial markers, as partially visible patterns and significant image degradation often contradict the fundamental assumptions of marker detection systems. To address this challenge, we propose a keypoint-based fiducial marker and a deep-learning-based detector that jointly handle occlusion and image degradation with minimal computational latency. First, we design four distinct keypoint patches that account for occlusions and maintain essential functionalities. A fiducial marker is then constructed by assembling six identical patches under geometric constraints. Second, a widely-used interest point detector network is optimized for the proposed marker design, resulting in robust keypoint detection under various types of image deformation and degradation. A geometric consistency check is subsequently applied to map imperfect keypoint detections to 6D marker poses in the image, effectively rejecting occlusions and potential network failures. Third, neural network quantization and parallel CPU processing are applied to minimize computational latency. Our experimental results demonstrate higher detection rates then other types of single marker in occluded environments, with improvements ranging from 17% to 40%. The proposed system is also evaluated under motion blur, dimming effects, and variations in scale and rotation. Additionally, the efficient computational design enables end-to-end processing at up to 749 FPS on a desktop PC and 138 FPS on an edge device, for 640 x 480 resolution images containing a single marker. Our code is available at: