Dynamic Risk Estimation and Mapping for Reinforcement Learning-Based Safe Robot Navigation in Dense Crowds

Abstract

In this thesis, we explore the field of robot navigation, specifically the fundamental challenge of ensuring safe and efficient operation in environments shared with dense pedestrian crowds. While advances in deep learning offer promising methods to predict human trajectories, previous approaches that directly incorporate these predictions into the control policy may compromise safety. This is due to the inherent uncertainty of human motion, which can make any single predicted path unreliable. To bridge this gap, this thesis proposes a novel deep reinforcement learning (DRL) framework that enables a mobile robot to navigate safely and efficiently through spaces with both static obstacles and dense pedestrian crowds by explicitly reasoning about predictive uncertainty. The policy generates control commands based on a unique state representation: a spatio-temporal risk map that fuses the static environment with the uncertainty of pedestrian predictions. To learn how to utilize this rich representation effectively, the policy is trained end-to-end with a novel reward function inspired by Model Predictive Control (MPC), guiding it to develop proactive behaviors based on the risk map. Through a series of experiments in diverse pedestrian-dense environments, we demonstrate that our framework achieves higher success rates and efficiency compared to state-of-the-art methods.