Enhanced Reward Function Design for Source Term Estimation Based on Deep Reinforcement Learning

Abstract

This study investigates the design of reward functions for deep reinforcement learning-based source term estimation (STE). Estimating the properties of unknown hazardous gas leakage using a mobile sensor, known as STE problems, is challenging due to environmental turbulence and sensor noise. To address this issue, the particle filter is employed to estimate the source term under noisy sensor measurements, and the deep Q-network is used to find the optimal source search policy. In deep reinforcement learning, selecting an appropriate reward function is crucial as it directly impacts the learning performance. Specifically, this paper first reviews existing reward functions based on penalty, distance, concentration, and entropy metrics. To overcome the limitations of existing rewards, we combine their strengths and propose new reward functions such as the Gaussian mixture model (GMM) variance-based reward and the GMM information gain-based reward. To validate the robustness of the proposed approach, simulations are conducted in two types of environments: basic and turbulent, by adjusting the parameters of the noise condition. The simulation results demonstrate that the proposed reward functions outperform existing ones and are particularly robust in noisy environments.