Q-learning-based stochastic model predictive control for green ammonia production

Abstract

Green ammonia production systems powered by intermittent renewable energy must meet periodic demand under tight unit and storage constraints. We propose Q-learning-based stochastic model predictive control, a methodology integrating a stochastic model predictive control framework with a Q-function as the terminal cost. The proposed method explicitly enforces hard constraints, effectively manages both short-term and longterm disturbances, and offers significant advantages in terms of on-line computational speed. Simulation results show that the proposed method outperforms Nonlinear Model Predictive Control, Double Deep Q-Network, and Q-learning-based Model Predictive Control baselines. The proposed method achieves the lowest total cost, minimal soft constraint penalties, and eliminates both tank overflow and ammonia demand shortfall, enabling practical, real-time operation.