Membrane cleaning is a crucial maintenance task in filtration processes. However, it is typically guided by manufacturer recommendations or engineers' personal experience without well-defined standards, which can cause inconsistent process control, unnecessary actions, and increased operating costs. Therefore, optimizing operating costs regarding membrane cleaning is essential. In this study, an industrial reverse osmosis (RO) membrane filtration process, which primarily provides treated water for ultrapure water (UPW) production, was investigated to optimize its membrane cleaning strategy. Deep learning-based models were established for long sequence time-series forecasting (LSTF) and applied to investigate the optimal clean-in-place (CIP) scenarios that minimize operating costs while extending the operating time of RO membrane. The attention-based forecasting model showed greater LSTF accuracy (R2 = 0.82) than a forecasting model combining 1D convolutional neural network and long short-term memory (R2 = 0.65). Reinforcement learning simulations revealed that adopting a higher pressure threshold for CIP reduces operating costs by considering the high cost of cleaning agents and RO membrane replacement. Furthermore, the modeling process demonstrated that performing CIP twice at the highest pressure threshold in this study would decrease the RO operating cost by 16.13 % while increasing RO operating time by 139.53 % compared to the previous operation strategy. The findings of this study provide a framework for proactive membrane cleaning, preventing both excessive and delayed cleaning events, and contributing to significant operating cost reductions in filtration processes and UPW production.