Implementation of reinforcement learning for enhanced pressure control in a 190,000-barrel crude distillation unit: The first full-scale commercial deployment

초록

We introduce the first successful full-scale implementation of a Reinforcement Learning-based Pressure Control System (RLPCS) within a Crude Distillation Unit (CDU) in the petroleum refining industry. Our approach utilizes an offline reinforcement learning (RL) technique that leverages log data from multiple operators’ manual operations to derive an optimal automated operational model. This method is safe as it does not interfere with the system currently in operation and does not incur additional operational costs. The trained model automates pressure control decisions, effectively resolving problems associated with manual control methods, such as high labor intensity and inconsistent operator control. To achieve this, we overcame various challenges related to training artificial intelligence (AI) in complex industrial environments. Our approach involved designing an appropriate Markov Decision Process (MDP) and incorporating a legacy control system for continuous data integration. We trained the Soft Actor–Critic (SAC) algorithm using offline data. However, applying the standard SAC algorithm to offline data and deploying this model directly in the online system posed substantial estimation problems. To address these, we integrated conservative loss and Return to Go into the SAC algorithm. Consequently, our model reduced operator intervention time by 84%, and variability in pressure control decreased significantly, resulting in a 12.8% reduction in cumulative errors. This demonstrates that RLPCS has reached the stage where it can be applied to permanent operations in real-world refining processes, not just a temporary demonstration. Building on this proven operational readiness, our work represents the industry's first large-scale commercial deployment of an RL-based pressure control model in a CDU. The novelty lies in the comprehensive integration of MDP formulation, model architecture, and algorithmic strategies, each carefully tailored to the complex industrial environment, thereby bridging the gap between theoretical advances and real-world application. © 2025 Elsevier Ltd

키워드