Reinforcement Learning-Driven Process Design: A Hydrodealkylation Example

Abstract

In this work, we present a follow-up work of reinforcement learning (RL)-driven process design using the Institute for Design of Advanced Energy Systems Process Systems Engineering (IDAES-PSE) Framework. Herein, process designs are generated as stream inlet-outlet matrices and optimized using the IDAES platform, the objective function value of which is the reward to RL agent. Deep Q-Network is employed as the RL agent including a series of convolutional neural network layers and fully connected layers to compute the actions of adding or removing any stream connections, thus creating a new process design. The process design is then informed back to the RL agent to refine its learning. The iteration continues until the maximum number of steps is reached with feasible process designs generated. To further expedite the RL search of the design space which can comprise the selection of any candidate unit(s) with arbitrary stream connections, we investigate the role of RL reward function and their impacts on exploring more complicated versus intensified process configurations. A sub-space search strategy is also developed to branch the combinatorial design space to accelerate the discovery of feasible process design solutions particularly when a large pool of candidate process units is selected by the user. The potential of the enhanced RL-assisted process design strategy is showcased via a hydrodealkylation example.