Reinforcement-Learning-Based Tracking Control of Waste Water Treatment Process under Realistic System Conditions and Control Performance Requirements

Qinmin Yang, Weiwei Cao, Wenchao Meng, Jennie Si

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

The tracking control of a wastewater treatment process (WWTP) is considered. The process is highly nonlinear, with strong coupling, difficult to model mathematically, and the operation is subject to unknown disturbances. We address this multivariable tracking control problem by applying the direct heuristic dynamic programming (dHDP)-based reinforcement learning control. The control goal is to track a desired reference of the dissolved oxygen (DO) concentration of the 5th aerobic zone (SO5) and nitrate concentration of the 2nd anoxic zone (SNO2) by manipulating the oxygen transfer coefficient of the 5th aerobic zone (KLa5) and internal recycle flow rate (Qa). The dHDP aims at achieving a minimal accumulated WWTP tracking error while dealing with strong coupling between the SO5 and SNO2 and eliminating unknown disturbances in the process. The proposed dHDP approach devises an optimal control strategy entirely driven by WWTP process data as an online learning control method.We have conducted extensive and systematic simulations based on the well-known BSM1 platform of the WWTP controlled by dHDP to compare and contrast performances with other methods.

Original languageEnglish (US)
Pages (from-to)5284-5294
Number of pages11
JournalIEEE Transactions on Systems, Man, and Cybernetics: Systems
Volume52
Issue number8
DOIs
StatePublished - Aug 1 2022

Keywords

  • Action strategy approximation
  • cost function estimation
  • direct heuristic dynamic programming (direct HDP or dHDP)
  • online learning
  • tracking control
  • wastewater treatment process (WWTP)

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Reinforcement-Learning-Based Tracking Control of Waste Water Treatment Process under Realistic System Conditions and Control Performance Requirements'. Together they form a unique fingerprint.

Cite this