IE­EE Tran­sac­ti­ons on Pow­er Elec­tro­nics Pu­bli­ca­ti­on on Safe Re­in­for­ce­ment Lear­ning-Ba­sed Tor­que Con­trol on a Per­ma­nent Ma­gnet Syn­chro­nous Mo­tor

Max Schenke, Barnabas Haucke-Korber and Oliver Wallscheid published an article about the theory and experimental investigation of reinforcement learning-based torque control in the IEEE Journal “Transactions on Power Electronics”. A focus point of this paper was safety during the training phase, which was a key element to learn a finite-set direct torque control policy in less than ten minutes. An exemplary training phase can be seen in the accompanying video. 

Video Link: YouTube

Paper Link: IEEE Xplore


Advances in the field of reinforcement learning (RL)-based drive control allow formulation of holistic optimization goals for the data-driven training phase. The resulting controllers feature efficient drive operation without the necessity of an a priori known plant model but, so far, conduction of the corresponding training phase in real-world drive systems has been applied only sparsely due to safety concerns. This contribution targets the challenging problem of self-learning torque control for a permanent magnet synchronous motor assuming a finite control set, i.e., the direct selection of switching actions instead of a modulator-based setup. In order to allow a secure and effective online training with real-world drive systems, the RL controller is monitored by a safeguarding algorithm that prevents application of unsafe switching actions, e.g., such that result in overcurrent. The accruing amount of measurement data is handled with the use of an edge computing pipeline to outsource the RL training from the embedded control hardware. The inference of the utilized artificial neural network in hard real time is realized with the use of a reconfigurable FPGA architecture. The resulting RL-based algorithm is able to learn a torque control policy in just ten minutes which has been validated during comprehensive real-world experiments. 

Published in: IEEE Transactions on Power Electronics
Date of Publication: 09 August 2023
DOI: 10.1109/TPEL.2023.3303651
Publisher: IEEE

Schematic structure of the safe reinforcement learning torque controller
Schematic structure of the safe reinforcement learning torque controller


business-card image

Maximilian Schenke

Leistungselektronik und Elektrische Antriebstechnik (LEA)

Reinforcement Learning in der elektrischen Antriebstechnik

E-Mail schreiben +49 5251 60-5487