An Adversarial Approach to Adaptive Model Predictive Control


Linear system
Adaptive control
Multi-armed bandits
Model predictive control
State-space representation

How to Cite

Wachel, P., & Rojas, C. . (2022). An Adversarial Approach to Adaptive Model Predictive Control. Journal of Advances in Applied & Computational Mathematics, 9, 135–146.


This paper presents a novel approach to introducing adaptation in Model Predictive Control (MPC). Assuming limited a priori knowledge about the process, we consider a finite set of possible models (a dictionary), and use the theory of adversarial multi-armed bandits to develop an adaptive version of MPC called adversarial adaptive MPC (AAMPC). Under weak assumptions on the dictionary components, we then establish theoretical bounds on the performance of AAMPC and show its empirical behaviour via simulation examples.


Aggelogiannaki E, Sarimveis H. Multiobjective constrained MPC with simultaneous closed-loop identification. Int. J. Adapt. Control and Sig. Proc., 20(4):145-173, 2006.

Arora S, Hazan E, Kale S. The multiplicative weights update method: a meta-algorithm and applications. Th. of Computing, 8:121-164, 2012.

Åström KJ, Wittenmark B. Adaptive Control, 2nd Edition. Addison-Wesley, 1995.

Auer P, Cesa-Bianchi N, Freund Y, Schapire RE. The nonstochastic multi-armed bandit problem. SIAM J. Comput., 32(1):48-77, 2002.

Bubeck S, Cesa-Bianchi N. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012.

Cesa-Bianchi N, Lugosi G. Prediction, Learning, and Games. Cambridge Univ. Press, 2006.

Coulson J, Lygeros J, Dörfler F. Data-enabled predictive control: In the shallows of the DeePC. In Proc. 18th Eur. Control Conf., pages 307-312, 2019.

Ebadat A, Annergren M, Larsson CA, Rojas CR, Wahlberg B, Hjalmarsson H, Sjöberg J. Application set approximation in optimal input design for model predictive control. In Proc. 13th Eur. Control Conf., pages 744-749, Strasbourg, France, 2014.

Ebadat A, Valenzuela PE, Rojas CR, Wahlberg B. Model predictive control oriented experiment design for system identification: A graph theoretical approach. J. Proc. Control, 52:75-84, 2017.

Goodwin GC, Seron MM, De Don A. Constrained Control and Estimation: An Optimisation Approach. Springer, 2005.

Grüne L, Pannek J. Nonlinear Model Predictive Control, 2nd Edition. Springer, 2017.

Hale ET, Qin SJ. Subspace model predictive control and a case study. In Proc. Amer. Control Conf., pages 4758-4763, 2002.

Heirung TAN, Foss B, Ydstie BE. MPC-based dual control with online experiment design. J. Proc. Control, 32:64-76, 2015.

Iannelli A, Khosravi M, Smith RS. Structured exploration in the finite horizon linear quadratic dual control problem. arXiv preprint arXiv:1910.14492, 2019.

Katayama T. Subspace Methods for System Identification. Springer, 2005.

Kumar PR. An adaptive controller inspired by recent results on learning from experts. In K.J. Å ström, G.C. Goodwin, and P.R. Kumar, editors, Adapt. Control, Filtering, and Sig. Proc., pages 199-204. Springer, 1995.

Larsson CA, Annergren M, Hjalmarsson H, Rojas CR, Bombois X, Mesbah A, Modén PE. Model predictive control with integrated experiment design for output error systems. In Proc. Eur. Control Conf., pages 3790-3795, Zurich, Switzerland, 2013.

Larsson CA, Ebadat A, Rojas CR, Bombois X, Hjalmarsson H. An application-oriented approach to dual control with excitation for closed-loop identification. Eur. J. of Control, 29:1-16, 2016.

Lattimore T, Szepesvári C. Bandit algorithms. Cambridge Univ. Pr., 2020.

Ljung L. System Identification: Theory for the User, 2nd Edition. Prentice Hall, 1999.

Löfberg J. YALMIP: A toolbox for modeling and optimization in MATLAB. In Proc. CACSD Conf., pages 284-289, Taipei, Taiwan, 2004.

Lorenzen M, Cannon M, Allgöwer F. Robust mpc with recursive model update. Automatica, 103:461-471, 2019.

Marafioti G, Bitmead RR, Hovd M. Persistently exciting model predictive control. Int. J. Adapt. Cont. Sig. Proc., 28(6):536-552, 2014.

Marafioti G. Enhanced model predictive control: dual control approach and state estimation issues. Ph.d. thesis, Department of Engineering Cybernetics, Norwegian University of Science and Technology (NTNU), Norway, 2010.

Mardi N. Data-driven subspace-based model predictive control. Ph.d. thesis, RMIT University, Australia, 2010.

Mesbah A. Stochastic model predictive control with active uncertainty learning: A survey on dual control. Ann. Rev. Control, 45:107-117, 2018.

Van Overschee P, De Moor BL. Subspace identification for linear systems: Theory-Implementation-Applications. Springer, 2012.

Parsi A, Iannelli A, Smith RS. An explicit dual control approach for constrained reference tracking of uncertain linear systems. IEEE Transactions on Automatic Control, 2022.

Patwardhan RS, Gopaluni RB. A moving horizon approach to input design for closed loop identification. J. Proc. Control, 24(3):188-202, 2014.

Raginsky M, Rakhlin A, Yuksel S. Online convex programming and regularization in adaptive control. In Proc. 49th IEEE Conf. Decision and Control, pages 1957-1962, Atlanta, USA, 2010.

Rallo G, Formentin S, Rojas CR, Oomen T, Savaresi SM. Data-driven -norm estimation via expert advice. In Proc. 56th IEEE Conf. Decision and Control, pages 1560-1565, Melbourne, Australia, 2017.

Rawlings JB, Mayne DQ, Diehl MM. Model Predictive Control: Theory, Comp., and Design, 2nd Ed. Nob Hill Pub., 2017.

Shouche MS, Genceli H, Nikolaou M. Effect of on-line optimization techniques on model predictive control and identification (mpci). Computers & Chemical Engineering, 26(9):1241-1252, 2002.

Shouche M, Genceli H, Vuthandam P, Nikolaou M. Simultaneous constrained model predictive control and identification of DARX processes. Automatica, 34(12):1521-1530, 1998.

Sutton RS, Barto AG. Reinforcement Learning, An Introduction, 2nd Ed. MIT Press, 2018.

Wachel P, Sliwinski, P. Aggregative modeling of nonlinear systems. IEEE Signal Processing Letters, 22(9):1482-1486, 2015.

Wachel P. Wiener system modelling by exponentially weighted aggregation. International Journal of Control, 90(11):2480-2489, 2017.

Willems JC, Rapisarda P, Markovsky I, De Moor BLM. A note on persistency of excitation. Syst. & Contr. Lett., 54(4):325-329, 2005.

Zhu YC. System identification for process control: recent experiment and outlook. In IFAC Symposium on System Identification, pages 89-103, Newcastle, Australia, 2006. Plenary presentation, available at

Zinkevich M. Online convex programming and generalized infinitesimal gradient ascent. In Proc. Int. Conf. Mach. Learn. (ICML), pages 928-936, 2003.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2022 Pawel Wachel, Cristian R. Rojas