Research ArticleOpen AccessGoogle Scholar indexed
On the Connection between the Hamilton-Jacobi-Bellman and the Fokker-Planck Control Frameworks
Dipartimento di Matematica, Università degli Studi di Salerno, Fisciano, Italy
Institut für Mathematik, Universit?t Würzburg, Würzburg, Germany
école Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia
- 1 Dipartimento di Matematica, Università degli Studi di Salerno, Fisciano, Italy
- 2 Institut für Mathematik, Universit?t Würzburg, Würzburg, Germany
- 3 école Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
- 4 King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia
Applied Mathematics·Volume 05 (2014)·Pages 2476–2484·Published 29 August 2014·DOI10.4236/am.2014.516239
Copy link · social · email
Abstract
In the framework of stochastic processes, the connection between the dynamic programming scheme given by the Hamilton-Jacobi-Bellman equation and a recently proposed control approach based on the Fokker-Planck equation is discussed. Under appropriate assumptions it is shown that the two strategies are equivalent in the case of expected cost functionals, while the Fokker-Planck formalism allows considering a larger classof objectives. To illustratethe connection between the two control strategies, the cases of an Itō stochastic process and of a piecewise-deterministic process are considered.
KeywordsHamilton-Jacobi-Bellman EquationFokker-Planck EquationOptimal Control TheoryStochastic Differential EquationsHybrid Systems
- Ghman, I.I. and Skorohod, A.V. (1972) Stochastic Differential Equations. Springer-Verlag, New York. http://dx.doi.org/10.1007/978-3-642-88264-7
- Bertsekas, D. (2007) Dynamic Programming and Optimal Control, Vols. I and II. Athena Scientific.
- Fleming, W.H. and Soner, H.M. (2006) Controlled Markov Processes and Viscosity Solutions. Springer-Verlag, Berlin.
- Annunziato, M. and Borzì, A. (2010) Optimal Control of Probability Density Func-tions of Stochastic Processes. Mathematical Modelling and Analysis, 15, 393-407. http://dx.doi.org/10.3846/1392-6292.2010.15.393-407
- Annunziato, M. and Borzì, A. (2013) A Fokker-Planck Control Framework for Multidimensional Stochastic Processes. Journal of Computational and Applied Mathematics, 237, 487-507. http://dx.doi.org/10.1016/j.cam.2012.06.019
- Cox, D.R. and Miller, H.D. (2001) The Theory of Stochastic Processes. Chapman & Hall/CRC, London.
- Fleming, W.H. and Rishel, R.W. (1975) Deterministic and Stochastic Optimal Control. Springer-Verlag, Berlin. http://dx.doi.org/10.1007/978-1-4612-6380-7
- Borkar, V.S. (2005) Controlled Diffusion Processes. Probability Surveys, 2, 213-244. http://dx.doi.org/10.1214/154957805100000131
- Falcone, M. (2008) Numerical Solution of Dynamic Programming Equations. In: Bardi, M. and Dolcetta, I.C., Eds., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Birkhauser.
- Kushner, H.J. (1990) Numerical Methods for Stochastic Control Problems in Continuous Time. SIAM Journal on Control and Optimization, 28, 999-1048. http://dx.doi.org/10.1137/0328056
- Forbes, M.G., Guay, M. and Forbes, J.F. (2004) Control Design for First-Order Processes: Shaping the Probability Density of the Process State. Journal of Process Control, 14, 399-410. http://dx.doi.org/10.1016/j.jprocont.2003.07.002
- Jumarie, G. (1992) Tracking Control of Nonlinear Stochastic Systems by Using Path Cross-Entropy and Fokker-Planck Equation. International Journal of Systems Science, 23, 1101-1114. http://dx.doi.org/10.1080/00207729208949368
- Kárny, M. (1996) Towards Fully Probabilistic Control Design. Automatica, 32, 1719-1722. http://dx.doi.org/10.1016/S0005-1098(96)80009-4
- Wang, H. (1999) Robust Control of the Output Probability Density Functions for Multivariable Stochastic Systems with Guaranteed Stability. IEEE Transactions on Automatic Control, 44, 2103-2107. http://dx.doi.org/10.1109/9.802925