STOCHASTIC CONTROL STRATEGIES AND ADAPTIVE CRITIC METHODS

Randa Herzallah, David Lowe

2008

Abstract

Adaptive critic methods have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, nonlinear and nonstationary environments. In this study, a novel probabilistic dual heuristic programming (DHP) based adaptive critic controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) adaptive critic method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterized by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the critic network is then calculated and shown to be equal to the analytically derived correct value.

Paper Citation

in Harvard Style

Herzallah R. and Lowe D. (2008). STOCHASTIC CONTROL STRATEGIES AND ADAPTIVE CRITIC METHODS . In Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-989-8111-30-2, pages 281-288. DOI: 10.5220/0001481902810288

in Bibtex Style

@conference{icinco08,
author={Randa Herzallah and David Lowe},
title={STOCHASTIC CONTROL STRATEGIES AND ADAPTIVE CRITIC METHODS},
booktitle={Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2008},
pages={281-288},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001481902810288},
isbn={978-989-8111-30-2},
}

in EndNote Style

TY - CONF
JO - Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - STOCHASTIC CONTROL STRATEGIES AND ADAPTIVE CRITIC METHODS
SN - 978-989-8111-30-2
AU - Herzallah R.
AU - Lowe D.
PY - 2008
SP - 281
EP - 288
DO - 10.5220/0001481902810288