NONLINEAR PROGRAMMING IN APPROXIMATE DYNAMIC PROGRAMMING - Bang-bang Solutions, Stock-management and Unsmooth Penalties
Olivier Teytaud, Sylvain Gelly
2007
Abstract
Many stochastic dynamic programming tasks in continuous action-spaces are tackled through discretization. We here avoid discretization; then, approximate dynamic programming (ADP) involves (i) many learning tasks, performed here by Support Vector Machines, for Bellman-function-regression (ii) many non-linear-optimization tasks for action-selection, for which we compare many algorithms. We include discretizations of the domain as particular non-linear-programming-tools in our experiments, so that by the way we compare optimization approaches and discretization methods. We conclude that robustness is strongly required in the non-linear-optimizations in ADP, and experimental results show that (i) discretization is sometimes inefficient, but some specific discretization is very efficient for ”bang-bang” problems (ii) simple evolutionary tools out-perform quasi-random in a stable manner (iii) gradient-based techniques are much less stable (iv) for most high-dimensional ”less unsmooth” problems Covariance-Matrix-Adaptation is first ranked.
DownloadPaper Citation
in Harvard Style
Teytaud O. and Gelly S. (2007). NONLINEAR PROGRAMMING IN APPROXIMATE DYNAMIC PROGRAMMING - Bang-bang Solutions,
Stock-management and Unsmooth Penalties . In Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 978-972-8865-82-5, pages 47-54. DOI: 10.5220/0001645800470054
in Bibtex Style
@conference{icinco07,
author={Olivier Teytaud and Sylvain Gelly},
title={NONLINEAR PROGRAMMING IN APPROXIMATE DYNAMIC PROGRAMMING - Bang-bang Solutions,
Stock-management and Unsmooth Penalties},
booktitle={Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2007},
pages={47-54},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001645800470054},
isbn={978-972-8865-82-5},
}
in EndNote Style
TY - CONF
JO - Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - NONLINEAR PROGRAMMING IN APPROXIMATE DYNAMIC PROGRAMMING - Bang-bang Solutions,
Stock-management and Unsmooth Penalties
SN - 978-972-8865-82-5
AU - Teytaud O.
AU - Gelly S.
PY - 2007
SP - 47
EP - 54
DO - 10.5220/0001645800470054