PREDICTION FOR CONTROL DELAY ON REINFORCEMENT LEARNING

Junya Saito, Kazuyuki Narisawa, Ayumi Shinohara

2012

Abstract

This paper addresses reinforcement learning problems with constant control delay, both for known case and unknown case. First, we propose an algorithm for known delay, which is a simple extension of the model-free learning algorithm introduced by (Schuitema et al., 2010). We extend it to predict current states explicitly, and empirically show that it is more efficient than existing algorithms. Next, we consider the case that the delay is unknown but its maximum value is bounded. We propose an algorithm using accuracy of prediction of states for this case. We show that the algorithm performs as efficient as the one which knows the real delay.

Paper Citation

in Harvard Style

Saito J., Narisawa K. and Shinohara A. (2012). PREDICTION FOR CONTROL DELAY ON REINFORCEMENT LEARNING . In Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: SSML, (ICAART 2012) ISBN 978-989-8425-95-9, pages 579-586. DOI: 10.5220/0003883405790586

in Bibtex Style

@conference{ssml12,
author={Junya Saito and Kazuyuki Narisawa and Ayumi Shinohara},
title={PREDICTION FOR CONTROL DELAY ON REINFORCEMENT LEARNING},
booktitle={Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: SSML, (ICAART 2012)},
year={2012},
pages={579-586},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003883405790586},
isbn={978-989-8425-95-9},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: SSML, (ICAART 2012)
TI - PREDICTION FOR CONTROL DELAY ON REINFORCEMENT LEARNING
SN - 978-989-8425-95-9
AU - Saito J.
AU - Narisawa K.
AU - Shinohara A.
PY - 2012
SP - 579
EP - 586
DO - 10.5220/0003883405790586