STATE AGGREGATION FOR REINFORCEMENT LEARNING USING NEUROEVOLUTION

Robert Wright, Nathaniel Gemelli

2009

Abstract

In this paper, we present a new machine learning algorithm, RL-SANE, which uses a combination of neuroevolution (NE) and traditional reinforcement learning (RL) techniques to improve learning performace. RL-SANE is an innovative combination of the neuroevolutionary algorithm NEAT(Stanley, 2004) and the RL algorithm Sarsa(l)(Sutton and Barto, 1998). It uses the special ability of NEAT to generate and train customized neural networks that provide a means for reducing the size of the state space through state aggregation. Reducing the size of the state space through aggregation enables Sarsa(l) to be applied to much more difficult problems than standard tabular based approaches. Previous similar work in this area, such as in Whiteson and Stone (Whiteson and Stone, 2006) and Stanley and Miikkulainen (Stanley and Miikkulainen, 2001), have shown positive and promising results. This paper gives a brief overview of neuroevolutionary methods, introduces the RL-SANE algorithm, presents a comparative analysis of RL-SANE to other neuroevolutionary algorithms, and concludes with a discussion of enhancements that need to be made to RL-SANE.

Paper Citation

in Harvard Style

Wright R. and Gemelli N. (2009). STATE AGGREGATION FOR REINFORCEMENT LEARNING USING NEUROEVOLUTION . In Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-8111-66-1, pages 45-52. DOI: 10.5220/0001658800450052

in Bibtex Style

@conference{icaart09,
author={Robert Wright and Nathaniel Gemelli},
title={STATE AGGREGATION FOR REINFORCEMENT LEARNING USING NEUROEVOLUTION},
booktitle={Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2009},
pages={45-52},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001658800450052},
isbn={978-989-8111-66-1},
}

in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - STATE AGGREGATION FOR REINFORCEMENT LEARNING USING NEUROEVOLUTION
SN - 978-989-8111-66-1
AU - Wright R.
AU - Gemelli N.
PY - 2009
SP - 45
EP - 52
DO - 10.5220/0001658800450052