SCALED GRADIENT DESCENT LEARNING RATE - Reinforcement learning with light-seeking robot

Kary Främling

2004

Abstract

Adaptive behaviour through machine learning is challenging in many real-world applications such as robotics. This is because learning has to be rapid enough to be performed in real time and to avoid damage to the robot. Models using linear function approximation are interesting in such tasks because they offer rapid learning and have small memory and processing requirements. Adalines are a simple model for gradient descent learning with linear function approximation. However, the performance of gradient descent learning even with a linear model greatly depends on identifying a good value for the learning rate to use. In this paper it is shown that the learning rate should be scaled as a function of the current input values. A scaled learning rate makes it possible to avoid weight oscillations without slowing down learning. The advantages of using the scaled learning rate are illustrated using a robot that learns to navigate towards a light source. This light-seeking robot performs a Reinforcement Learning task, where the robot collects training samples by exploring the environment, i.e. taking actions and learning from their result by a trial-and-error procedure.

Paper Citation

in Harvard Style

Främling K. (2004). SCALED GRADIENT DESCENT LEARNING RATE - Reinforcement learning with light-seeking robot . In Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO, ISBN 972-8865-12-0, pages 3-11. DOI: 10.5220/0001138600030011

in Bibtex Style

@conference{icinco04,
author={Kary Främling},
title={SCALED GRADIENT DESCENT LEARNING RATE - Reinforcement learning with light-seeking robot},
booktitle={Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,},
year={2004},
pages={3-11},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001138600030011},
isbn={972-8865-12-0},
}

in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,
TI - SCALED GRADIENT DESCENT LEARNING RATE - Reinforcement learning with light-seeking robot
SN - 972-8865-12-0
AU - Främling K.
PY - 2004
SP - 3
EP - 11
DO - 10.5220/0001138600030011