Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks

Stefan Glüge, Ronald Böck, Andreas Wendemuth

2012

Abstract

We introduce an extended Backpropagation Through Time (eBPTT) learning algorithm for Segmented-Memory Recurrent Neural Networks. The algorithm was compared to an extension of the Real-Time Recurrent Learning algorithm (eRTRL) for these kind of networks. Using the information latching problem as benchmark task, the algorithms’ ability to cope with the learning of long-term dependencies was tested. eRTRL was generally better able to cope with the latching of information over longer periods of time. On the other hand, eBPTT guaranteed a better generalisation when training was successful. Further, due to its computational complexity, eRTRL becomes impractical with increasing network size, making eBPTT the only viable choice in these cases.

Download


Paper Citation


in Harvard Style

Glüge S., Böck R. and Wendemuth A. (2012). Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks . In Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012) ISBN 978-989-8565-33-4, pages 451-456. DOI: 10.5220/0004103804510456

in Bibtex Style

@conference{ncta12,
author={Stefan Glüge and Ronald Böck and Andreas Wendemuth},
title={Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks},
booktitle={Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012)},
year={2012},
pages={451-456},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004103804510456},
isbn={978-989-8565-33-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012)
TI - Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks
SN - 978-989-8565-33-4
AU - Glüge S.
AU - Böck R.
AU - Wendemuth A.
PY - 2012
SP - 451
EP - 456
DO - 10.5220/0004103804510456