Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks
Stefan Glüge, Ronald Böck, Andreas Wendemuth
2012
Abstract
We introduce an extended Backpropagation Through Time (eBPTT) learning algorithm for Segmented-Memory Recurrent Neural Networks. The algorithm was compared to an extension of the Real-Time Recurrent Learning algorithm (eRTRL) for these kind of networks. Using the information latching problem as benchmark task, the algorithms’ ability to cope with the learning of long-term dependencies was tested. eRTRL was generally better able to cope with the latching of information over longer periods of time. On the other hand, eBPTT guaranteed a better generalisation when training was successful. Further, due to its computational complexity, eRTRL becomes impractical with increasing network size, making eBPTT the only viable choice in these cases.
DownloadPaper Citation
in Harvard Style
Glüge S., Böck R. and Wendemuth A. (2012). Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks . In Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012) ISBN 978-989-8565-33-4, pages 451-456. DOI: 10.5220/0004103804510456
in Bibtex Style
@conference{ncta12,
author={Stefan Glüge and Ronald Böck and Andreas Wendemuth},
title={Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks},
booktitle={Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012)},
year={2012},
pages={451-456},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004103804510456},
isbn={978-989-8565-33-4},
}
in EndNote Style
TY - CONF
JO - Proceedings of the 4th International Joint Conference on Computational Intelligence - Volume 1: NCTA, (IJCCI 2012)
TI - Extension of Backpropagation through Time for Segmented-memory Recurrent Neural Networks
SN - 978-989-8565-33-4
AU - Glüge S.
AU - Böck R.
AU - Wendemuth A.
PY - 2012
SP - 451
EP - 456
DO - 10.5220/0004103804510456