BIDIRECTIONAL HIERARCHICAL NEURAL NETWORKS - Hebbian Learning Improves Generalization

Mohammad Saifullah, Rita Kovordanyi, Chandan Roy

2010

Abstract

Visual pattern recognition is a complex problem, and it has proven difficult to achieve satisfactorily in standard three-layer feed-forward artificial neural networks. For this reason, an increasing number of researchers are using networks whose architecture resembles the human visual system. These biologically-based networks are bidirectionally connected, use receptive fields, and have a hierarchical structure, with the input layer being the largest layer, and consecutive layers getting increasingly smaller. These networks are large and complex, and therefore run a risk of getting overfitted during learning, especially if small training sets are used, and if the input patterns are noisy. Many data sets, such as, for example, handwritten characters, are intrinsically noisy. The problem of overfitting is aggravated by the tendency of error-driven learning in large networks to treat all variations in the noisy input as significant. However, there is one way to balance off this tendency to overfit, and that is to use a mixture of learning algorithms. In this study, we ran systematic tests on handwritten character recognition, where we compared generalization performance using a mixture of Hebbian learning and error-driven learning with generalization performance using pure error-driven learning. Our results indicate that injecting even a small amount of Hebbian learning, 0.01 %, significantly improves the generalization performance of the network.

Paper Citation

in Harvard Style

Saifullah M., Kovordanyi R. and Roy C. (2010). BIDIRECTIONAL HIERARCHICAL NEURAL NETWORKS - Hebbian Learning Improves Generalization . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010) ISBN 978-989-674-028-3, pages 105-111. DOI: 10.5220/0002835501050111

in Bibtex Style

@conference{visapp10,
author={Mohammad Saifullah and Rita Kovordanyi and Chandan Roy},
title={BIDIRECTIONAL HIERARCHICAL NEURAL NETWORKS - Hebbian Learning Improves Generalization},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)},
year={2010},
pages={105-111},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002835501050111},
isbn={978-989-674-028-3},
}

in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2010)
TI - BIDIRECTIONAL HIERARCHICAL NEURAL NETWORKS - Hebbian Learning Improves Generalization
SN - 978-989-674-028-3
AU - Saifullah M.
AU - Kovordanyi R.
AU - Roy C.
PY - 2010
SP - 105
EP - 111
DO - 10.5220/0002835501050111