Theoretical Notes on Unsupervised Learning in Deep Neural Networks

Vladimir Golovko, Aliaksandr Kroshchanka

2016

Abstract

Over the last decade the deep neural networks are the powerful tool in the domain of machine learning. The important problem is training of deep neural network, because learning of such a network is much complicated compared to shallow neural networks. This is due to the vanishing gradient problem, poor local minima and unstable gradient problem. Therefore a lot of deep learning techniques were developed that permit us to overcome some limitations of conventional training approaches. In this paper we investigate the unsupervised learning in deep neural networks. We have proved that maximization of the log-likelihood input data distribution of restricted Boltzmann machine is equivalent to minimizing the cross-entropy and to special case of minimizing the mean squared error. The main contribution of this paper is a novel view and new understanding of an unsupervised learning in deep neural networks.

Paper Citation

in Harvard Style

Golovko V. and Kroshchanka A. (2016). Theoretical Notes on Unsupervised Learning in Deep Neural Networks . In Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 2: NCTA, (IJCCI 2016) ISBN 978-989-758-201-1, pages 91-96. DOI: 10.5220/0006084300910096

in Bibtex Style

@conference{ncta16,
author={Vladimir Golovko and Aliaksandr Kroshchanka},
title={Theoretical Notes on Unsupervised Learning in Deep Neural Networks},
booktitle={Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 2: NCTA, (IJCCI 2016)},
year={2016},
pages={91-96},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006084300910096},
isbn={978-989-758-201-1},
}

in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Joint Conference on Computational Intelligence - Volume 2: NCTA, (IJCCI 2016)
TI - Theoretical Notes on Unsupervised Learning in Deep Neural Networks
SN - 978-989-758-201-1
AU - Golovko V.
AU - Kroshchanka A.
PY - 2016
SP - 91
EP - 96
DO - 10.5220/0006084300910096