Bayesian Statistics as an Alternative to Gradient Descent in Sequence Learning

Rainer Spiegel

Abstract


Recurrent neural networks are frequently applied to simulate sequence learning applications such as language processing, sensory-motor learning, etc. For this purpose, they often apply a truncated gradient descent (=error correcting) learning algorithm. In order to converge to a solution that is congruent with a target set of sequences, many iterations of sequence presentations and weight adjustments are typically needed. Moreover, there is no guarantee of finding the global minimum of error in a multidimensional error landscape resulting from the discrepancy between target values and the network?s prediction. This paper presents a new approach of inferring the global error minimum right from the start. It further applies this information to reverse-engineer the weights. As a consequence, learning is speeded-up tremendously, whilst computationally-expensive iterative training trials can be skipped. Technology applications in established and emerging industries will be discussed.

Full Text:

PDF


Copyright (c) 2017 Rainer Spiegel


International Journal of Emerging Technologies in Learning. ISSN: 1863-0383
Creative Commons License SPARC Europe Seal
Indexing:
Web of Science ESCI logo Engineering Information logo INSPEC logo DBLP logo ELSEVIER Scopus logo EDiTLib logo EBSCO logo Ulrich's logo Google Scholar logo Microsoft® Academic Search