Bayesian Statistics as an Alternative to Gradient Descent in Sequence Learning

Rainer Spiegel


Recurrent neural networks are frequently applied to simulate sequence learning applications such as language processing, sensory-motor learning, etc. For this purpose, they often apply a truncated gradient descent (=error correcting) learning algorithm. In order to converge to a solution that is congruent with a target set of sequences, many iterations of sequence presentations and weight adjustments are typically needed. Moreover, there is no guarantee of finding the global minimum of error in a multidimensional error landscape resulting from the discrepancy between target values and the network?s prediction. This paper presents a new approach of inferring the global error minimum right from the start. It further applies this information to reverse-engineer the weights. As a consequence, learning is speeded-up tremendously, whilst computationally-expensive iterative training trials can be skipped. Technology applications in established and emerging industries will be discussed.

Full Text:


Copyright (c) 2017 Rainer Spiegel

International Journal of Emerging Technologies in Learning. ISSN: 1863-0383
Creative Commons License SPARC Europe Seal
Web of Science ESCI logo Engineering Information logo INSPEC logo DBLP logo ELSEVIER Scopus logo EDiTLib logo EBSCO logo Ulrich's logo Google Scholar logo Microsoft® Academic SearchDOAJ logo