Exploiting inherent relationships in RNN architectures.
basic_science · Level V
Where this comes from
- Record sourced from PubMed, PMID 12662618.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
We provide the relationship between the learning rate and the slope of a nonlinear activation function of a neuron within the framework of nonlinear modular cascaded systems realised through Recurrent Neural Network (RNN) architectures. This leads to reduction in the computational complexity of learning algorithms which continuously adapt the weights of such architectures, because there is a smaller number of independent parameters to optimise. Results are provided for the Gradient Descent (GD) learning algorithm and the Extended Recursive Least Squares (ERLS) algorithm, using a general nonlinear activation function of a neuron. The results obtained degenerate into the corresponding results for single RNNs, when considering only one module in such cascaded systems.