Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures With Gradient Learnings.

Bukovsky, Ivo; Dohnal, Gejza; Benes, Peter M; Ichiji, Kei; Homma, Noriyasu · IEEE Trans Neural Netw Learn Syst · 2023

Where this comes from

Abstract

This letter summarizes and proves the concept of bounded-input bounded-state (BIBS) stability for weight convergence of a broad family of in-parameter-linear nonlinear neural architectures (IPLNAs) as it generally applies to a broad family of incremental gradient learning algorithms. A practical BIBS convergence condition results from the derived proofs for every individual learning point or batches for real-time applications.