Orbit-based universal approximation via hypercyclicity on compact-open topology.

Abstract

We establish compact-open topology uniform universal approximation driven by hypercyclic orbits of an affine composition operator. In one dimension, for suitable continuous activation functions, we show that a fixed weighted translation operator is hypercyclic on the closure, in the compact-open topology, of the class of functions represented by one-hidden-layer neural networks. Consequently, there exists a single seed function in this closure whose iterates approximate every target uniformly on compact sets. We also prove density of periodic points, and hence chaoticity of the operator. Under the corresponding compact-wise density assumption for deeper and higher-dimensional network classes, the same orbit-based approximation mechanism extends to arbitrary depth and dimension. Thus approximation is achieved by moving along the orbit of one fixed seed under one fixed operator, rather than by re-optimizing parameters for each target.