Hypercube Neural Topologies: Enhancing Depth Efficiency and Gradient Flow in Deep Networks.

Abstract

We propose a neural network architecture grounded in high-dimensional hypercube topology. In contrast to conventional sequential or skip-connected designs, the proposed approach maps layers to the vertices of an $n$ -dimensional hypercube and establishes interlayer relationships through its edges. This topological arrangement substantially reduces the effective depth, even for models with a large number of layers, while enabling a rich ensemble of shallow and diverse learning paths. Consequently, the architecture improves gradient propagation and preserves trainability even in the absence of residual shortcuts. It also demonstrates strong numerical robustness and remains stably trainable under pure half-precision (FP16) arithmetic without auxiliary precision mechanisms. To facilitate practical adoption, we provide algorithms that systematically assign layers to hypercube vertices and construct directed edge connections. Under equivalent layer counts, the proposed architecture consistently outperforms traditional 1-D residual networks. The improvement is observed in both feature learning efficiency and representational capacity and becomes more pronounced as network depth increases. Moreover, the proposed topology provides a regularizing effect that mitigates over-parameterization and supports stable training across varying data scales. We validate these design principles through extensive experiments.