A Nonlinear Singular Value Theory for Neural Networks

[Submitted on 7 May 2026 (v1), last revised 30 Jul 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Recently Brown et al. [2025] established a singular value decomposition (SVD) for maps (especially nonlinear) satisfying certain norm conditions. We prove that most modern neural architectures admit this nonlinear SVD (NLSVD) representation—with no change in input–output behavior—and enumerate the classes covered. In this factorization the network is a left-invertible nonlinear map followed by a final linear layer. Moreover, the left-invertible factor is norm-preserving, so distances in the embedding (activations before the final linear layer) calibrate directly to distances in input space. We introduce a flexible architecture that yields an explicit decomposition at training time, a data-driven algorithm for estimating the representation from trained models, and the mathematical foundations for nonlinear analogues of row and null spaces in neural networks. Empirical case studies illustrate uses of the theory for latent-space pullback (visualization and data generation), bias detection, and membership-inference robustness under training. Altogether, these foundations support new approaches to core problems in neural-network analysis.

Submission history

From: Robert Bridges [view email]
[v1] Thu, 7 May 2026 20:55:14 UTC (1,601 KB)
[v2] Thu, 30 Jul 2026 19:29:56 UTC (1,979 KB)