Automatic Stability and Recovery for Neural Network Training

[Submitted on 24 Jan 2026 (v1), last revised 24 Jul 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Training modern neural networks is increasingly fragile, with rare but severe destabilizing updates often causing irreversible divergence or silent performance degradation. Existing optimization methods primarily rely on preventive mechanisms embedded within the optimizer, offering limited ability to detect and recover from instability once it occurs. We introduce a supervisory runtime stability framework that treats optimization as a controlled stochastic process. By isolating an innovation signal derived from secondary measurements, such as validation probes, the framework enables automatic detection and recovery from destabilizing updates without modifying the underlying optimizer. We provide theoretical runtime safety guarantees that formalize bounded degradation and recovery. Our implementation incurs minimal overhead and is compatible with memory-constrained training settings.

Submission history

From: Barak Or [view email]
[v1] Sat, 24 Jan 2026 15:14:54 UTC (725 KB)
[v2] Fri, 24 Jul 2026 07:01:24 UTC (1,456 KB)