Do Neural Networks Lose Plasticity in a Gradually Changing World?

[Submitted on 9 Feb 2026 (v1), last revised 16 Jun 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Continual learning has become a trending topic in machine learning. Recent studies have discovered an interesting phenomenon called loss of plasticity, referring to neural networks gradually losing the ability to learn new tasks. However, existing plasticity research largely relies on benchmarks with abrupt task transitions, without examining whether the abruptness itself contributes to the observed plasticity loss. In this paper, we investigate the role of transition abruptness by simulating gradually changing environments through input/output interpolation and task sampling. We perform theoretical and empirical analysis, showing that the severity of plasticity loss is closely tied to the abruptness of task transitions, and can be substantially reduced when the environment changes gradually.

Submission history

From: Tianhui Liu [view email]
[v1] Mon, 9 Feb 2026 22:01:50 UTC (1,341 KB)
[v2] Tue, 16 Jun 2026 19:13:34 UTC (2,303 KB)