Learning What to Remember: Observability-Safe Memory Retention via Constrained Optimization for Long-Horizon Language Agents

[Submitted on 9 Jun 2026 (v1), last revised 29 Jun 2026 (this version, v6)]

View PDF HTML (experimental)

Abstract:Long-horizon language agents accumulate observations, reasoning traces, and retrieved facts exceeding context windows, making memory retention a fundamental resource-allocation problem. Existing systems treat retention as local and do not model long-term consequences under observability constraints. To fill this gap, we formulate memory retention as a constrained stochastic optimization with budget feasibility, evidence utility, and delayed costs including miss, reacquisition, and stale penalties. We show this multi-step problem is NP-hard, making exact solution intractable. Moreover, deployment decisions must be made under partial observability. To address these challenges, we propose OSL-MR (Observability-Safe Learning for Memory Retention), a learning-augmented framework that enforces a strict separation between online-observable features and offline-available supervision. OSL-MR combines an evidence learner trained from realized evidence with a Mixed-Score heuristic that serves as a deployable online-safe baseline and an inductive prior. The policy learns query-conditioned evidence from interaction data and remains deployable under the same constraints. Experiments on LoCoMo and LongMemEval show OSL-MR outperforms recency-based, Generative Agents-style, and other heuristic baselines, especially under tight budgets. The Mixed-Score prior improves precision and recall, and sensitivity analysis shows robustness across cost settings. On small solvable instances, single-step optimization is insufficient to anticipate future demand shifts, while OSL-MR stays significantly closer to the dynamic-programming optimum, confirming the necessity of the sequential formulation and reinforcing our learning-guided approximation. These results establish constrained stochastic optimization and optimization-guided learning as a principled foundation for memory management in long-horizon agents.

Submission history

From: Mingyang Liu [view email]
[v1] Tue, 9 Jun 2026 09:15:33 UTC (4,277 KB)
[v2] Thu, 11 Jun 2026 09:47:38 UTC (4,283 KB)
[v3] Tue, 16 Jun 2026 14:01:50 UTC (4,297 KB)
[v4] Thu, 18 Jun 2026 02:39:41 UTC (4,299 KB)
[v5] Mon, 22 Jun 2026 08:45:48 UTC (8,286 KB)
[v6] Mon, 29 Jun 2026 08:38:19 UTC (3,454 KB)