Offline Diffusion Policy for Multi-User Delay-Constrained Scheduling

[Submitted on 22 Jan 2025 (v1), last revised 10 Jun 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Effective multi-user delay-constrained scheduling is crucial in various real-world applications, including embodied AI, instant messaging, live streaming, and data center management, where efficient resource allocation is required among users with diverse delay sensitivities. In these scenarios, schedulers must make real-time decisions to satisfy both delay and resource constraints without prior knowledge of system dynamics, which are often time-varying and challenging to estimate. {Current learning-based methods typically require online interactions with actual systems during the training stage. Therefore, these approaches are often difficult or impractical, as they can significantly degrade system performance and incur substantial service costs.} To address these challenges, we propose a novel offline reinforcement learning-based algorithm, named underline{S}cheduling By underline{O}ffline Learning with underline{C}ritic Guidance and underline{D}iffusion Model (SOCD), to learn efficient scheduling policies purely from pre-collected emph{offline data}. SOCD innovatively employs a diffusion policy, complemented by a sampling-free critic network for policy guidance. By integrating the Lagrangian multiplier optimization into the offline reinforcement learning, SOCD efficiently trains high-quality constraint-aware policies exclusively from available datasets, eliminating the need for online interactions with the system. Experimental results demonstrate that SOCD is resilient to various system dynamics, including partially observable and large-scale environments, and delivers superior performance compared to existing methods.

Submission history

From: Zhuoran Li [view email]
[v1] Wed, 22 Jan 2025 15:13:21 UTC (853 KB)
[v2] Wed, 10 Jun 2026 06:14:49 UTC (1,018 KB)