ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion

[Submitted on 8 Jul 2025 (v1), last revised 2 Jul 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Multimodal emotion and intent recognition is essential for automated human-computer interaction, It aims to analyze users’ speech, text, and visual information to predict their emotions or intent. One of the significant challenges is that missing modalities due to sensor malfunctions or incomplete data. Traditional methods that attempt to reconstruct missing information often suffer from over-coupling and imprecise generation processes, leading to suboptimal outcomes. To address these issues, we introduce an Attention-based Diffusion model for Missing Modalities feature Completion (ADMC). Our framework independently trains feature extraction networks for each modality, preserving their unique characteristics and avoiding over-coupling. The Attention-based Diffusion Network (ADN) generates missing modality features that closely align with authentic multimodal distribution, enhancing performance across all missing-modality scenarios. Moreover, ADN’s cross-modal generation offers improved recognition even in full-modality contexts. Our approach achieves state-of-the-art results on the IEMOCAP and MIntRec benchmarks, demonstrating its effectiveness in both missing and complete modality scenarios.

Submission history

From: Wei Zhang [view email]
[v1] Tue, 8 Jul 2025 03:08:52 UTC (2,052 KB)
[v2] Thu, 2 Jul 2026 13:55:00 UTC (6,961 KB)