Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing

[Submitted on 5 Oct 2025 (v1), last revised 17 Jun 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Large language models (LLMs) achieve strong performance on metaphor detection and interpretation tasks, yet it remains unclear what such behavioral success reveals about metaphor processing. We present a diagnostic analysis that examines the limits of behavioral evidence by probing three complementary dimensions: semantic attribute alignment, lexical invariance, and syntactic sensitivity. Using geometric probing, we assess whether model-generated interpretations align with reference semantic attributes; through context-varying substitution, we analyze the stability of lexical associations between metaphorical and literal expressions; and via controlled syntactic perturbations, we examine sensitivity in metaphor detection. Our analysis reveals that LLM-generated interpretations can exhibit semantic drift relative to reference attributes; stable lexical anchors persist across contextual conditions, potentially supporting conventional metaphors while biasing novel metaphors requiring contextual integration; and detection performance is sensitive to syntactic irregularities. These findings suggest that strong behavioral performance may reflect heterogeneous underlying signals, highlighting the need for caution when interpreting metaphor benchmarks as evidence of robust, integrated semantic understanding.

Submission history

From: Fengying Ye [view email]
[v1] Sun, 5 Oct 2025 09:45:51 UTC (3,715 KB)
[v2] Wed, 17 Jun 2026 10:49:32 UTC (2,850 KB)