Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

[Submitted on 5 Nov 2024 (v1), last revised 9 Jun 2026 (this version, v2)]

View PDF HTML (experimental)

Abstract:Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and Rényi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing textit{Conditional-Vendi} and textit{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and LLM tasks show that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse samples. The codebase is available at this https URL.

Submission history

From: Mohammad Jalali [view email]
[v1] Tue, 5 Nov 2024 05:30:39 UTC (5,111 KB)
[v2] Tue, 9 Jun 2026 06:28:35 UTC (9,054 KB)