The Prompt Quality Report: What 1,000 Scored Prompts Reveal

작성자

카테고리:

← 피드로
DEV Community · Francisco Ferreira · 2026-07-08 개발(SW)
Cover image for The Prompt Quality Report: What 1,018 Scored Prompts Reveal

Francisco Ferreira

Update: This analysis is now maintained as a living benchmark at The State of Prompt Quality. The frozen Q3 2026 edition — with the full methodology and charts — lives here. Figures below match that edition (n = 1,018).

I run a prompt evaluator. Every prompt submitted gets scored 0–100 by a fixed LLM-judge rubric across four dimensions — clarity, specificity, structure, and robustness. I aggregated the anonymous scores from 1,018 real prompts (May–July 2026). The prompt text itself never enters the dataset — only scores and structural metadata.

Some of the numbers surprised me. Here they are.

The average prompt scores 54/100

Median 60. Only 10.5% clear 75, the score where a prompt is consistent enough to trust in repeated use. About a third score below 50. Most prompts aren’t broken — they’re mediocre in a very specific way: they work once, in the demo, with clean input.

Robustness is the weak spot in 96% of prompts

This is the finding. Three dimensions cluster together; one collapses:

Dimension Average Structure 64.6 Clarity 63.0 Specificity 57.6 Robustness 31.5

Robustness — does the prompt survive empty, messy, or off-topic input — was the weakest dimension in 96% of prompts. Not a tail effect: it holds across every use case and language in the dataset. The two lowest-scoring behaviors of all eight measured were bad-input resilience (30/100) and edge-case coverage (32/100).

People polish the wording. Nobody writes the line that handles the bad day.

What actually moves the score

I tagged every prompt for four structural habits and compared average scores with and without each:

Habit Avg. lift Prompts that use it Declaring an output format +29 79% Setting constraints (what not to do) +24 55% Giving the model a persona +17 70% Including one example +10 5%

The sleeper is examples. Smallest lift on paper, but only 5% of prompts include one — the most underused habit by far. Everyone has heard “give the model examples.” Almost nobody does it.

The honest caveats

  • Selection bias. These are prompts people chose to submit to an evaluator, often because they suspected something was wrong. Scores likely skew lower than the general population. Read every figure as “prompts submitted for evaluation,” never “all prompts.”
  • Correlation, not controlled experiment. The lift numbers are associations, not proof of causation.
  • Segments under n = 150 don’t get standalone claims.

If you fix one thing

Add one line for the bad day:

“If the input is empty, unclear, or off-topic, say so and ask for clarification instead of guessing.”

That single sentence addresses the behavior 85% of prompts score under 50 on.

Full report with distribution charts and the complete methodology: The State of Prompt Quality — Q3 2026. Data is CC BY 4.0. The evaluator is PromptEval — mine; the report has no paywall.

원문에서 계속 ↗

코멘트

답글 남기기

이메일 주소는 공개되지 않습니다. 필수 필드는 *로 표시됩니다