Your LLM Fallback Probably Isn't a Fallback

At 04:00 UTC, every model call through our LLM gateway started returning
HTTP 400. Not some calls. All of them. Our tier-1 CI gate flagged it, and the
fix was committed at 04:26 UTC the same morning — about 26 minutes end to end.

This is the post-mortem.

What happened

DeepSeek retired two API model names — deepseek-chat and deepseek-reasoner —
at their V4 cutover around 2026-07-24 15:59 UTC. The replacements are
deepseek-v4-pro and deepseek-v4-flash.

Our gateway config still declared both retired names. Starting roughly twelve
hours after the retirement, every model request routed through the gateway hit a
400 with the body:

The supported API model names are deepseek-v4-pro or deepseek-v4-flash, but
you passed .

A live API check confirmed the shape of the cutover with four requests, same
valid key:

Model name Response deepseek-v4-pro HTTP 200 deepseek-v4-flash HTTP 200 deepseek-chat HTTP 400 deepseek-v4-pro-quantized HTTP 400

The two working names are the replacements. The two retired names — the ones our
config referenced — returned 400. The fourth row is a name that does not exist
at all, included because an earlier reading of a truncated error message had
suggested it; shipping it would have left the platform broken. We’ll come back
to that.

Why the fallback didn’t help

We had a fallback configured. Three separate model references in our policy
config — the default CLI/workflow model, the chat model, and the shared fallback
model — all pointed at the two retired names. All three lived under the same
vendor and the same API key.

When the primary call returned 400, the gateway tried the fallback. The log told
the story in two adjacent lines: the 400 from the provider, and then
Error doing the fallback: carrying the identical error. The fallback died in
the same instant as the primary because it was the same thing wearing a
different label.

This is the structural problem. A fallback that shares a provider and an API key
with its primary is not resilience. It protects against a single model
misbehaving. It protects against nothing at the provider level — an outage, a
credential revocation, or a model rename all take down primary and fallback
together. It also makes the “does the fallback work?” test structurally
unpassable during any provider incident, because a red result cannot distinguish
“fallback broken” from “provider down.”

How we found it

Our tier-1 CI test gate runs a minimal LLM reachability check on every push. On
the failing run it reported:

llm_reachability.py: FAIL: P2 'deepseek-reasoner' reachable
FAIL: P3 pi node executes end-to-end
FAIL: P4 chat succeeded despite dead primary

Enter fullscreen mode Exit fullscreen mode

The gate’s drift check flagged it as a regression — 108 of 109 checks passing,
one new failure. The immediately preceding run of that same gate had been fully
green, which proved the cause was external, not something we had just committed.

A gate that catches a provider-side model retirement in a single CI run is the
difference between finding out from a dashboard and finding out from a user.

The fix

Two lines of gateway config: repoint the two aliases at deepseek-v4-pro and
deepseek-v4-flash. The alias names were left unchanged, so nothing downstream
that referenced them had to change.

The mapping was chosen by running the live API check above — not by reading the
error string. An earlier reading of a truncated error message had suggested a
model name (deepseek-v4-pro-quantized) that does not exist. Shipping that
would have left the platform broken. Four seconds of curl beat a confident
guess.

After the fix, both aliases returned HTTP 200 from inside the running pod, and
the tier-1 gate returned to fully green with no drift.

What we filed as the real defect

The config fix closed the immediate outage. The real defect is the fallback
design: same provider, same key, shared fate. We filed it as a tracked gap. It
will stay open until the fallback crosses a provider boundary and carries
independent credentials.

Check your own stack — today

Here is a concrete checklist you can run against your own setup right now:

1. Does your fallback cross a provider boundary? If primary and fallback
   both live under the same vendor and the same API key, they share a fate.
   A provider outage, credential revocation, or model deprecation will take
   both out simultaneously.

2. Do you have a reachability canary? A single HTTP 200 check per declared
   model, running in CI on every push (or on a short cron), would have caught
   this in minutes. The check should be cheap — one token of output is enough
   to prove the model is alive.

3. Would you find out from a gate or from a user? If your answer is “a
   user,” your monitoring has a coverage gap. The test that catches this does
   not need to be in production — a CI gate that fails on a model returning
   400 is sufficient and costs almost nothing.

This incident came from a small platform still under active development. The
lesson generalises to any system that routes LLM requests through a gateway.