문맥 길이 초과: 요청을 줄이고 결과를 확인합니다.

Context Length Exceeded: Reduce the Request and Verify the Result

Fix Context length exceeded by accounting for the total budget, not by randomly deleting half the prompt. Identify the model and measure each request component first. Then remove mechanical duplicates, irrelevant history, and heavy tool results. After resending, verify both that the error is gone and that required facts and answer completeness remain intact.

What fills the context window

Context is the working memory of one request, not just the latest user prompt. In simplified form:

system instructions
+ conversation history
+ current message
+ images and documents
+ tool definitions
+ tool results
+ output / thinking budget
= total context usage

Enter fullscreen mode Exit fullscreen mode

Anthropic’s context-window documentation explicitly includes the system prompt, all messages, images, documents, tool definitions, tool results, and generated response. Prompt caching changes the cost of reused tokens, but it does not remove them from the window.

Do not hard-code a “universal limit”: the context window and overflow behavior depend on the selected model and API. Check the current model card on the day you configure it.

Find the heaviest component

Component What to measure Safe reduction System instructions Repeated rules and long examples Merge duplicates and retain mandatory constraints History Tokens by message and old branches Remove irrelevant branches or replace them with a verifiable summary Files and RAG fragments Each document’s size, duplicates, and low-relevance chunks Lower top_k, deduplicate, and pass only the required sections Tool definitions Unused tools and long descriptions Pass only the tools needed for the current step Tool results Complete JSON, logs, HTML, base64, and repeated responses Keep required fields, links, and identifiers; store large data outside the prompt Output budget max_tokens and thinking budget Reserve a realistic amount or split a large result into stages

Claude has a separate Token Counting API that accounts for messages and tools before a request is sent. For another provider, use its own counter if available. A local tokenizer is useful for early warnings, but its estimate is not a guaranteed server-side count for a different model.

Open the current BetterToken API documentation, run one short test request, and find it in Dashboard. Compare input, output, and cache tokens before and after trimming the context: lower input tokens confirm that the fix reached the real call. Dashboard does not show the full prompt or replace preflight token counting; it verifies your own request after the fact.

Reduce size without losing meaning

1. Remove mechanical duplicates

Look for repeated system rules, the same file in several messages, duplicate RAG chunks, repeated schemas, and complete logs pasted more than once. This is the safest stage because it reduces size without changing the task.

2. Remove irrelevant history

Separate long-lived facts from the temporary flow of the conversation. Preserve goals, accepted decisions, mandatory constraints, and open questions. Remove old reasoning, rejected alternatives, and tool results that have already been processed, or compact them into a structured summary.

A bad summary says “we discussed the integration.” A useful one records the chosen endpoint, schema version, accepted constraints, confirmed facts, and next step.

3. Compact files, RAG, and tool results

Pass relevant sections instead of an entire document. In a tool result, keep the fields required by the next step instead of the full HTTP response or log. Do not discard sources or mandatory data merely to make the request fit; split the work into several verifiable stages instead.

4. Leave room for the answer

Input and output share the total budget. If the request nearly fills the window, the model may not have enough room for a complete answer. Reduce optional input, set a realistic output budget, or split the result into parts. Remove mechanical duplicates before critical facts.

5. Change models only after measuring

A larger-context model may be the right choice for a document that cannot be split safely. Moving to a larger window without removing duplicates only postpones the next failure and may reduce information density.

Minimal preflight check

components = count_by_section(request)
estimated_input = sum(components)
reserved_output = requested_output_budget

if estimated_input + reserved_output approaches current_model_window:
    remove exact duplicates
    drop irrelevant history
    compact tool results and retrieved chunks
    count again

send only after required facts and constraints remain present

Enter fullscreen mode Exit fullscreen mode

approaches is intentionally not replaced by a fixed percentage. The necessary headroom depends on counter accuracy, the model, thinking, and the specific API’s behavior.

How to verify the fix

Compare the new request against this checklist:

1. The API no longer returns context length exceeded or prompt is too long.
2. The response completes normally instead of being cut off by the output budget.
3. The response includes all mandatory facts, constraints, and the required format.
4. Quotes or links still correspond to the supplied sources.
5. Tool calls use the right arguments, and important results were not lost during compaction.
6. New input usage is genuinely lower than before.

If the error disappears but the model forgets a key constraint, the fix failed. Restore the mandatory block and free space by removing less relevant history or a heavy tool result. If the answer is cut off, inspect the output budget separately; it is another part of the same total window.

Short conclusion

The working sequence is: identify the model → count components → remove exact duplicates → extract irrelevant history → compact files and tool results → leave room for the answer → resend → verify quality. This addresses the cause without turning context into an arbitrarily truncated set of facts.

Sources

• Claude: context windows
• Claude: token counting
• Claude API errors
• OpenRouter: errors and debugging
• BetterToken API reference

Originally published on the BetterToken blog.

BetterToken provides pay-as-you-go access to AI model APIs through
OpenAI-compatible and Anthropic-compatible endpoints — useful if you are wiring
Claude Code, Codex, or your own tooling to a custom base URL.
See the docs to get started.