Last month, a side project died at the API checkout. The code worked. The credit card did not.
The fix is not a bigger budget. The fix is a smaller one.
This tutorial builds a working LLM endpoint from zero. Every step ends with a verification command. You need a terminal, Python 3.11+, and about thirty minutes.
Why cost control is the real feature
LLM prices keep dropping. My API bills did not. The reason: I never measured usage before adding features.
Everyone is talking about agent memory right now. Token accounting is the boring sibling nobody writes about. This tutorial closes that gap.
The service you build summarizes incoming text under a hard token budget. It tracks every token it spends. It fails loudly when the budget is exceeded.
What we are building
- A Python service with one endpoint:
POST /summarize - A
TokenBudgetclass that estimates, truncates, and tracks - A deployment target that costs nothing
- A verification step for each stage
Free tokens still have limits. The budget class makes those limits visible.
Step 0 — Get the two free things
MonkeyCode offers free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode’s product outreach.
At the time of writing, the free tier includes 10 million tokens. Quotas change. Verify the current numbers in the dashboard before you build on them.
You need three values:
- An API key
- The current base URL from the docs
- The current free model ID
Export them as environment variables:
export MONKEYCODE_API_KEY="..."
export MONKEYCODE_BASE_URL="..."
export MONKEYCODE_MODEL="..."
Enter fullscreen mode Exit fullscreen mode
Verification:
curl -s "$MONKEYCODE_BASE_URL/models" \
-H "Authorization: Bearer $MONKEYCODE_API_KEY"
Enter fullscreen mode Exit fullscreen mode
The exact path lives in the current docs. If the response lists models, you are ready.
Step 1 — Scaffold the project
mkdir budget-llm && cd budget-llm
python3 -m venv .venv
source .venv/bin/activate
pip install httpx
Enter fullscreen mode Exit fullscreen mode
Verification:
python -c "import httpx; print(httpx.__version__)"
Enter fullscreen mode Exit fullscreen mode
One dependency. That keeps the free server deployment boring. Boring is what you want in production.
Step 2 — Write the token budget
Create budget.py. The example assumes an OpenAI-compatible chat endpoint. Confirm the request shape in the current docs before running.
# budget.py
import os
import httpx
class TokenBudget:
def __init__(self, limit: int, base_url: str = "", api_key: str = "", model: str = ""):
self.limit = limit
self.spent = 0
self._base_url = base_url or os.environ["MONKEYCODE_BASE_URL"]
self._api_key = api_key or os.environ["MONKEYCODE_API_KEY"]
self._model = model or os.environ["MONKEYCODE_MODEL"]
self._client = None
def _get_client(self) -> httpx.Client:
if self._client is None:
self._client = httpx.Client(
base_url=self._base_url,
headers={"Authorization": f"Bearer {self._api_key}"},
timeout=30.0,
)
return self._client
@staticmethod
def estimate(text: str) -> int:
# Heuristic: about four characters per token.
return max(1, len(text) // 4)
def fit(self, text: str) -> str:
budget = self.limit - self.spent - 100 # reserve room for the reply
if budget <= 0:
raise RuntimeError("Token budget exhausted")
while self.estimate(text) > budget:
text = text[: len(text) // 2]
return text
def summarize(self, text: str) -> str:
prompt = self.fit(text)
response = self._get_client().post(
"/chat/completions",
json={
"model": self._model,
"messages": [
{"role": "system", "content": "Summarize in three sentences."},
{"role": "user", "content": prompt},
],
},
)
response.raise_for_status()
data = response.json()
usage = data.get("usage", {})
self.spent += usage.get("total_tokens", self.estimate(prompt))
return data["choices"][0]["message"]["content"]
Enter fullscreen mode Exit fullscreen mode
The fit method is the safety valve. It halves the text until it fits. It never guesses about the reply size.
Step 3 — Run it locally
python - <<'PY'
from budget import TokenBudget
tb = TokenBudget(limit=2000)
text = open("README.md").read() * 10
print(tb.summarize(text))
print("spent:", tb.spent)
PY
Enter fullscreen mode Exit fullscreen mode
Verification: the output is three sentences. The spent value is below 2000. Use a real article for the first real run, not a README.
If the script raises “Token budget exhausted”, the truncation path is working. That is a pass, not a failure. A budget you cannot hit is not a budget.
Step 4 — Deploy to the free server
Create server.py with the standard library only. No FastAPI. No uvicorn. No extra install step.
# server.py
import json
import os
from http.server import BaseHTTPRequestHandler, HTTPServer
from budget import TokenBudget
tb = TokenBudget(limit=int(os.environ.get("TOKEN_LIMIT", "2000")))
class Handler(BaseHTTPRequestHandler):
def do_POST(self):
length = int(self.headers.get("Content-Length", 0))
payload = json.loads(self.rfile.read(length))
try:
summary = tb.summarize(payload["text"])
self.send_response(200)
self.end_headers()
self.wfile.write(json.dumps(
{"summary": summary, "spent": tb.spent}
).encode())
except Exception as exc:
self.send_response(429)
self.end_headers()
self.wfile.write(json.dumps({"error": str(exc)}).encode())
def log_message(self, *args):
pass
HTTPServer(("0.0.0.0", 8000), Handler).serve_forever()
Enter fullscreen mode Exit fullscreen mode
Push the folder to the free server. The exact deploy command is in the current dashboard. The pattern is always the same: upload the folder, set the environment variables, run python server.py.
Verification:
curl -s -X POST https://<your-free-server>/ \
-H "Content-Type: application/json" \
-d '{"text": "Paste a long article here and watch the summary appear."}'
Enter fullscreen mode Exit fullscreen mode
Expect JSON with a summary and a spent value. If you get a 429 with “Token budget exhausted”, the endpoint is alive and honest. The response includes spent on purpose. You can graph it later or ignore it now.
Step 5 — Add a budget regression check
# verify_budget.py
from budget import TokenBudget
tb = TokenBudget(limit=100)
long_text = "word " * 10_000
fitted = tb.fit(long_text)
assert tb.estimate(fitted) <= 100, "budget not enforced"
assert tb.estimate(fitted) > 0, "empty prompt"
print("budget check passed:", tb.estimate(fitted), "tokens")
Enter fullscreen mode Exit fullscreen mode
Run it:
python verify_budget.py
Enter fullscreen mode Exit fullscreen mode
Add this file to your repo. Future you will thank present you. This check runs without any network call.
When the free tier is enough
Situation Free tier Paid tier Weekend prototype Yes No Internal tool, low traffic Yes Maybe Production traffic No Yes Strict data residency Check first Check firstThe free tier is a starting line. It is not a finish line.
Limitations
- The 10 million token figure is current at the time of writing. It will change.
- The token estimate is a heuristic, not a tokenizer. Real tokenizers are more accurate.
- The standard-library server is single-threaded. One slow request blocks the next.
- Free servers may sleep or restart. Do not store state on disk.
- The API shape may differ from this example. Read the current docs.
- Rate limits exist. I did not measure them here.
Who should not use this
- Teams with compliance rules about data location
- Apps that need sub-second latency under load
- Anyone who needs a production SLA
Free tiers do not offer SLAs. Plan accordingly.
Run the five steps this weekend
If the budget check fails, the tutorial is working as intended. The point is not the free stuff. The point is a repeatable path from idea to deployed endpoint.