A research agent is a loop with three tools and two things that must not live in the prompt: the budget that stops it, and the check that every citation points at a page it actually read. Put those in the prompt and they are suggestions. Put them in code and the agent is something you can leave running.
The loop, and where the budget goes
The loop itself is small. Every agent is the same loop: call the model with the conversation and the tool definitions, if it asks for a tool run it, append the result, repeat until it answers or you stop it.
# agent.py
MAX_STEPS = 8 # model turns
MAX_SEARCHES = 4
MAX_FETCHES = 6
MAX_TOTAL_TOKENS = 60_000
def research(question):
budget = Budget(MAX_STEPS, MAX_SEARCHES, MAX_FETCHES, MAX_TOTAL_TOKENS)
messages = [{"role": "system", "content": SYSTEM},
{"role": "user", "content": question}]
sources = {} # id -> {url, title, text}
while budget.step():
res = post("/chat/completions", {
"model": MODEL, "temperature": 0,
"messages": messages, "tools": TOOLS,
})
budget.charge(res.get("usage"))
msg = res["choices"][0]["message"]
messages.append(msg)
calls = msg.get("tool_calls") or []
if not calls:
return finalise(msg["content"], sources)
for call in calls:
out = dispatch(call, budget, sources)
messages.append({"role": "tool",
"tool_call_id": call["id"],
"content": out})
# Budget exhausted: force an answer from what we have.
messages.append({"role": "user", "content":
"Budget exhausted. Answer now using only the sources already "
"retrieved. Say plainly what you could not establish."})
res = post("/chat/completions", {"model": MODEL, "temperature": 0,
"messages": messages})
return finalise(res["choices"][0]["message"]["content"], sources)
Enter fullscreen mode Exit fullscreen mode
The tool-calling request and response fields — tools, tool_calls, the tool role — follow the OpenAI-compatible shape that gateways and most providers implement. Other providers use different names for the same three ideas. Check the reference for the API you are calling; the loop above does not change, only the field names do.
The final forced answer is the part that turns a budget from a failure into a feature. An agent that hits its ceiling and returns an error has wasted everything it spent; one that is told to answer from what it has and to name the gaps returns something useful nine times out of ten.
Three tools, no more
Tool Description search(query) Returns 8 results: title, url, snippet. No page content. Cheap. fetch(url) Returns extracted main text, truncated, with a source id assigned by your code. quote(source_id, needle) Returns the surrounding paragraph if the needle is in that source, else an error. This is what makes citations honest.Resist adding more. Tool selection accuracy degrades as the tool list grows, and every tool description is tokens on every turn. Three tools with careful descriptions beat nine with vague ones.
The search tool is the one place you need a third-party service — a search API, or your own index. Whichever you pick, wrap it so the agent sees a fixed shape: a list of {title, url, snippet} objects. Then changing provider is one function, and the agent’s behaviour does not shift under it.
The budget as code, not as a prompt
“Use at most four searches” in a system prompt is followed most of the time, which is the worst possible reliability for a spending control. Enforce it in the dispatcher and tell the model what happened.
class Budget:
def __init__(self, steps, searches, fetches, tokens):
self.steps, self.searches = steps, searches
self.fetches, self.tokens = fetches, tokens
self.used_tokens = 0
def step(self):
if self.steps <= 0 or self.used_tokens >= self.tokens:
return False
self.steps -= 1
return True
def charge(self, usage):
if usage:
self.used_tokens += usage.get("total_tokens", 0)
def dispatch(call, budget, sources):
name = call["function"]["name"]
args = json.loads(call["function"]["arguments"] or "{}")
if name == "search":
if budget.searches <= 0:
return ("SEARCH BUDGET EXHAUSTED. Do not search again. "
"Work with the sources you already have.")
budget.searches -= 1
return json.dumps(search_api(args["query"])[:8])
if name == "fetch":
if budget.fetches <= 0:
return "FETCH BUDGET EXHAUSTED. Answer from retrieved sources."
if not allowed(args["url"]):
return "REFUSED: host not permitted."
budget.fetches -= 1
sid = "S" + str(len(sources) + 1)
page = extract(args["url"])
sources[sid] = page
return ("source_id=" + sid + " title=" + page["title"] + "\n\n"
+ page["text"][:8000])
if name == "quote":
src = sources.get(args["source_id"])
if not src:
return "NO SUCH SOURCE. You may only quote sources you fetched."
return (find_paragraph(src["text"], args["needle"])
or "NOT FOUND in that source. Do not cite it for this claim.")
return "UNKNOWN TOOL"
Enter fullscreen mode Exit fullscreen mode
Returning a clear message instead of throwing is deliberate. The model reads it, understands the constraint, and adapts — an error message written for the model to read is a design surface, and a stack trace in a tool result wastes tokens and produces confusion.
allowed(url) is the other non-negotiable. An agent that fetches arbitrary URLs will eventually fetch an internal address, a link-local metadata endpoint or a file scheme. Allow http and https only, resolve the host and reject private ranges, and cap the response size.
Fetching a page without drowning in it
A fetched page is the single largest source of token waste in this design. A news article is 1,500 tokens; the HTML around it is 40,000.
- Cap the download. Refuse anything over a couple of megabytes and stop reading at that point rather than after.
- Strip scripts, styles, navigation and footers, then take the text of the largest content block. Extraction quality varies enormously by site, and a bad extraction looks like a bad answer.
- Truncate to a fixed token budget per source — 8,000 characters is about 2,000 tokens and enough for almost any article. Say in the tool result that it was truncated.
- Keep the full text server-side under the source id, so
quotecan search all of it even though the model only saw the first part.
That last point is the quiet win: the model reasons over a summary-sized view and verifies against the whole document, which is cheaper and more accurate than putting everything in the context.
Citations that are checked
Require the answer to carry claims and citations as data, then verify each one before anything is returned.
def finalise(answer, sources):
"""Answer format: claims as 'text [S3]'. Verify every marker."""
used = set(re.findall(r"\[(S\d+)\]", answer))
unknown = used - set(sources)
if unknown:
return {"ok": False, "error": "cited sources never fetched: "
+ ", ".join(sorted(unknown))}
uncited = [s for s in split_sentences(answer)
if is_factual(s) and not re.search(r"\[S\d+\]", s)]
return {
"ok": True,
"answer": answer,
"sources": {sid: sources[sid]["url"] for sid in used},
"uncited_claims": uncited, # surface these, do not hide them
}
Enter fullscreen mode Exit fullscreen mode
The check that a cited source id exists costs nothing and catches the single most damaging failure — a plausible URL the agent never opened. Fabricated citations are the highest-cost hallucination class precisely because they look like diligence.
The stronger check is the quote tool: instruct the agent that any claim it intends to cite must first be confirmed by a successful quote call, and log the ratio of claims to successful quote calls. When that ratio drifts, quality has drifted, and you found out from a counter rather than from a reader.
What one question costs
A typical run inside the budgets above:
turn 1 system + tools + question 1,200 in, 80 out
turn 2 + search results (8 x ~40 tokens) 1,600 in, 90 out
turn 3 + fetched source 1 (~2,000 tokens) 3,800 in, 100 out
turn 4 + fetched source 2 6,000 in, 110 out
turn 5 + fetched source 3 8,300 in, 120 out
turn 6 + quote results 8,800 in, 400 out
TOTAL 29,700 in, 900 out
The context grows every turn because the whole conversation is resent.
That quadratic-ish growth, not the number of calls, is why a step limit
matters: turn 12 of a runaway agent costs several times turn 3.
At $0.15 / $0.60 per million tokens:
29,700/1e6 x 0.15 = $0.0045
900/1e6 x 0.60 = $0.0005
~ half a cent per question, plus search API fees.
Ten thousand questions a month: ~$50 of tokens. The search API is likely
to be the larger line — price it before you design around it.
Enter fullscreen mode Exit fullscreen mode
Prices move; put your own model’s posted rate into the last two lines. The structural point survives any price change: the input side dominates, so trimming what goes back into the context on each turn is where the money is. Cost control for agents is mostly context control.
Per-run cost is the number that decides whether an agent ships, and it is awkward to compute from token counts once several models and a retry path are involved. Multigrid attaches a cost to each request and lets you tag them, so one research run has one number against it rather than a reconstruction from logs.
Making a run reproducible
“It gave a different answer this morning” is the hardest bug in an agent, because by the time you look, the web has changed and so has the search ranking. The fix is to record the run as a trace and make replay possible without any network calls.
# One row per run, one row per event, and the events are enough to replay.
CREATE TABLE run (
id TEXT PRIMARY KEY,
question TEXT NOT NULL,
model TEXT NOT NULL,
prompt_v TEXT NOT NULL,
started REAL NOT NULL,
outcome TEXT
);
CREATE TABLE event (
run_id TEXT NOT NULL,
seq INTEGER NOT NULL,
kind TEXT NOT NULL, -- 'model_call' | 'tool_call' | 'tool_result'
payload TEXT NOT NULL, -- the exact request or response, as sent
tokens INTEGER,
ms INTEGER,
PRIMARY KEY (run_id, seq)
);
Enter fullscreen mode Exit fullscreen mode
Storing the exact request bodies rather than a summary is what makes this worth having. With them, replay is a function that serves stored responses in sequence instead of making calls, so you can change the prompt and see what the model would have done with that evidence — which is the only way to attribute a behaviour change to the prompt rather than to the web. Recording and replaying model interactions is the same technique applied to tests.
- Cache fetched pages within a run by URL, and across runs for a short window. Two turns asking for the same page is common, and the cache makes it free as well as consistent.
- Pin the model, not the alias. A version string moves under you. Behaviour can change without the name changing, and the trace is the only evidence you will have.
- Temperature 0 is not determinism. It reduces variance; it does not eliminate it, and it does nothing about the search results changing. Expecting byte-identical reruns will waste a day.
The three ways it goes wrong
- It searches, searches again with a synonym, and never fetches. Almost always a tool description problem: the search description promises more than it delivers, so the model keeps expecting content in the results. Say explicitly that search returns snippets only and that answering requires a fetch.
- It answers from the snippets. Cheap and often nearly right, which is what makes it dangerous — snippets are chosen by a search engine to match the query, not to be true. Enforce it in
finalise: a claim citing a source id that was never fetched is already rejected by the code above. - It loops on a failing fetch. A site returns 403, the model tries again, and again. Cache failures by URL within a run and return “this URL failed earlier and will not be retried”. A one-line cache; stopping conditions are the difference between an agent and a bill.
답글 남기기