실행할 사내 AI 가시성 추적 비용

The cost of collecting AI answers in-house is rarely the API bill you avoid. It is the engineering time to reach production quality on each engine, plus the maintenance that follows for as long as you run it. Building is the right call when the collection layer is itself your product or you need data no provider returns; the build vs buy comparison covers that decision, and the framework below fills in the numbers behind it.

Definitions

• Collection layer: everything between “we have a prompt” and “we have a structured record of what the engine answered”: execution, parsing, retries and storage.
• Analysis layer: what you build on top: share of voice, citation tracking, competitor reports, alerts. This is where customers compare products.
• Run: one prompt, on one engine, in one market, collected once.

What building involves

Consumer AI products such as ChatGPT, Perplexity, Gemini, Copilot, Grok and Google AI Mode show answers that include web search, citations, product cards and local results. Those surfaces live in the consumer applications, not in the model providers’ developer APIs. Each of the components below exists once per engine.

1. Collection runs

You run automated collection against each product and keep throughput inside what each product allows. That is infrastructure to provision, scale, patch and monitor, sized for your peak run.

2. Parsing that keeps up with interface changes

Each engine renders answers differently: inline citations, grouped sources, shopping cards, maps, follow-up queries. Each needs its own parser, a fixture set, and tests. When an interface changes, the failure can be silent: the run succeeds and a field such as sources comes back empty. Catching that takes data-quality monitoring per field, per engine.

3. Market coverage

Answers differ by country, and for local intent by region. Reporting per market requires collecting from each market and validating that the stored answers reflect it.

4. Queueing, retries and storage

At volume, partial failure is the normal case. You need a queue with priorities, retries that cannot create duplicates, per-run deadlines, result storage and a way to push results to downstream systems. That is a distributed-systems project in its own right.

5. Terms, compliance and review

Each consumer product has its own terms of service. Collecting from them at volume is a decision for your legal and compliance review, per engine, before engineering starts, and again when terms change.

6. On-call

Interfaces change without notice and on no schedule. Somebody owns the pager for each engine.

A cost framework you can fill in

Team cost, engine count and quality bar vary too much for universal numbers. Fill in this table with your own.

Variable Meaning Your number E Engines you need B Engineer-months to reach production quality on one engine M Engineer-months of maintenance per engine, per month C Loaded cost of one engineer-month I Monthly infrastructure: compute, storage, monitoring R One-off review and compliance cost T Months you plan to run it

Then:

• Build, cost over T months = E × B × C + E × M × C × T + I × T + R
• Buy, cost over T months = the monthly price of the plan that covers your credits (below) × T

Adjust for three effects the formulas miss:

1. Build is paid before value. E × B months pass before the first reliable dataset; for a product that competes on time to market, that delay has its own cost.
2. Maintenance scales with engines, not volume. A fifth engine adds a fifth parser and a fifth pager rotation, even if you run few prompts on it.
3. Opportunity cost. Every engineer-month in the collection layer is one not spent on the analysis layer.

Computing the buy side

Buying is priced in credits per request. As async tasks, the base price per request is:

Engine Credits per request ChatGPT 5 Gemini 4 Copilot 5 Google AI Mode 4 Google Search 3 Google News 2

Perplexity and Grok are paused for now and coming back soon; requests to them are refused and not billed. Synchronous calls add 2 credits each. Some options add to the price: on ChatGPT, +2 once if include.rawResponse or shopping is on (searchQueries and ads are free); on AI Mode, +1 per expanded product cluster, up to 6; on Google Search, +2 per extra page and +2 once for the AI Overview; on Google News, +2 per extra page. Credits are reserved at the maximum and charged at the actual cost on success; failed requests are charged nothing, and queued tasks you clear are never charged. The details are in credits.

Worked example 1: a large program

20,000 prompts on ChatGPT and Gemini in the US, plus 2,000 Google News queries in 10 markets:

• Runs per sweep: 20,000 × 2 + 2,000 × 10 = 60,000.
• Credits for the assistants: 20,000 × (5 + 4) = 180,000.
• Credits for Google News: 2,000 × 10 × 2 = 40,000.
• Credits per full sweep, async, no add-ons: 180,000 + 40,000 = 220,000.

A weekly sweep (4 a month) is 880,000 credits, inside the Growth plan’s 1,350,000. A daily sweep is 6,600,000 a month and needs an Enterprise plan. Plan prices and a volume estimator are on the pricing page.

Worked example 2: a focused program

200 prompts on ChatGPT and Gemini in the US, plus 50 Google News queries in 3 markets, weekly (4 runs a month):

• Credits per prompt: 5 + 4 = 9.
• Per sweep: 200 × 9 + 50 × 3 × 2 = 1,800 + 300 = 2,100 credits.
• Per month: 2,100 × 4 = 8,400 credits, inside the Lite plan’s 37,500.

Unused monthly credits do not roll over, so size the plan to your steady cadence, not your peak. The cost planning guide covers sampling and cadence choices that change these numbers, and sampling vs census covers when you can run fewer prompts without losing the signal.

What the buy side returns

Compare the output as well as the price. Each engine returns structured JSON under result, with fields that differ by engine:

• All AI engines: text, sources[], and markdown with include.markdown.
• ChatGPT: citationPills[], entities[], shoppingCards[] (with include.shopping), map[]. The searchQueries[] (free with include.searchQueries), inlineProducts[] (with include.shopping) and ads[] (with include.ads) fields exist but come back empty on the answers served today.
• Gemini: citationPills[], places[].

For volume, tasks go through POST /v1/async/task/batch (up to 500 per request) with idempotency keys and webhooks. Retries, deadlines and queueing are part of the API contract rather than code you maintain; see async tasks.

What you still build when you buy

Buying the collection layer moves engineering to the parts that differ between products:

• Prompt set and schedule. Which prompts, engines and markets, and how often. Prompt set design and monitoring cadence cover the choices.
• Submission and retries. Batching, idempotency keys derived from each task’s meaning, and a retry policy that never duplicates work; see idempotency keys.
• A webhook receiver. Signature verification over the raw body, deduplication by task.id, and storage; see verifying webhook signatures.
• Normalization. Field names differ by engine (shopping_cards on Perplexity, shoppingCards on ChatGPT; citation ids 0-based on ChatGPT and 1-based elsewhere), so map them into your own schema once.
• The analysis layer. Mentions, citations, share of voice and reports, which is what customers pay for.

SDKs and workflow integrations are listed under integrations; for AI agents, the MCP server exposes each engine as a tool.

Decision criteria

Situation Lean Answers feed an analytics, SEO or visibility product Buy You need several engines and several markets Buy Time to first reliable dataset matters Buy Collection itself is the product you sell Build You need a field no provider returns Build that part, scoped to one engine One engine, low volume, and spare engineering capacity Either; compare with the framework

A hybrid also works: buy the engines and markets a provider covers, and build narrowly where you need something specific.

How to evaluate in a week

1. Pick your ten hardest prompts: the ones with shopping, local or citation-heavy answers that break parsers.
2. Run them on every engine you need. The free tier (500 credits, once, no card) works on every live engine, so you can evaluate the assistants before paying: 10 prompts as synchronous calls on ChatGPT and Gemini cost 10 × (9 + 2 × 2) = 130 credits. As async tasks, 10 × 9 = 90.
3. Compare field by field against what your pipeline or manual checks show: citations present, ordering, product cards, places.
4. Check the failure path: send a malformed request and a burst of requests, and confirm your code handles 422, 429 and batch item errors as described in the errors post.
5. Price your volume with the credit math above and your cadence.
6. Fill in the build table with your own team’s estimates and compare.

Pitfalls in the comparison

• Comparing one engine built to several engines bought. Price the scope you need.
• Ignoring maintenance. First-version cost is the smaller term over any horizon longer than a few months.
• Treating developer model APIs as equivalent. A completion from a model API does not include the consumer app’s web search, citations or product cards; see provider APIs.
• Pricing at peak cadence. Credits do not roll over; plan for the steady state.
• Skipping the terms review. It belongs at the start of a build plan, not after launch.

The table version of this comparison is at build vs buy; to run your first prompts, start with the quickstart.