I Built an OpenAI-Compatible Gateway to Control LLM Costs

As an indie developer building AI products, I kept hitting the same wall: how do I control LLM costs without vendor lock-in?

Direct API keys meant zero visibility into per-project spending. Month-end surprise bills. No way to cap costs per feature. Rate limit headaches.

So I built CostLLM — an OpenAI-compatible API gateway that sits between your code and the LLM provider.

What it does

• Virtual API Keys — Create separate keys per project, environment, or team member
• Usage Tracking — Real-time token consumption dashboard
• Budget Controls — Hard caps prevent overspend
• Rate Limiting — Configurable per key
• Drop-in Compatibility — Works with any OpenAI SDK

One line change

# Before
client = OpenAI(api_key="sk-proj-...")

# After
client = OpenAI(
    base_url="https://api.costllm.ai/v1",
    api_key="YOUR_COSTLLM_API_KEY"
)

Enter fullscreen mode Exit fullscreen mode

Built with

• Python (FastAPI) gateway + backend
• Redis for rate limiting
• PostgreSQL for user & usage data
• Nginx reverse proxy
• Creem for payments

Supported models

Model Type deepseek-v4-flash Fast chat deepseek-v4-pro Complex reasoning deepseek-chat General purpose deepseek-reasoner Deep analysis

What I learned

Building a production API gateway taught me a lot about concurrency, streaming, error handling, and payment webhooks. The hardest part wasn’t the code — it was designing a pricing model that works for both free users and growing teams.

Try it

• Website: https://costllm.ai
• API docs: https://documenter.getpostman.com/view/56610873/2sBY4MtfzU
• Free tier: $10 signup credit, no credit card required

I’d love feedback from other developers working with LLM APIs — especially around cost visibility, key management, and what you’d want from a gateway.