How I Built a $0 LLM Production Stack with 46 Free APIs

How I Built a $0 LLM Production Stack with 46 Free APIs

And why every “free LLM API list” goes stale within a week

The Problem

I needed LLMs for a side project but couldn’t justify $50-500/month for API costs. Every “free LLM API” list I found had the same problems:

1. Stale within days — providers change limits, add credit card requirements, deprecate models
2. Markdown tables — not machine-readable, can’t plug into code
3. No verification — no way to know if a provider actually works today
4. No deployment path — just a list, no config for Hermes/LiteLLM/Portkey

So I built free-llm-atlas: 46 free LLM API providers, auto-probed daily, structured JSON + gateway configs.

The Zero-Cost Stack

After testing 40+ endpoints, my production fallback chain:

Layer 1 (Speed):     Groq — 300+ tok/s, 14.4K req/day
Layer 2 (Multimodal): Google AI Studio — 2M context, vision/audio, 1.5K req/day
Layer 3 (Reasoning): NVIDIA NIM — Nemotron Ultra 1M ctx, function calling, 40 RPM
Layer 4 (Chinese):   Z.AI GLM-4 — 1M ctx, strong Chinese, 60 RPM

Enter fullscreen mode Exit fullscreen mode

Covers 95% of workloads at $0.

The 17 Permanent Free Providers (No Credit Card)

Provider Best For Rate Limit Context Groq Speed 14.4K req/day 131K Google AI Studio Multimodal / Long context 1.5K req/day 2M NVIDIA NIM Reasoning / Function calling 40 RPM 1M Cerebras Extreme speed 2.6K tok/s 1M Cloudflare Workers AI Edge inference 10K neurons/day 128K Cohere RAG / Embeddings 1K req/month 16K Mistral EU data residency 1 RPS 32K HuggingFace Model variety $0.10/mo credits Varies GitHub Models GPT-5, o4-mini free 150 req/day 128K OpenRouter 14 free models 50 RPD 1M (Nemotron) Nebius New free tier 100 RPM 128K OVHcloud EU GDPR Anonymous 2 RPM 4K Inference.net New permanent free Unknown Unknown LLM7.io Claude/GPT access 30 RPM 200K Requesty Router 200 RPM Varies Z.AI Chinese 60 RPM 1M Coze Agent platform 100/day 128K

Why Daily Probing Matters

Static lists rot. Providers:

• Change rate limits (Groq dropped from 14.4K → 1K RPD in 2026)
• Add credit card requirements overnight
• Deprecate models without notice
• Shut down endpoints silently

free-llm-atlas runs GitHub Actions daily at 06:00 UTC — every provider, every endpoint, every day.

probe.py → test /models + /chat/completions → measure latency, success, tokens/sec, context
→ update providers.json → commit if changed → Git history = uptime dashboard

Enter fullscreen mode Exit fullscreen mode

Usage: 30 Seconds to Production

git clone https://github.com/happyyboxx/free-llm-atlas
cd free-llm-atlas

# Find free providers needing NO credit card
python3 -c "
import json
d = json.load(open('data/providers.json'))
for p in d['providers']:
    if p['tier']=='permanent_free' and not p.get('requires_card'):
        print(f'✅ {p[\"name\"]}: {p.get(\"rate_limit\",\"N/A\")}')
"

# Probe all 46 providers
pip install httpx pyyaml
python3 scripts/probe.py --all

# Export gateway config
python3 scripts/probe.py --export-config litellm > config.yaml

Enter fullscreen mode Exit fullscreen mode

Gateway Configs: Drop-in Replacement

# Auto-generated litellm.yaml
model_list:
  - model_name: llama-3.1-70b-groq
    litellm_params:
      model: groq/llama-3.1-70b-versatile
      api_base: https://api.groq.com/openai/v1
      max_tokens: 8192
    fallback: [gemini-flash, nim-nemotron, z-ai-glm]

Enter fullscreen mode Exit fullscreen mode

One command switches entire stack. Zero code changes when a provider fails.

The Hidden Limits Nobody Talks About

Everyone compares daily request limits. The real bottleneck is tokens/minute (TPM):

Provider Daily Req TPM Real Limit Groq 14,400 6,000 Concurrency: ~12 msg/min Together 100 100,000 Daily limit hits first NIM ~2,400 40 RPM Batch-friendly Google AI Studio 1,500 1,000,000 Effectively unlimited OpenRouter (free) 50 20 RPM Very low

TPM determines concurrency, not daily requests. Match provider to YOUR bottleneck.

GitHub Actions = Free Infra

The probe runs on GitHub Actions (free tier):

• 0 infrastructure cost
• Runs daily at 06:00 UTC
• Updates providers.json with live status
• Generates gateway configs
• Commits changes → Git history = uptime dashboard

Zero maintenance. Zero cost.

Contributing

The project aggregates from:

• cheahjs/free-llm-api-resources (28K+ ⭐)
• tashfeenahmed/freellmapi (17K+ ⭐)
• mnfst/awesome-free-llm-apis (6K+ ⭐)
• nejib1/Free-LLM

PRs welcome for new providers, probe fixes, or doc improvements.

Try It

git clone https://github.com/happyyboxx/free-llm-atlas
cd free-llm-atlas
python3 scripts/probe.py --all
python3 scripts/probe.py --export-config litellm

Enter fullscreen mode Exit fullscreen mode

Star ⭐ if this saves you money on LLM inference.

Built because I was tired of paying for inference during development. Now my entire LLM stack costs $0.