I Built an AI Router That Decomposes Your Prompt and Dispatches to the Best Model

When you call an LLM API, you pick a model and hope it’s the right one. Claude for reasoning, GPT for code, Gemini for long context… you’re doing the routing manually.

I got tired of this, so I built Flint — a smart routing engine that does the model selection for you.

How It Works

Instead of forwarding your prompt to a single model, Flint:

1. Decomposes — analyzes your prompt and identifies which capability domains it needs (research, analysis, code, content, math…)
2. Dispatches — routes each sub-task to the best model for that domain
3. Executes — runs them in parallel
4. Synthesizes — weaves the results into one coherent answer

You: "Compare Python and Go for an API gateway"
         ↓
   [Decompose]
    /         Research    Analysis
(qwen3.7)   (deepseek)
    \         /
   [Synthesize]
         ↓
   Structured comparison with performance data

Enter fullscreen mode Exit fullscreen mode

Real Example

Here’s what Flint produced for “Compare Python and Go for building an API gateway”:

1. Performance — Go compiles to native code, handles tens of thousands of concurrent connections. Python’s GIL limits throughput even with async frameworks.

2. Concurrency — Go’s goroutines (~4KB stack) outperform Python’s asyncio by an order of magnitude for I/O-heavy workloads.

3. Ecosystem — Go dominates the gateway space (Traefik, Caddy, Kong plugins). Python has frameworks but lacks dedicated gateway projects.

The research subtask fetched actual benchmark data, the analysis subtask structured the comparison, and the synthesizer wove it together. One prompt, multiple models, one answer.

The Stack

• Backend: FastAPI + SQLite, deployed on a $20 VPS
• Router engine: Custom Python pipeline — decompose, dispatch, execute, verify, synthesize
• Models: DeepSeek V4, Qwen3.7, Kimi K2, GLM-5, plus 30+ others through CloudWalk
• API: OpenAI-compatible — change one base_url and keep your existing code

Try It

There’s a live demo — no signup needed. Type any question and see Flint’s routing vs. a direct model call side by side.

• Demo: flintapi.ai/demo
• API: pip install openai + set base_url="https://flintapi.ai/v1"
• Code: github.com/moozechen/flintapi

Open to feedback — what would you want a smart router to do that it doesn’t do yet?