Stop Dumping Your Entire Repository Into AI

One of the strangest workflows in modern software development looks like this:

1. Ask ChatGPT a question.
2. Realize it doesn’t have enough context.
3. Copy a few files.
4. Add architecture notes.
5. Paste recent changes.
6. Explain what you’re trying to do.
7. Hope you included the right information.

The problem isn’t the AI.

The problem is context retrieval.

After building TokenCap’s Knowledge Graph, Debug Handoff, Change Intelligence, and Context Packing systems, I realized something:

We had become very good at generating context.

We were still terrible at finding it.

That’s what led to TokenCap v0.7 — Smart Retrieval Engine.

The Question

Instead of asking:

Which files should I send to the AI?

What if you could simply ask:

tokencap ask "How does authentication work?"

Enter fullscreen mode Exit fullscreen mode

and let the tool figure out the rest?

How It Works

The new retrieval engine searches across multiple sources of project intelligence:

• Knowledge Graph relationships
• Dependency chains
• Project Memory
• Recent code changes
• Risk analysis
• Impact scores

It doesn’t just find matching files.

It finds the files that matter.

Beyond Keyword Search

A search for:

tokencap ask "debug login redirect"

Enter fullscreen mode Exit fullscreen mode

automatically switches into debug mode.

A search for:

tokencap ask "review payment flow"

Enter fullscreen mode Exit fullscreen mode

switches into review mode.

A search for:

tokencap ask "explain dashboard architecture"

Enter fullscreen mode Exit fullscreen mode

switches into architecture mode.

The retrieval strategy changes based on what you’re trying to achieve.

Following the Connections

One thing I wanted to avoid was returning isolated files.

Real systems are connected.

If a file is relevant, its dependencies are often relevant too.

TokenCap traverses the graph to find related files, dependents, and neighboring modules before building the final context package.

Context Isn’t Free

Even after finding the right files, another challenge remains:

Token budgets.

The retrieval engine reuses TokenCap’s Context Packing system to decide what deserves:

• Full source code
• Structural outlines
• Summaries
• References only

This allows larger repositories to fit into practical AI context limits without losing important information.

From Context Generation to Context Retrieval

The evolution of TokenCap has been interesting:

• v0.1 — Project Snapshot
• v0.2 — Knowledge Graph & Context Memory
• v0.3 — Debug Handoff
• v0.4 — Change Intelligence
• v0.5 — Context Packing
• v0.6 — Graph Intelligence
• v0.7 — Smart Retrieval

For the first six versions, the focus was:

Generate better context.

With v0.7, the focus becomes:

Retrieve the right context.

That’s a subtle difference.

But I think it’s the difference between AI tools that know code and AI tools that understand projects.

Try It

Website: tokencap.vansharora.app

NPM: https://www.npmjs.com/package/tokencap

npm install -g tokencap

Enter fullscreen mode Exit fullscreen mode

Then ask your repository a question:

tokencap ask "How does authentication work?"

Enter fullscreen mode Exit fullscreen mode

Instead of manually assembling context, let the repository tell the AI what matters.

Feedback, ideas, and contributions are always welcome.