Why I Stopped Using Vector RAG for Coding Agents (And Used Git Markdown Instead)

If you use Cursor, Claude Code, or Windsurf daily, you’ve probably hit this wall:

You spend 45 minutes explaining your architecture, your API contracts, and why you never use a certain library pattern. The agent gets it, writes great code, and you finish the feature.

Next morning, in a fresh session:

“Let’s implement this! I will use [the exact pattern you ruled out yesterday] and rebuild [a helper function that already exists].”

It feels like babysitting a brilliant junior engineer with amnesia.

The Two Usual “Solutions” (And Why They Break)

1. The Giant CLAUDE.md / .cursorrules Dump

The first instinct is dumping every rule, schema, and architectural pattern into a project instructions file.

• The Problem: It murders your baseline token usage. Every single prompt sends 4,000+ tokens of context before you even type “fix this bug”. Worse, models suffer from context dilution—when everything is in the prompt, nothing is prioritized.

2. Vector DBs & Semantic RAG

The second instinct is building or adopting a vector-based memory tool (embedding chunks of past chats/code).

• The Problem: Semantic similarity $\neq$ authoritative truth. If you refactored an auth flow three times, a vector search for “how does auth work” will retrieve all three versions with near-identical similarity scores. The agent cannot distinguish between a deprecated experiment and today’s standard.

What Actually Works: Deterministic, Topic-Scoped Markdown via MCP

Instead of guessing via embeddings or overloading the system prompt, the cleanest architecture treats agent memory like codebase documentation:

.opencontext/
├── architecture.md
├── api-contracts.md
├── state-management.md
└── rejected-approaches.md

Enter fullscreen mode Exit fullscreen mode

How it works under the hood:

[New Coding Session]
│
▼

1. Agent queries MCP index (~100 tokens)
("Available topics: architecture, api-contracts, state-management...")
│
▼
2. Agent fetches ONLY what it needs for the current task
(e.g., read_context("api-contracts"))
│
▼
3. Agent writes code adhering to exact invariants
│
▼
4. Architectural change? Agent mutates the markdown file in-place

Enter fullscreen mode Exit fullscreen mode

The 3 Core Principles

1. Index-First Token Gating

Instead of dumping 15 KB of architectural docs into the system prompt, the agent starts every session with a tiny, auto-generated index:

{
  "topics": ["auth-flow", "error-handling", "database-conventions"],
  "total_files": 3
}

Enter fullscreen mode Exit fullscreen mode

Cost: ~100 tokens. The model calls read_context("auth-flow") only if the prompt touches authentication.

2. In-Place Mutation over Infinite Append

Vectors fail because they append indefinitely. With topic-scoped markdown files, the agent updates the existing document when an architectural decision changes. There are no competing versions in vector space.

3. Git as the Universal Sync Layer

Because memory is just flat .md files inside the repository:

• Zero Cloud Lock-in: No API keys, no external vector databases, no extra monthly subscriptions.
• PR Reviews for AI Memory: When an agent updates .opencontext/api-contracts.md, it shows up directly in your GitHub pull request diff.
• Cross-Agent Parity: You can start a task in Claude Code, commit your branch, open Cursor, and Cursor’s MCP server immediately reads the exact same context.

Example: Setting Up Local MCP Memory

You can connect a lightweight local MCP server to Cursor or Claude Code in seconds.

In your claude.json or Cursor MCP settings:

{
  "mcpServers": {
    "opencontext": {
      "command": "npx",
      "args": ["-y", "opencontext-mcp"]
    }
  }
}

Enter fullscreen mode Exit fullscreen mode

Now, instead of re-explaining constraints, you can simply tell your agent:

“Check the database conventions in our context and scaffold the new user billing schema.”

The agent checks the index, reads .opencontext/database-conventions.md, adheres to your patterns, and moves on.

Conclusion & Discussion

We don’t need complex vector pipelines for single-repository agent memory.

Plain text in Git has been the source of truth for software engineering for 20 years. Giving agents deterministic read/write access to structured markdown via MCP solves context persistence without the overhead.

How are you currently preventing your coding agents from losing context across sessions? Are you sticking with static rule files, using RAG, or building internal tools?