Dev-Code 구축: RAG 메모리 및 VS 코드 통합을 갖춘 에이전트 AI 코딩 어시스턴트

I recently completed Dev-Code, an AI coding assistant project built around Agentic AI, RAG-style memory, and developer-tool integration.

GitHub repo:

Dev-Code

Dev-Code is an A2A ReAct debugging agent for Python and JavaScript code. It runs broken code in a local sandbox, analyzes the error, searches memory and web context, generates a fix with Gemini when needed, verifies the fix, and logs the full trace for review.

What It Does

• Runs Python and JavaScript code through a guarded subprocess sandbox.
• Detects supported languages through a registry and can learn new one-liner runners from web search.
• Chunks large files with tree-sitter (306 supported languages) so only the relevant function/class is sent to the Fixer, then splices the fix back into the full file.
• Shows a visible ReAct-style trace with specialist agents.
• Uses an A2A flow: OrchestratorAgent, AnalyzerAgent, FixerAgent, and VerifierAgent.
• Stores verified fixes in Supabase-backed vector memory, using Gemini embeddings with asymmetric retrieval (separate document/query embedding modes) and a cosine-similarity threshold to reject superficially-similar but unrelated past fixes.
• …

Dev-Code runs broken Python or JavaScript code, analyzes the error, searches memory for previous verified fixes, generates a new fix when needed, verifies the result, and logs the full trace.

The goal was not just to build another AI wrapper around a model. The goal was to explore what it takes to make an AI assistant behave more like a real developer tool.

The Core Idea

Many AI coding demos work like this:

Code -> LLM -> Suggested fix

Enter fullscreen mode Exit fullscreen mode

Dev-Code uses a more structured workflow:

Broken code
  -> run the code
  -> analyze the error
  -> search fix memory
  -> verify remembered fixes
  -> generate a fix if memory fails
  -> verify the generated fix
  -> save successful fixes for future use

Enter fullscreen mode Exit fullscreen mode

That flow matters because confident text is not the same thing as a validated fix.

Agentic AI Architecture

Dev-Code uses a multi-agent A2A ReAct-style pipeline:

• VerifierAgent runs the code and checks whether the issue is fixed.
• AnalyzerAgent reads the runtime error and extracts useful debugging context.
• OrchestratorAgent coordinates memory search, model usage, verification, and logging.
• FixerAgent calls Gemini when a remembered fix is not enough.

Splitting the system this way made it easier to reason about the workflow. Each agent has a focused responsibility instead of one large prompt trying to do everything.

RAG-Style Fix Memory

One of the most useful parts of the project is the fix memory.

When Dev-Code successfully verifies a fix, it stores that fix in Supabase-backed vector memory using Gemini embeddings. On future runs, it searches memory before calling the model again.

The idea is simple:

Failure -> verified fix -> memory -> future retrieval -> faster debugging

Enter fullscreen mode Exit fullscreen mode

This helped me understand that RAG is not just about retrieving documents. For coding tools, retrieval needs to be tied to validation.

A remembered fix should not be trusted only because it looks similar. It should be tested.

A Real Memory Similarity Bug

One important bug appeared in the memory retrieval layer.

The system needed to avoid two problems:

• missing previous fixes that should have matched,
• accepting superficially similar but unrelated fixes.

The fix involved improving the retrieval logic around asymmetric embedding usage and cosine-similarity thresholding. Dev-Code now uses separate document/query embedding modes and rejects weak matches below the configured threshold.

The lesson was clear: retrieval quality depends on embedding strategy, similarity scoring, thresholds, and verification. RAG systems can fail quietly if these pieces are not handled carefully.

Code Chunking for Larger Files

Large source files create another problem. Sending an entire file to a model can be noisy and inefficient.

Dev-Code uses tree-sitter-based chunking to identify relevant functions or classes, send the useful section to the Fixer, and splice the corrected chunk back into the full file.

This makes the assistant more practical because it can work with larger code while keeping the model context focused.

VS Code and MCP Integration

Dev-Code includes a VS Code extension and a local MCP server.

The extension lets a user select broken code in the editor, run the debugging workflow, and inspect the full agent trace in a side panel.

The MCP server exposes tools such as:

• run_python_code
• search_python_error
• search_fix_memory
• debug_code

This was one of the most important parts of the project because developer tools are most useful when they fit into the developer’s existing workflow.

Production-Level Lessons

The most valuable learning did not come from the happy path. It came from debugging real issues.

1. Verification Is Essential

An AI-generated answer is not enough. Dev-Code verifies fixes by running the code again.

That turns the assistant from “the model suggests this” into “the system has evidence this works.”

2. User Environments Are Complicated

One real issue involved Python runner resolution. The VS Code extension needed to use the correct Python interpreter instead of relying on a fragile PATH lookup.

This is the kind of issue that appears when a project moves beyond a controlled demo and starts behaving like a local developer tool.

3. Documentation Can Be the Right Decision

A related JavaScript PATH risk was documented as a known limitation instead of being rushed into an unproven fix.

That was a useful production-level lesson. Not every theoretical risk should immediately become code. Sometimes the better decision is to validate, document, and prioritize based on evidence.

4. Fallbacks Matter

Dev-Code logs runs to MySQL when available, but falls back to local JSONL logs if the database is unavailable.

That fallback keeps the project usable even when optional infrastructure is missing.

Tech Stack

The project uses:

• Python
• FastAPI
• Streamlit
• Gemini
• Supabase vector memory
• MySQL
• JSONL fallback logging
• tree-sitter
• MCP
• TypeScript
• VS Code extension APIs

What I Learned

This project taught me that building AI developer tools is not only about prompting.

The hard parts are:

• designing the agent workflow,
• retrieving the right context,
• validating model output,
• handling local execution,
• preserving useful traces,
• supporting imperfect user environments,
• documenting known limitations honestly.

Dev-Code is complete as a project, but the bigger lesson is this:

AI coding assistants become much more useful when they are grounded in memory, verification, integration, and production-level engineering judgment.