ChatGPT에서 AI 상담원으로: 2022년과 2026년 사이에 실제로 변경된 사항

I recently gave this talk in English to my classmates at an English school in Baguio, the Philippines. Most of them had used ChatGPT. Almost none of them had used an AI agent. And the gap between those two experiences turned out to be much harder to explain than I expected.

The problem is that if you don’t understand how the pieces stack up, an AI agent looks like magic. And when a tool looks like magic, you can’t get good at using it — you just poke at it and hope.

So this is my attempt at the explanation I wish I’d had. It walks through what got invented, in the order it got invented, and what each layer made possible. I’ve traded some technical precision for intuition. The goal is that by the end, nothing an agent does should surprise you.

Written for: anyone who has used ChatGPT but never used a coding agent, and anyone who has heard “LLM,” “RAG,” and “MCP” without ever getting a feel for what they actually are.

The timeline

Here’s the whole story in one view. Each row exists because the row above it wasn’t enough.

When What arrived What it made possible Nov 2022 ChatGPT (GPT-3.5) Answers from what the model already learned 2023 RAG goes mainstream Answers grounded in your documents Jun 2023 OpenAI function calling The model can trigger real actions Sep 2024 OpenAI o1 (reasoning models) Plan first, then execute Nov 2024 Anthropic’s MCP Tool access becomes an open standard 2025 The year of AI agents Autonomous multi-step work Oct 2025 Agent Skills Reusable expertise, written in Markdown 2026 Agents as infrastructure Hosted, scheduled, long-running agents

Now let’s go layer by layer.

1. The LLM itself

A large language model is trained on an enormous amount of text, and it answers by predicting what comes next, one token at a time.

That’s it. That’s the whole trick. When ChatGPT launched in November 2022 and hit a million users in five days, what people were reacting to was a system that had read most of the internet and could talk about it. You ask, and it answers out of what it already absorbed during training.

Two consequences follow immediately, and they explain most of what came next:

It doesn’t know your stuff. Your company’s internal wiki, your codebase, yesterday’s news — none of that was in the training data.

It doesn’t know that it doesn’t know. Because the model generates the most plausible next token rather than looking anything up, it will produce a confident, fluent, completely fabricated answer just as readily as a correct one. This is what people mean by hallucination, and it’s not a bug that got patched — it’s a direct consequence of how the thing works. Everything in the rest of this article is, in one way or another, an attempt to compensate for it.

2. The context window

The context window is how much text the model can hold in view at once — your prompt, the conversation history, any documents you pasted in, and its own answer, all counted together in tokens.

As of mid-2026, the ceiling has moved a lot:

Model Context window Claude Opus 4.8 1M tokens Claude Sonnet 5 1M tokens Gemini 3 Pro 1M tokens GPT-5.2 256K tokens Claude Haiku 4.5 200K tokens

A million tokens is roughly a few thousand pages. That sounds like enough to stop worrying — but it isn’t, for two reasons.

First, a bigger window costs more and runs slower. Every token in the window gets processed on every single turn.

Second, and more importantly: a model with a huge context window still doesn’t attend equally well to everything in it. Bury the one line that matters in the middle of 900,000 tokens of noise and the model may well miss it. More context is not automatically better context.

This is why “just paste in everything” is a losing strategy, and it’s the seed of the idea that closes out this article.

3. RAG: giving the model knowledge it never learned

So the model doesn’t know your stuff. There are two ways to fix that, and they’re often confused with each other.

Option A: fine-tuning. Continue training the model on your data so the knowledge is baked into its weights. This genuinely works, but it’s expensive, it needs a large clean dataset, evaluating whether you made the model better or worse is its own research problem, and the moment your data changes you have to do it again. For “the model should know our internal docs,” it’s almost always the wrong tool.

Option B: RAG (Retrieval-Augmented Generation). Leave the model alone. Keep your knowledge in a separate searchable store. When a question comes in, search the store first, then hand the model both the question and the search results, and ask it to answer using them.

User: "How do I request paid leave?"
   ↓
App: searches the internal knowledge base
   ↓
App: sends the LLM → the question + the retrieved passages
   ↓
LLM: reads the passages, answers from them

Enter fullscreen mode Exit fullscreen mode

RAG won. Not because it’s more elegant, but because it’s operational — update a document and the system’s knowledge updates instantly, with no retraining. And crucially, the model is now answering from text you actually put in front of it, rather than from memory. That’s the first real dent in hallucination.

The idea to hold onto: the model gained an external memory it could look things up in.

4. Function calling → MCP: giving the model hands

RAG lets a model read the world. It still can’t touch it.

In June 2023, OpenAI shipped function calling, and the shape of it is simpler than most people expect. You describe your tools to the model in plain text. The model can’t execute anything — it just says which tool it wants and with what arguments. Your code runs it. Then you hand the result back.

User: "What's the weather in Tokyo?"    (+ a get_weather tool description)
   ↓
LLM: "I'd like to call get_weather(city='Tokyo')"
   ↓
App: actually calls the weather API      ← your code, not the model
   ↓
App: sends the result back to the LLM
   ↓
LLM: "It's sunny in Tokyo, high of 15°C."

Enter fullscreen mode Exit fullscreen mode

That gap — the model asks, your code acts — is the single most important thing to internalize about agents. An agent has no hands of its own. Everything it does, it does by asking your program to do it. Which means every guardrail you want, you build there.

Then came the integration problem. Every tool needed custom glue for every AI product. N tools times M apps.

In November 2024, Anthropic released MCP (Model Context Protocol) and open-sourced it: one standard way for an AI application to connect to a tool or data source. Build an MCP server for your service once, and any MCP-speaking client can use it. People called it “USB-C for AI,” which is a good enough analogy.

It worked. By 2026 MCP has been adopted by OpenAI, Google, and Microsoft, with over 10,000 public servers in the wild. A protocol that ships from one vendor and gets adopted by its direct competitors is rare, and it happened because the alternative — everyone maintaining bespoke integrations forever — was worse for everybody.

5. Reasoning models: think first, then answer

Everything so far still assumed one question in, one answer out. Ask, generate, done.

OpenAI’s o1 in September 2024 broke that assumption. A reasoning model thinks before it answers — it works the problem out internally, sometimes at considerable length, and only then produces a response.

Non-reasoning Reasoning Ask → answer, in one shot Think → plan → execute → answer Good for lookups and simple Q&A Good for genuinely hard problems A vague prompt gives a vague answer A vague prompt often survives — the model works out what you meant

That last row is the underrated one. Reasoning models made prompt-engineering tricks much less necessary. The model has room to figure out your intent instead of needing it spelled out perfectly on the first try.

By 2026 this has become adaptive: models decide for themselves how much thinking a given task deserves, and you steer it with a single “effort” dial rather than by hand-tuning a token budget. Cheap question, quick answer. Hard question, the model takes its time.

6. 2025: the pieces click together

Nothing fundamentally new was needed for agents. Everything was already on the table. In 2025 it just got assembled:

• Reasoning → the model can plan
• Function calling / MCP → it can act
• RAG → it can look things up
• Large context → it can hold the whole task in view

Put those in a loop — plan, act, observe the result, adjust, act again — and you have an agent.

The contrast that makes this concrete is early GitHub Copilot. Copilot autocompleted the next few lines. Single shot, no memory, no plan. Useful, but small.

A modern coding agent will change code across a dozen files, run the test suite, read the failure, search the web for the error, form a hypothesis, fix it, and re-run the tests — without you in the loop. Same underlying model family. The difference is entirely the loop.

Two things arrived on top of this that are worth knowing:

Agent Skills (October 2025). A skill is a folder with a Markdown file in it describing how to do something — a procedure, a house style, a checklist. The agent loads it only when the task calls for it. It’s a way to hand an agent expertise without bloating every prompt.

A common misconception is that Skills replaced MCP. They didn’t, and the distinction is worth getting right: MCP connects an agent to the outside world. Skills teach an agent how to do a job. You use both. Skills also happen to be plain Markdown, which means the person who owns the process can maintain it — no engineer required.

Agents as infrastructure (2026). Agents now run as hosted services: long-lived, scheduled, with persistent memory across sessions and a managed sandbox to work in. An agent that runs every night and reports in the morning is now ordinary.

The part that actually matters

Now the mindset shift — because everything above is the what, and this is the so what.

From prompt engineering to context engineering

The old game was writing the perfect prompt. Find the magic phrasing, discover the incantation, collect templates.

That game is mostly over. Reasoning models are good enough at inferring intent that clever phrasing has stopped being the bottleneck.

The new game is deciding what the model gets to see. Its context window is finite, and everything competing for space in it is a tradeoff:

What goes in the window Examples Instructions Project rules, conventions, a CLAUDE.md-style file Tools Which MCP servers and skills are available right now History The conversation so far Retrieved knowledge Whatever RAG pulled in

Every token you spend on one is a token you can’t spend on another. And as we saw in section 2, a full context window is not a well-used one.

So the skill is no longer phrasing. It’s curation — deciding what’s relevant, what’s noise, and what to leave out. That’s an engineering problem, and it’s a much more interesting one.

From “instructing AI” to “thinking with AI”

The old loop:

Human instructs → AI produces → Human judges → repeat

Enter fullscreen mode Exit fullscreen mode

You do the thinking. The AI does the typing. You grade the output.

The new loop:

Human ⟷ AI: think together
Human ⟷ AI: converge on a direction
Human ⟷ AI: build it

Enter fullscreen mode Exit fullscreen mode

Then Now Human’s role Instructor, judge Collaborator, navigator AI’s role Executor Thinking partner Interaction One-way Back and forth Goal The perfect one-shot output A good result, arrived at together When it fails “My prompt was bad” “I didn’t share enough context”

That last row is the whole article in one line. When an agent goes wrong today, the fix is almost never a better-worded prompt. It’s that the agent didn’t have something it needed — a file, a constraint, a reason, a piece of the picture that was in your head and never made it into the window.

Which is exactly the problem that closed out the last section. The two are the same problem.

Where to start

If you’ve read this far and never driven an agent, the next move is not to read more. It’s to open one and give it a real task from your actual work — not a toy. Watch what it does. Watch where it goes wrong. Then ask yourself the diagnostic question:

What did I know that it didn’t?

Answer that, put the answer in front of it, and go again. That loop is the skill. Everything else in this article is just background for it.

Written by a Japanese software engineer relocating to Canada. I write about AI engineering, and about what it’s like to rebuild a career in a second language. Corrections and disagreement are very welcome in the comments.