Episode 2 — Who Actually Runs My Pipeline?

Junior Engineer: Last week you left me hanging. I push code, GitHub gets it — then what? Whose computer is actually running my tests?

Senior Engineer: Honestly, most engineers use this stuff for months without ever asking that question. Good that you did.

The Push

Senior Engineer: Let’s slow it down. You type git push. What actually leaves your machine?

Junior Engineer: The code, I guess? My commits?

Senior Engineer: More specific than that. Git doesn’t upload your whole project every time — it sends only what’s changed, packed into a compact Git-native format. Not a ZIP of your folder. Just the new commits, efficiently.

Junior Engineer: I’ve never thought about what the wire format looks like.

Senior Engineer: And you don’t need to, not for this. Git internals are their own series — we’ll get there. All you need today is: your commits arrive.

Point is — your commits arrive. GitHub stores them. The branch pointer updates. Your PR, if there is one, now shows a new commit.

Junior Engineer: And then a pipeline starts?

Senior Engineer: No. Not yet. Something has to tell it to start.

The Webhook

Senior Engineer: GitHub doesn’t inherently know that your repo cares about pushes. GitHub runs millions of repos. Most of them don’t have CI. So after storing your commit, GitHub does one thing: it fires an event.

Internally, this looks like a notification — “hey, a push event just happened on this repo, on this branch, here are the commit details.” GitHub then checks: does anyone care about this event?

Junior Engineer: Who’s “anyone”?

Senior Engineer: Anyone who registered a webhook. A webhook is just a URL that says “when X happens, send an HTTP POST to this address with the details.” External services register webhooks on your repo all the time — Slack bots, monitoring tools, deployment services. And GitHub’s own CI system, GitHub Actions, effectively registers one too.

Junior Engineer: So GitHub Actions is… listening through a webhook like everything else?

Senior Engineer: Conceptually, yes. Under the hood it’s more integrated than a third-party webhook — GitHub doesn’t literally POST to itself over the internet — but the mental model holds. An event occurs, something is listening, and that something decides what to do next.

📝 Production Note

Webhooks are how most modern systems communicate about “something happened.” Stripe uses them to tell your server a payment succeeded. Twilio uses them to deliver incoming SMS. GitHub uses them to notify CI systems that a push occurred. The pattern is always the same: event source, HTTP POST, payload with details, receiver decides what to do.

🎤 Explain It In One Minute

Imagine you’re explaining GitHub Actions to a teammate — without using the words webhook, runner, or workflow.

Can you explain what happens after git push?

If yes — you understand the idea, not just the terminology.

Workflow Discovery

Junior Engineer: Okay, so the push event fires. What does GitHub Actions do with it?

Senior Engineer: It does something surprisingly simple. It looks at your repo.

Specifically, it checks for a directory:

.github/workflows/

Enter fullscreen mode Exit fullscreen mode

If that directory doesn’t exist — nothing happens. Your push lands, your branch updates, and that’s the end of it. No pipeline. No runner. No tests. Silence.

Junior Engineer: What if the directory exists but is empty?

Senior Engineer: Same thing. GitHub looks for workflow files inside that directory — YAML files, specifically — and checks whether any of them are configured to trigger on a push event. If none match, silence again.

Junior Engineer: So the workflow file is what actually connects the push to the pipeline.

Senior Engineer: Exactly. The event is just noise until a workflow file says “I care about this.”

What Is a Workflow?

Junior Engineer: Before we go further — what actually is a workflow, conceptually? Not the YAML syntax, but the idea.

Senior Engineer: A workflow is a description of: “when this event happens, run these steps, in this order, on this type of machine.”

That’s it. Three things:

1. Trigger — what event starts it? A push? A PR? A manual button click? A schedule?
2. Jobs — logical groupings of steps. A “test” job, a “build” job, a “deploy” job.
3. Steps — the actual commands. npm ci, npm test, docker build, whatever your project needs.

Junior Engineer: And all of that lives in one YAML file?

Senior Engineer: One YAML file per workflow. A repo can have many workflows — one for PRs, one for deploys, one for nightly cleanup, whatever you need. Each is independent.

Why YAML?

Junior Engineer: Why YAML, though? Why not just a JavaScript file or a Python script that says what to run?

Senior Engineer: Few reasons, and they’re all about trust and constraint.

First — a JS or Python file is executable. It can do anything. Delete files, make network requests, mine crypto. You’re handing GitHub a script and saying “run this on your machines.” That’s a hard sell at GitHub’s scale — millions of repos, many of them public, many of them from strangers.

Second — YAML is declarative. You describe what you want, not how to do it. “Run these steps in this order on this machine.” The CI system decides how to interpret and execute that. You can’t write a while(true) loop in workflow YAML. You can’t open a socket. The format itself constrains what’s possible.

Junior Engineer: But YAML can get complex.

Senior Engineer: It can, and people complain about it constantly. But the alternative — “just let anyone run arbitrary code” — is worse. YAML is the compromise between “expressive enough to define a pipeline” and “restricted enough that GitHub can safely run it on shared infrastructure.”

Junior Engineer: You said “shared infrastructure.” Is that where runners come in?

Senior Engineer: Almost — but one thing first, so you don’t walk away with the wrong idea. YAML isn’t what makes GitHub secure. GitHub still executes real commands — npm test, docker build, whatever your steps say. YAML just gives you a structured, constrained way to describe the workflow. The actual security comes later — from isolated runners, permissions, and execution controls. Don’t confuse the format with the safety mechanism.

Why Events Trigger Workflows

Junior Engineer: Why not just run the workflow on every push to any branch? Why does the trigger matter?

Senior Engineer: Because you don’t always want it to run.

Your main branch — yes, run everything. Tests, builds, maybe a deploy.

Someone’s feature branch named experiment/delete-everything — maybe run tests, but absolutely do not deploy.

A branch named dependabot/chore-update-lodash — run tests, sure, but do you really need a full build and deploy preview for a patch version bump of a utility library?

The trigger lets you say: “this workflow runs on pushes to main only” or “this workflow runs on pull requests but not on pushes to branches.” It’s about matching effort to risk.

Junior Engineer: What happens if someone pushes to a branch and no workflow triggers?

Senior Engineer: Nothing. That’s the correct behavior. Not every code change needs a pipeline.

🪞 If I asked you this in an interview

“What actually happens when you push code to a GitHub repo with GitHub Actions configured?”

Git sends a pack file with the new commits to GitHub. GitHub stores them and fires a push event. GitHub Actions checks .github/workflows/ for workflow files that match that event. If it finds one, it schedules the workflow. If not, nothing happens — the push lands with no pipeline.

The Runner Problem

Junior Engineer: Okay. So the workflow matches. GitHub knows what to run. Now — whose machine actually executes it?

Senior Engineer: Now we hit the real question. And it’s one that surprises people.

GitHub does not run your code on its own servers.

Junior Engineer: Wait — really?

Senior Engineer: Really. Think about it from GitHub’s perspective. Your workflow says npm install. Then npm test. Your tests might spin up a database. They might download gigabytes of dependencies. They might run for forty-five minutes. They might have a memory leak that hogs CPU.

Now multiply that by every push to every repo on GitHub, happening every second, all day, forever.

Junior Engineer: That would be absurdly expensive.

Senior Engineer: It would also be a security nightmare. Your code could try to read other repos’ data, probe internal network addresses, exploit the host kernel. Running arbitrary user code on shared servers is one of the hardest problems in computing. GitHub’s core product is storing your code safely — not executing it unpredictably.

So instead, GitHub does something smarter.

What Is a Runner?

Senior Engineer: A runner is a machine — a real or virtual computer — that exists to do one thing: receive workflow instructions from GitHub, check out your code, and execute your steps.

It’s not GitHub’s server in the traditional sense. It’s a separate machine, running a piece of software called the runner agent, that:

• Listens for jobs assigned to it by GitHub
• Creates an isolated environment for each job
• Checks out your repo’s code into that environment
• Runs each step in order
• Reports results — pass or fail — back to GitHub
• Cleans up

Junior Engineer: So the runner is basically a worker that does what the workflow tells it?

Senior Engineer: Exactly. GitHub is the coordinator. The runner is the executor. Your workflow YAML is the instructions.

📒 Senior Engineer’s Notebook

The runner doesn’t think. It doesn’t decide what to run. It receives a job definition from GitHub and executes it step by step, reporting each step’s result. All the intelligence lives in GitHub’s scheduling layer — the runner is muscles, not brain.

Hosted Runners

Junior Engineer: Where does the runner machine come from?

Senior Engineer: Two options. First: GitHub-hosted runners.

GitHub maintains a pool of virtual machines — thousands of them — across AWS and Azure. When your workflow triggers, GitHub picks an available machine from that pool, starts a fresh VM for your job, runs the runner agent on it, hands it your workflow, and tears the VM down when it’s done.

Junior Engineer: Fresh every time?

Senior Engineer: Fresh every time. You never get a machine that someone else just used. No leftover files, no cached state from a stranger’s repo, no chance of cross-contamination. The VM is born, it runs your job, and it dies.

Think of it like a hotel. Every guest gets a separate room. When they check out, housekeeping cleans it before the next guest ever sees it. You never inherit the last guest’s mess — and the next guest never sees yours. Hosted runners work the same way, one job per “room,” cleaned before the next one checks in.

Junior Engineer: That must be expensive for GitHub.

Senior Engineer: It is. That’s why GitHub puts limits on how many minutes you can use per month on the free tier, and charges for more. Every minute your workflow runs is a minute of a real VM costing real money.

Junior Engineer: What does the machine look like? What’s on it?

Senior Engineer: GitHub offers several runner images — pre-configured VM templates with different tools pre-installed. There’s one for Ubuntu Linux with Node.js, Python, Java, Docker, and a bunch of CLI tools. There’s one for macOS. There’s one for Windows. You pick which one your job needs.

Junior Engineer: So if I need Go installed, I either pick an image that has it or install it myself in a step?

Senior Engineer: Yes. The runner image is a starting point, not a straitjacket. Most real workflows have early steps that install anything the image is missing.

Self-Hosted Runners

Junior Engineer: You said there were two options.

Senior Engineer: Second option: self-hosted runners. You provide the machine. Your company buys a server, or spins up a VM in your own AWS account, installs the runner agent software on it, and registers it with GitHub. Now when a workflow triggers, GitHub can assign jobs to your machine instead of its own.

Junior Engineer: Why would a company do that instead of using GitHub’s?

Senior Engineer: Several reasons. Let me walk through the real ones, not the theoretical ones.

Cost at scale. If you’re running five hundred workflows a day, each taking ten minutes, that’s eighty-three hundred minutes per day. GitHub charges for that. A dedicated server might be cheaper at that volume.

Custom tooling. Your company uses an internal VPN, a proprietary database, a custom compiler, an on-premises artifact registry that isn’t accessible from the public internet. A GitHub-hosted runner can’t reach any of that. Your machine, on your network, can.

Compliance. Remember the bank from last week? They can’t run code on machines they don’t control. Regulatory requirements might mandate that all execution happens on infrastructure the company owns and audits.

Speed. If your repo is enormous — think monorepo with gigabytes of history — just checking out the code takes time. A self-hosted runner with a local cache or a fast network connection to your Git server can shave minutes off every run.

🚨 Beginner Mistakes

“Self-hosted runners are always better.” They’re not. You now own patching, security updates, disk space, and availability. If your self-hosted runner goes down at 2 AM, your pipeline is broken until you wake up and fix it. GitHub-hosted runners have that problem instead.

“I’ll put a self-hosted runner on my personal laptop.” Technically possible. Terrible idea. Your laptop goes to sleep, your pipeline breaks. Your laptop joins a coffee shop WiFi, your pipeline is now running on a public network with your repo’s code on disk. Don’t.

Why Runners Exist — The Real Reason

Junior Engineer: But why the runner concept at all? Why not just have GitHub SSH into a server and run commands directly?

Senior Engineer: Because that’s what the developer in our Friday story did, and we spent an hour talking about why it was a problem.

No audit trail. No isolation. No consistency between runs. No way to reproduce what happened. A runner fixes all of that:

• Every step is logged — GitHub receives a status update for each step as it runs. You can see exactly what command ran, what it output, and how long it took.
• The environment is defined — you specify the OS, the tools, the environment variables. Same workflow, same machine template, same result.
• Isolation is enforced — your job can’t see other jobs. It can’t access the host system. It runs in a constrained environment.
• Lifecycle is managed — the machine is created for your job and destroyed after. No leftover state, no drift.

The runner is the answer to “how do we run arbitrary code safely, repeatably, and observably?”

Runner Lifecycle

Junior Engineer: Can you walk through the actual sequence? From trigger to first command?

Senior Engineer: Sure. Here’s what happens, step by step, the moment your workflow matches a push event:

1. Queue. GitHub places your workflow job in a queue. If you have three jobs and only two runners available, one waits.

2. Assignment. A runner picks up the job. If it’s GitHub-hosted, a fresh VM starts first. If it’s self-hosted, the runner agent signals “I’m ready” and receives the job.

3. Environment setup. The runner creates an isolated working directory. It sets up environment variables — including ones GitHub injects automatically, like GITHUB_SHA for your commit hash and GITHUB_REF for your branch name.

4. Checkout. Usually the first real step in any workflow — the runner pulls your code from GitHub into its working directory. There’s a standard action called actions/checkout that almost every workflow uses as its first step.

5. Step execution. The runner walks through each step in your workflow. For each one, it starts a shell — bash on Linux, PowerShell on Windows — runs the command, captures the output and exit code, and reports back to GitHub.

6. Result. Every step either passes or fails. If any step fails, the job stops and is marked failed. If all steps pass, the job is marked passed.

7. Teardown. The runner reports final status to GitHub. On GitHub-hosted runners, the VM is destroyed. On self-hosted runners, the working directory is cleaned and the agent waits for the next job.

Security Isolation

Junior Engineer: You mentioned isolation. What does that actually mean in practice?

Senior Engineer: On GitHub-hosted runners, each job gets its own VM. Not just its own directory — its own virtual machine. Separate kernel, separate filesystem, separate network namespace. Your job literally cannot see another job running on the same physical host, because there is no shared anything at the VM level.

Junior Engineer: And on self-hosted?

Senior Engineer: That’s where it gets interesting — and where companies get cautious.

On a self-hosted runner, jobs run as a process on a machine you control. The runner agent does apply some isolation — each job gets its own working directory and environment — but it’s not a VM boundary. Two jobs on the same self-hosted runner share the same kernel and the same filesystem.

Junior Engineer: So one job could theoretically read another job’s files?

Senior Engineer: If someone wrote a step that did that, yes. Which is why most companies that take this seriously use one runner per job — spin up a fresh VM or container for each workflow run, run the agent inside it, destroy it after. You lose the speed advantage of reusing a warm machine, but you gain real isolation.

Junior Engineer: What about environment variables? If one workflow sets a secret, can another workflow on the same runner see it?

Senior Engineer: GitHub injects secrets differently than regular environment variables — they’re masked in logs and scoped to the specific workflow run. But the fundamental trust boundary on self-hosted runners is your infrastructure, not GitHub’s. If you don’t trust the code running on your runner, you have a bigger problem than environment variable leaking.

🏢 Office Reality

• “The runner is flaky.” Means jobs on that runner fail intermittently — maybe the machine is underpowered, maybe the disk is full, maybe there’s a network issue between the runner and GitHub. Not your code’s fault, but your pipeline’s problem.
• “We need more runners.” Means the queue is backing up. Jobs are waiting too long for an available machine. Common during peak hours when everyone’s pushing code before standup.
• “That runner is pinned to the main branch.” Means that specific runner only accepts jobs from workflows triggered on main. Other branches use different runners — maybe cheaper or less powerful ones.
• “The runner image is stale.” The pre-configured template hasn’t been updated in months. It’s missing security patches or new tool versions. Someone needs to rebuild it.

Startup vs Enterprise Runners

Junior Engineer: How does this look different at a five-person startup versus a bank?

Senior Engineer: At the startup — GitHub-hosted runners, default Ubuntu image, maybe one self-hosted runner for a special internal tool. Nobody’s thought about it much. It works until it doesn’t.

At the bank — self-hosted runners on infrastructure they control, inside a VPN, with custom runner images built and audited by a security team. Each image is versioned and tagged. There’s a pipeline that builds the runner images. Yes — a pipeline that builds the machines that run the pipelines. That layer of meta is where enterprises live.

Junior Engineer: A pipeline for the pipeline?

Senior Engineer: You’ll see that pattern more than you’d expect. Companies automate everything — including the automation itself.

🪞 If I asked you this in an interview

“What’s the difference between a GitHub-hosted runner and a self-hosted runner?”

GitHub-hosted runners are temporary VMs managed by GitHub — fresh for each job, pre-configured with common tools, but on infrastructure you don’t control. Self-hosted runners are machines you provide — they can access your internal network and custom tooling, but you’re responsible for maintaining, securing, and patching them.

“Why doesn’t GitHub just run pipeline code on its own servers?”

Because running arbitrary user code at scale is expensive and dangerous. User code could consume excessive resources, access other repos’ data, or exploit the host. Runners provide isolation — each job gets its own environment — and GitHub can control, meter, and tear down that environment when the job finishes.

Whiteboard Moment

git push
    ↓
GitHub receives pack file, stores commits
    ↓
Push event fires
    ↓
GitHub checks .github/workflows/ for matching workflows
    ↓
Match found → job queued
    ↓
Runner picks up job (VM starts if hosted)
    ↓
Runner sets up environment, injects variables
    ↓
actions/checkout pulls your code
    ↓
Runner executes each step, reports results
    ↓
Job passes/fails → VM destroyed (hosted) or cleaned (self-hosted)
    ↓
GitHub updates workflow status in your repo

Enter fullscreen mode Exit fullscreen mode

The same journey, stripped to its bones:

Developer
    ↓
Git Push
    ↓
GitHub
    ↓
Workflow Parser
    ↓
Queue
    ↓
Runner
    ↓
Shell
    ↓
npm ci
    ↓
npm test
    ↓
Result

Enter fullscreen mode Exit fullscreen mode

Junior Engineer: Okay. So the runner has my code. It has a fresh environment. It’s ready to go. But — how does it know what commands to run? The workflow YAML is sitting on GitHub, not on the runner.

Senior Engineer: Is it?

Junior Engineer: …it’s not? The runner already has it?

Senior Engineer: Remember step four. actions/checkout. What do you think it checked out?

Junior Engineer: My code.

Senior Engineer: Your code — which lives in a repo that contains .github/workflows/.

Junior Engineer: So the workflow file is part of the repo that gets checked out onto the runner?

Senior Engineer: Exactly. But here’s the thing — the runner doesn’t read the YAML from the checkout. It already received the job definition before checkout happened. GitHub parsed the workflow, broke it into steps, and sent the runner a structured job payload. The YAML is instructions for GitHub. The runner receives executed instructions.

Junior Engineer: What’s the difference?

Senior Engineer: The difference is where the intelligence lives. The runner doesn’t interpret YAML. It receives “run step 1: npm ci, run step 2: npm test” as a list of commands. The YAML parsing, the variable substitution, the conditional logic — all of that happened on GitHub’s side before the runner ever saw it.

Junior Engineer: So the runner is dumber than I thought.

Senior Engineer: The runner is designed to be dumb. Dumb is secure. Dumb is predictable. The runner is a shell executor. It receives a command, runs it, reports the result. That’s the job.

What You Should Be Able to Explain Now

(Without looking at Google)

Can you explain:

• What actually leaves your machine when you run git push?
• What a webhook is and why it matters for CI?
• How GitHub discovers whether a repo has any workflows?
• What a runner is, in plain language?
• Why GitHub doesn’t execute your code directly on its own servers?
• The difference between hosted and self-hosted runners — and when you’d pick each?
• What “isolation” means for a runner, and why it matters?
• Why the runner doesn’t parse your YAML file itself?

If yes — you now understand more about CI infrastructure than most engineers who’ve been using GitHub Actions for a year.

Junior Engineer: The runner received its instructions. It’s about to run npm ci as its first real command. But wait — where do the dependencies come from? Does the runner already have them? Does it download them every time?

Senior Engineer: Now you’re asking the right question. And the answer is where pipelines either get fast or get painful.

(End of Episode 2)