The backend was green and the video kept buffering: a QoE debugging playbook

TL;DR

Backend metrics can’t see rebuffering, slow startups, or failed plays; those happen inside the player. We’ll instrument an HTML5/hls.js player to capture real QoE signals in ~40 lines, look at what the numbers mean, and then be honest about where DIY telemetry stops scaling and what the managed options cost.

📦 Code: github.com/USER/player-qoe-starter (replace before publishing)

The incident that teaches everyone this lesson

A user writes in: “the video keeps stopping.” You check Grafana. Origin 5xx: flat zero. CDN hit ratio: healthy. Uptime: perfect. You close the ticket as “cannot reproduce,” and next week there are four more.

Nothing is lying. Your dashboards describe the server side of the socket, and playback quality is a client-side phenomenon. Here’s the mismatch in table form:

Your dashboard sees The viewer feels 200 OK, 38 ms TTFB 6 seconds of spinner before frame one Successful segment fetches Two rebuffers during minute 3 Healthy origin Player error after a bitrate switch Requests per second Nothing: they gave up and left

A segment can download successfully but slower than the play head consumes it. That’s a rebuffer, and no access log will ever contain it. So let’s capture it where it happens.

1. Instrument the player in ~40 lines

No vendors yet; the browser gives you most of the signal for free.

// qoe.js
export function instrument(video, hls) {
  const session = {
    startupMs: null,
    rebufferCount: 0,
    rebufferMs: 0,
    bitrateSwitches: 0,
    fatalError: null
  };

  let clickedPlayAt = null;
  let stallStartedAt = null;

  video.addEventListener("play", () => {
    if (clickedPlayAt === null) clickedPlayAt = performance.now();
  });

  video.addEventListener("playing", () => {
    if (session.startupMs === null && clickedPlayAt !== null) {
      session.startupMs = Math.round(performance.now() - clickedPlayAt);
    }
    if (stallStartedAt !== null) {
      session.rebufferMs += Math.round(performance.now() - stallStartedAt);
      stallStartedAt = null;
    }
  });

  video.addEventListener("waiting", () => {
    // 'waiting' after playback started = a real stall, not startup
    if (session.startupMs !== null && stallStartedAt === null) {
      session.rebufferCount++;
      stallStartedAt = performance.now();
    }
  });

  video.addEventListener("error", () => {
    session.fatalError = video.error?.code ?? "unknown";
  });

  if (hls) {
    hls.on(Hls.Events.LEVEL_SWITCHED, () => session.bitrateSwitches++);
    hls.on(Hls.Events.ERROR, (_e, data) => {
      if (data.fatal) session.fatalError = data.details;
    });
  }

  return session;
}

Enter fullscreen mode Exit fullscreen mode

Wire it up and dump the session on unload:

// player.js
const video = document.querySelector("video");
const hls = new Hls();
hls.loadSource("/video/lesson1/index.m3u8");
hls.attachMedia(video);

const session = instrument(video, hls);

addEventListener("visibilitychange", () => {
  if (document.visibilityState === "hidden") {
    navigator.sendBeacon("/qoe", JSON.stringify(session));
    console.table(session);
  }
});

Enter fullscreen mode Exit fullscreen mode

Watch a video on throttled DevTools network and you’ll see something like:

┌─────────────────┬──────────┐
│ startupMs       │ 4180     │
│ rebufferCount   │ 3        │
│ rebufferMs      │ 9640     │
│ bitrateSwitches │ 4        │
│ fatalError      │ null     │
└─────────────────┴──────────┘

Enter fullscreen mode Exit fullscreen mode

That table is more diagnostic truth about the complaint than your entire backend stack currently has.

💡 Tip: sendBeacon on visibilitychange is the reliable exfil pattern. unload handlers get skipped on mobile and on killed tabs, while a beacon survives the page dying mid-request. Buffer events and flush periodically too, and you’ll even catch the sessions that crash.

2. Reading the numbers

Three metrics carry most of the weight:

• Startup time (play intent to first playing): viewers abandon fast when this stretches; keep an eye on the p95, not the average.
• Rebuffer ratio (rebufferMs / totalWatchMs): the canonical QoE number. Even small single-digit percentages feel awful in a session.
• Playback failure rate: sessions ending in a fatal error, sliced by browser/device/network before anything else. Failures cluster; averages hide them.

Startup time also decomposes: manifest fetch, first segment fetch, and decode each contribute. When p95 startup spikes, split it into those pieces before blaming the CDN.

⚠️ Note: the waiting event fires for seeks too. Production-grade trackers separate seek-induced stalls from network stalls (check video.seeking); our snippet keeps it simple.

3. Where DIY hits the ceiling

Our 40 lines capture five signals for one session in one tab. The gap between this and answering “which users had a bad Tuesday and why” is, concretely:

1. Sessionization: stitching beacons into views, views into viewers, across reloads.
2. Dimensions: device, OS, browser, network type, geo, CDN, rendition on every event.
3. Weighted aggregation: rebuffer ratio weighted by watch time, percentile startup by cohort.
4. Storage and retention: beacons arrive forever; someone owns that pipeline now.
5. Alerting: “failure rate for Android/Chrome on ISP X doubled” is the actual product.

I’ve built this in-house before. The v1 was a fun sprint; the maintenance was a permanent tax. Commercial QoE tools exist because items 1 through 5 are a product, not a feature. For scale reference, dedicated platforms track far more than our five signals: Mux Data exposes 30+ metrics, and FastPix’s Video Data captures 50+ playback data points per view session.

4. The managed options, honestly

Current public pricing for the two developer-facing routes, plus DIY for contrast:

Option Free tier Beyond free Signals DIY (above) your time your time, forever ~5 until you build more Mux Data 100K views/mo $0.60/1K (PAYG); $499/mo Media plan incl. 1M 30+ metrics FastPix Video Data 100K views/mo usage-based 50+ per session

Notes from having used both sides of this table:

• If your video already runs through Mux, Mux Data is the natural add; same ecosystem.
• FastPix bundles Video Data with its video platform (encoding, delivery, player), so if your video is already there, the telemetry is a feature flag away rather than an integration: the player SDKs report into it, there’s a real-time dashboard, and the same data is available over the API. It’s also independently adoptable; you don’t need to move your pipeline to use the analytics.
• One caveat on FastPix: the docs are written for developers comfortable living in an API reference. Fine for us; the non-technical folks on your team may want a walkthrough before they can self-serve.
• Whatever you pick, keep the DIY instrumentation knowledge. Understanding what waiting vs stalled vs a fatal ERROR means is how you’ll interrogate any vendor dashboard instead of nodding at it.

5. The playbook

Next time “the video keeps stopping” lands in your queue:

1. Don’t ask about their wifi yet.
2. Check whether you have session-level playback data. If not, that’s the actual bug; ship the 40 lines above today.
3. Slice failures and rebuffer ratio by device, browser, and network before anything else. Bad QoE clusters.
4. Before buying or building a telemetry pipeline, check what your existing video stack already collects. Bundled analytics you’re not using is the cheapest observability you’ll ever ship.

What’s next

Two follow-ups: wire the beacon endpoint into something queryable (a POST /qoe handler plus a table gets you shockingly far for a small app), and if you’re on hls.js, look at CMCD, which standardizes shipping these client metrics to your CDN with hls.js 1.7 supporting CMCD v2. The viewer’s side of the socket has plenty to say; the only question is whether you’re listening.