Benchmark video APIs yourself: upload to playable, measured across five providers

TL;DR

Vendor benchmark pages test the file size that flatters them. We’re going to build a small Node harness that measures the two numbers that decide your upload UX, time-to-upload and time-to-playable, then look at real results across FastPix, Mux, api.video, Cloudinary, and Gumlet. The punchline: the ranking flips depending on file size, so the only benchmark that matters is the one you run on your own footage.

📦 Code: github.com/USER/video-api-benchmark, replace before publishing

Why roll your own

Every video API has a benchmark page and every one of them wins its own benchmark. That’s not fraud, it’s selection. Upload performance depends heavily on file size, and providers optimize for different parts of the pipeline. So instead of trusting a leaderboard, let’s measure the thing your users actually feel: the “processing…” spinner between hitting upload and the video being playable.

We’ll measure two numbers per provider:

• Upload time, how long to push the bytes.
• Time-to-ready, upload plus processing, until a playable HLS manifest exists.

1. The measurement harness 🛠️

The pattern is the same for every provider: create an asset, upload the file, then poll until the asset reports ready. We time the whole thing. Here’s the reusable core.

// bench/measure.mjs, node 20+
import { performance } from "node:perf_hooks";

export async function timed(label, fn) {
  const t0 = performance.now();
  const result = await fn();
  const ms = performance.now() - t0;
  console.log(`${label}: ${(ms / 1000).toFixed(2)}s`);
  return { ms, result };
}

// Poll a status function until it returns "ready" (or throws on error/timeout).
export async function pollUntilReady(getStatus, { intervalMs = 1500, timeoutMs = 600000 } = {}) {
  const start = Date.now();
  for (;;) {
    const status = await getStatus();
    if (status === "ready") return;
    if (status === "errored") throw new Error("asset processing errored");
    if (Date.now() - start > timeoutMs) throw new Error("timed out waiting for ready");
    await new Promise((r) => setTimeout(r, intervalMs));
  }
}

Enter fullscreen mode Exit fullscreen mode

💡 Tip: run every provider from the same machine and the same network, back to back. Cross-machine numbers are noise.

2. Throttle the network so the numbers mean something

Uploads on your office fiber tell you nothing about a user on 4G. Cap the bandwidth. On Linux you can shape the interface with tc; simplest is to run the harness behind a throttled network namespace or a proxy. For a quick and dirty cap that works cross-platform, route your requests through a throttling proxy:

# using @sitespeed.io/throttle to cap the whole machine to ~10 Mbps up/down
npx @sitespeed.io/throttle --up 10000 --down 10000 --rtt 40
# run your benchmark in another shell, then:
npx @sitespeed.io/throttle --stop

Enter fullscreen mode Exit fullscreen mode

The published results below were all captured on a 4G profile capped at 10 Mbps, which is the profile that actually separates these providers.

3. A provider adapter (FastPix as the example)

Each provider needs a tiny adapter with two methods: createAndUpload(file) and status(id). The direct-upload pattern is nearly identical everywhere. Here’s one against FastPix using its on-demand endpoint.

// bench/providers/fastpix.mjs
const BASE = "https://api.fastpix.io/v1/on-demand";
const auth =
  "Basic " + Buffer.from(`${process.env.FP_KEY_ID}:${process.env.FP_SECRET}`).toString("base64");

export async function createAndUpload(fileUrl) {
  // Create an asset from a source URL (server-side import).
  const res = await fetch(BASE, {
    method: "POST",
    headers: { Authorization: auth, "Content-Type": "application/json" },
    body: JSON.stringify({ inputs: [{ type: "video", url: fileUrl }] }),
  });
  if (!res.ok) throw new Error(`create failed: ${res.status} ${await res.text()}`);
  const { data } = await res.json();
  return data.id;
}

export async function status(id) {
  const res = await fetch(`${BASE}/${id}`, { headers: { Authorization: auth } });
  const { data } = await res.json();
  return data.status; // "preparing" | "ready" | "errored"
}

Enter fullscreen mode Exit fullscreen mode

The Mux, api.video, and Cloudinary adapters follow the same shape against their own create/status endpoints and their own auth. FastPix, like most of these, authenticates with Basic auth using an access token id and secret key, and playback later comes off a https://stream.fastpix.io/<playbackId>.m3u8 URL. Wire each adapter into the runner:

// bench/run.mjs
import { timed, pollUntilReady } from "./measure.mjs";
import * as fastpix from "./providers/fastpix.mjs";

const SOURCE = process.env.SOURCE_URL; // your representative test file, hosted

async function benchProvider(name, p) {
  const { ms: uploadMs, result: id } = await timed(`${name} upload`, () =>
    p.createAndUpload(SOURCE),
  );
  const { ms: readyMs } = await timed(`${name} time-to-ready`, () =>
    pollUntilReady(() => p.status(id)),
  );
  return { name, uploadSec: uploadMs / 1000, readySec: (uploadMs + readyMs) / 1000 };
}

console.table([await benchProvider("FastPix", fastpix) /*, ...others */]);

Enter fullscreen mode Exit fullscreen mode

4. What the results actually looked like

I ran two files through this harness on the same 4G/10 Mbps profile. Here’s the big one first, a 177.2 MB clip, across the four providers that completed it (Cloudinary failed this run):

Provider Overall Upload Time-to-ready Cold startup FastPix 86 15.2s 29.4s 1.9s Mux 83 47.7s 53.3s 905ms api.video 79 17.0s 49.2s 2.99s Gumlet 74 24.8s 268.9s 1.25s

FastPix led on getting the file uploaded and playable. But look at the last column: Mux painted the first frame fastest (905ms vs 1.9s). Two different “fast.”

Now the same harness on a smaller 64.9 MB file, all five completing:

Provider Overall Upload Time-to-ready Cloudinary 95 16.6s 20.1s Mux 83 24.6s 67.8s api.video 77 15.0s 85.1s Gumlet 77 17.6s 191.6s FastPix 73 29.7s 58.9s

The winner flipped. Cloudinary processed the small file in about 2 seconds and ran away with it, while FastPix, first on the big file, landed fifth here. FastPix still processed faster than Mux (27.6s vs 42.4s), but its upload leg on this smaller file was slower, and on a 64.9 MB file the upload is a bigger share of the total.

⚠️ Read a bitrate table carefully. FastPix and api.video use content-aware encoding, so they hit similar visual quality at a lower rendition bitrate (FastPix averaged ~700 Kbps on the big file). Lower bitrate here means less bandwidth billed, not a worse picture. Don’t rank providers by bitrate as if higher wins.

5. Turning results into a decision

The pattern held: providers tuned for raw processing speed win on small files, and providers tuned for the upload path win as files grow. That maps to your workload:

• Large source files (long-form, screen recordings, camera masters): weight the upload/ingest leg. In separate tests, FastPix processed a 798.5 MB / 28-minute file in 1m52s vs Kaltura’s 7m08s, and uploaded a 1.46 GB file in 2m45s vs Mux’s 7m40s. The advantage compounds with size.
• Short clips where first-frame speed rules: weight cold startup, where Mux was consistently quick.

And pricing sits under all of it, on axes a benchmark won’t show:

Provider Encoding Delivery Analytics FastPix free (standard plan) ~$0.00096/min @1080p QoE free up to 100K views/mo Mux per-minute per-minute Mux Data from $499/mo (1M views) api.video free $0.0017/min included Cloudflare Stream free $1 / 1,000 min basic

Prices drift, so check the live pages before committing: FastPix, Mux, api.video, Cloudflare Stream.

💡 One non-benchmark cost: these are API-first platforms, not no-code CMSes. If you need a drag-and-drop dashboard for non-technical uploaders, budget to build that front-end yourself on top of whichever API wins.

What’s next

Fork the harness, swap SOURCE_URL for a file that looks like what your product actually ingests, and add adapters for the two or three providers on your shortlist. Run it throttled, three times, and average. The leaderboard you build from your own footage is the only one that stays true. From there, the same measurement loop extends to per-resolution delivery cost and to QoE once you wire up a player and start logging rebuffer events.