I turned plain English (and any live website) into API tests

Hi 👋 This is my first post here, and I want to share a side project I built as a QA engineer — because it scratches an itch I think a lot of us have.

Here’s the itch: for one business flow, I’d end up building the same thing three times — a Postman collection, a Playwright API test, and a JMeter plan. Same requests, same variables, same assertions, just typed out three different ways. It’s slow, it’s boring, and the three copies drift apart the moment anything changes.

So I built TestFlow Agent to do it once. It works in two ways, and I want to walk through both — because they solve two pretty different situations.

Part 1 — When it’s your service: just describe the flow

The first mode is for when you have control over the API you’re testing — it’s your team’s service, you know the endpoints, maybe it even lives in the same repo.

In that case, you don’t need to record anything. You just type the test in plain English:

Create an enrollment for a Texas member.
Get available plans for Texas. Select a Silver plan.
Submit enrollment with effective date 01/01/2026.
Validate enrollment status is ACTIVE.

Enter fullscreen mode Exit fullscreen mode

…and TestFlow Agent reads the intent — the state, the plan, the date, whether it’s a happy path or a negative case — and produces the API sequence:

POST {{baseUrl}}/auth/token
POST {{baseUrl}}/members
GET  {{baseUrl}}/plans?state=TX
POST {{baseUrl}}/enrollments
GET  {{baseUrl}}/enrollments/{{enrollmentId}}

Enter fullscreen mode Exit fullscreen mode

From that, it generates the Postman collection, the Playwright spec, and the JMeter plan — all sharing the same variables. And because the repo ships a small mock API, you can hit Run and watch the flow actually execute end-to-end (enrollment created, status ACTIVE, IDs chained). It’s not just static text; it runs.

This part is great for teams testing their own services: you already know the flow, so describing it is faster than clicking through it.

But it has a limit — it works best when the flow is known and the service is “yours.” Which led me to the part I’m actually more excited about.

Part 2 — When it’s any app: just record it

The second mode needs zero knowledge of the API. You point it at any website, click through your flow by hand, and it figures out the tests for you.

You enter a URL, hit Start Discovery, and a real browser opens. You log in and do whatever you’d normally do — I tested it against a live OrangeHRM demo — then hit Stop. Behind the scenes it recorded all the network traffic, and now it hands you the same three artifacts (Postman / Playwright / JMeter), built from what you actually did.

The catch is that raw browser traffic is messy, and making it into tests that run again is where most of the real work went. A few of the things it has to get right:

• Figuring out what’s a variable. IDs, tokens, employee numbers — it detects them and chains them across steps, so step 4 uses the ID that step 2 created.
• Naming them sensibly. It names a variable from the JSON field it came from (employee.id → employeeId), not from the URL — because on real apps every URL says “employees” whether or not that’s what the ID means.
• Handling login tokens. Captured CSRF tokens are dead by the next run, so instead of replaying a stale one, it re-fetches a fresh token before logging in. That’s the difference between a test that logs in and one that just 403s.
• Cutting the noise. Fonts, analytics, i18n bundles, tracking pixels — all filtered out, so you’re left with the actual business API calls.
• Not corrupting things. A tiny one that bit me: an ID of 7 must never get find-and-replaced inside a number like 4700. Short IDs are only matched on whole tokens.

You don’t have to think about any of that — you click through the app, and you get runnable tests. That’s the part I think anyone can benefit from, on any app, without touching the source code.

A quick before/after

Here’s one real request it captured from that OrangeHRM session:

What the browser sent:

PUT /web/index.php/api/v2/performance/reviews/17/evaluation/employee
{ "ratings": [ { "kpiId": 1, "rating": null } ] }

Enter fullscreen mode Exit fullscreen mode

What it generated:

PUT {{baseUrl}}/web/index.php/api/v2/performance/reviews/{{reviewId}}/evaluation/employee
{ "ratings": [ { "kpiId": "{{kpiId}}", "rating": null } ] }

Enter fullscreen mode Exit fullscreen mode

reviewId and kpiId are now variables that resolve from a generated environment file — so you import it and run, no manual filling-in.

Try it

It’s open source (MIT) — Node + Express + React — with a one-command Docker start:

git clone https://github.com/sshankar07/test-flow-agent
cd test-flow-agent
docker compose up --build     # then open http://localhost:5173

Enter fullscreen mode Exit fullscreen mode

(The live-recording part opens a real browser, so run that bit natively — the README shows both ways.)

👉 Repo: https://github.com/sshankar07/test-flow-agent

That’s it — my first dev.to post! If you’re in testing/automation, I’d love to know: what would you want it to generate next — k6? RestAssured? A CI pipeline? Drop a comment 🙌