A green test suite tells you one thing: the implementation you wrote satisfies the tests you wrote.
It doesn’t tell you whether the version one small edit away would pass too. If swapping < for <= would have sailed through, the suite was never checking that boundary.
Coding agents make this sharper. They write the code and the tests from the same assumption, the suite goes green, and nobody stops to ask what else it would accept.
So I built TestSlop. Point it at a diff and it finds one nearby alternative implementation your tests also accept, then shows the input that tells the two apart.
One sentence: TestSlop finds one nearby “Evil Twin” implementation that behaves differently but still passes the same tests.
That GIF runs a three-line quantity check. As text:
ORIGINAL quantity <= 0
EVIL TWIN quantity <= 1
ORIGINAL 3 / 3 tests passed
EVIL TWIN 3 / 3 tests passed
Missing witness: quantity = 1
Enter fullscreen mode Exit fullscreen mode
Both passed. The two versions only disagree at quantity = 1, and no test covers that value.
What is an Evil Twin?
An Evil Twin is a nearby implementation that behaves differently but still passes the same tests.
The missing witness is the input that separates them. Here it is quantity = 1, the simplest input where the two versions give different answers.
TestSlop hands you the pair and the witness. Which version matches your contract stays your call.
A real agent workflow
Most of this repo’s public face went through an agent workflow. This slice is the part that pushed the sanitized history to GitHub and checked the rendered README, then returned a release verdict:
How it works
The diagram is the whole loop: shift one boundary comparison, rerun your tests against the alternative in a scratch copy, and print both results with the witness. Your source tree stays untouched.
A real run against vercel/ms
One fixture in the repo replays a real ms commit, the one that added month formatting, against its preserved test suite.
The original and the Evil Twin (msAbs >= y changed to msAbs > y) both passed 163 of 163 tests. The missing witness is exactly one year: at 31,557,600,000 ms the output moves from 1y to 12mo.
The formatter tests checked one millisecond past a year and never exactly one year. One historical example, replayable offline. It doesn’t show that TestSlop catches shipped bugs in general.
What it doesn’t do
TestSlop is narrow on purpose. It looks at JavaScript and TypeScript boundary comparisons on changed lines, so plenty of diffs produce no Twin at all.
- No Twin found doesn’t mean your code is correct. It can mean the tests are tight, or the diff is a shape TestSlop doesn’t handle.
- A Twin passing isn’t automatically a bug. It’s evidence your suite accepts that behavior too.
- Tests run with your normal user permissions in a scratch copy. That protects your source; it isn’t a security sandbox.
- It doesn’t replace general-purpose mutation testing. Different tool, different question.
Try it and tell me what breaks
The code is on GitHub: github.com/hyukvoid/TestSlop
It runs from a local build (npm install, then npm run demo). It isn’t published to npm. The repo keeps the research notes from the three phases that narrowed the project down to this idea.
If you’ve used mutation testing or coding-agent workflows, I’d be curious whether one concrete counterexample feels more useful than a score.