I built vexyo, conformance and regression testing for MCP servers

It started as a personal need

A while back I was working on a project with a few MCP servers wired into it. Every so often something would go subtly wrong. The server hadn’t crashed, the logs were clean, but the AI client on the other end just behaved oddly. It would skip a tool, misread a result, or wander off. And every time, I’d lose an hour or two bisecting commits and re-reading the spec, trying to figure out which small thing my server had quietly stopped doing correctly.

After the third or fourth time, I got tired of debugging the same class of problem by hand. What I actually wanted was something that would connect to the server, check it against the spec, and just tell me plainly what broke and where, before it ever reached a client. I couldn’t find a tool that did that the way I wanted, so I built one.

What vexyo does

vexyo is a CI-native testing harness for MCP servers. It does two things.

Conformance. It connects to your server exactly like an AI client would and runs 16 rules across initialization, discovery, error semantics, and transport, checking it against the MCP spec (2025-11-25). Every finding cites the exact section of the spec it enforces, so you’re never guessing:

✗ discovery/tools-list-available
  tools/list failed despite the tools capability being advertised
  spec: Server Features §Tools / Listing Tools

15 pass · 1 fail · exit 1

Enter fullscreen mode Exit fullscreen mode

Regression. It records how your tools actually behave (golden sets), then flags three kinds of drift on every commit (schema, behavioral, and coverage), so a change that quietly breaks a working tool fails your build instead of shipping.

It exits non-zero on failure, so it gates CI by default. The GitHub Action prints a job summary you can act on, with the rule, severity, spec citation, and fix, right in the PR. That job summary is exactly the thing I wished I’d had during those debugging sessions.

It works with any language

vexyo talks to your server over its transport (stdio or Streamable HTTP), not your runtime. So it tests MCP servers written in Python, TypeScript, Go, PHP, anything. Point it at a command or an HTTP URL:

import { defineConfig } from '@vexyo/cli/config';

export default defineConfig({
  specVersion: '2025-11-25',
  target: { transport: 'stdio', command: 'python', args: ['-m', 'my_server'] },
});

Enter fullscreen mode Exit fullscreen mode

Try it

npm i -D @vexyo/cli
npx vexyo init
npx vexyo run

Enter fullscreen mode Exit fullscreen mode

That scaffolds a config, points it at your server, and gives you a first result in a couple of minutes.

• Docs: https://vexyo.dev
• Repo: https://github.com/vexyohq/vexyo
• It’s free and open source (Apache-2.0).

It solved my own problem first, and I’m genuinely curious what breaks first for other people. If you run it against your server, I’d love to hear what it caught (or got wrong).

PS. If you try it, tell me what it caught, or what it missed. That’s how it gets better. Questions and issues are very welcome.