I shipped an LLM efficiency + security kernel — and deleted my own best idea

Six months ago I set out to make LLMs “smarter” by orchestrating many of them
together. I measured it. It didn’t work. Here’s what I shipped instead — and why
the failure is the part I’m proudest of.

The idea that failed (and why I’m telling you)

The plan was “mitosis”: split a task across several LLMs, let them multiply and
compete, then synthesize the best answer. It sounds great in a pitch deck.

On ground-truth executed tests, it made correctness worse:

• Baseline 95% → mitosis 83% (−17 passing tests; synthesis corrupted answers that were already correct)
• At 4–6× the cost
• Confirmed across three independent experiments (code correctness, security remediation, verified cross-model selection)

Every gain was ≤ 0. So I deleted it. The full evaluation — including the failure —
is in the repo’s FINDINGS.md.

The lesson: an idea that survives a pitch is not the same as an idea that
survives a measurement.

What actually worked — and shipped

BIOMA is a small, local, provider-agnostic kernel (Rust core + a thin Python
layer) that sits in front of any LLM call and hardens the payload in-process,
before it leaves your machine.

from bioma.firewall_client import CognitiveFirewall

fw = CognitiveFirewall(vault={"db_password": DB_PW})   # secrets to protect
h = fw.shield(history, "refactor this function")
#   h.prompt / h.system  -> clean, dehydrated, secret-free payload
#   h.telemetry          -> saturation, red_alert, apoptosis_reduction, kernel_latency_us

import anthropic                                        # or google.genai, or openai
msg = anthropic.Anthropic().messages.create(
    model="claude-sonnet-5", max_tokens=1024,
    system=h.system or "", messages=[{"role": "user", "content": h.prompt}])

Enter fullscreen mode Exit fullscreen mode

Three mechanisms, all measured:

1. Efficiency — context apoptosis

Each context block gets a metabolic weight and a half-life; low-value blocks (old
logs, resolved chatter) are purged before dispatch.

• −80% input tokens typically; up to −97% on long, noisy sessions.
• A real 16-round session: 47,890 → 2,022 input tokens, apoptosis latency ~1.6µs, 0/16 dispatch errors.

2. Security — a cognitive firewall

• Secret redaction: vaulted values never reach the model (inbound and in the response).
• Cognitive-DDoS / prompt-flood detection: an n-gram saturation scan flags floods → 0x0F red alert → apoptosis.
• Timeout guard on every dispatch.
• Red-team run: 0 secrets leaked (2 redacted); a 32,317-token flood dehydrated to 13 tokens in 0.6µs; a code-injection loop contained by timeout.

3. Speed — a lock-free hormonal bus

An atomic in-memory signalling substrate (~5µs) carries the alert state.
(Throughput benched at ~2M signals/s.)

No lock-in

Anthropic, Google, OpenAI, or a local model — same layer. You harden the payload
here and hand it to your SDK.

Why fair-source, not open source

The license is FSL-1.1-MIT: the code is source-available (read it, run it,
build on it), free for any non-competing use, and it auto-converts to MIT
after two years. I’m a solo dev — I wanted it visible and auditable without
someone reselling it as-is. It’s not OSI open source, and I’d rather say that
plainly than blur the line.

The point

BIOMA isn’t magic. The whole thing is one discipline: measure everything, and
keep only what survives the measurement — even when that means deleting the
feature you started with.

Repo (Rust + Python, benchmarks, and the honest FINDINGS.md):
https://github.com/jonathascordeiro20/bioma-framework

What would you attack first? I’ll be in the comments — especially happy to go deep
on the firewall’s saturation heuristic or the mitosis eval.