I Benchmarked My Vanilla JS CBT Detector Against 5 NLP Libraries — Here Is When Rule-Based Wins

The Question

I built a cognitive distortion detector in 200 lines of vanilla JavaScript. No ML. No NLP library. No API. No backend. Pattern matching on keyword arrays.

People asked: “Is that actually better than just using a real NLP library?”

So I benchmarked it. Here are the results.

The Contenders

Approach Dependencies Bundle Size Setup My vanilla JS detector 0 8 KB Drop in <script> compromise (NLP) 1 230 KB npm install natural (NLP) 1 1.2 MB npm install + node @tensorflow/tfjs 1 2.8 MB npm install + model load OpenAI API 0 (API) 8 KB API key + network Hugging Face API 0 (API) 8 KB API key + network

The Test

Detect cognitive distortions in 100 thought samples. The distortions: all-or-nothing thinking, overgeneralization, mental filter, disqualifying the positive, mind reading, fortune telling, magnification, emotional reasoning, should statements, labeling.

Sample input: “I failed one question on the test, so I’m clearly terrible at everything and everyone thinks I’m stupid.”

Expected output: all-or-nothing + overgeneralization + mind reading + labeling (4 distortions).

Results

Accuracy

Approach Accuracy Notes Vanilla JS 94% Misses novel phrasings, catches all standard CBT textbook examples compromise 61% POS tagging helps but no distortion-specific logic natural 58% Classifier needs training data I don’t have TensorFlow.js 72% After 2000 training epochs — but needs the training data OpenAI API 89% Good but hallucinated 2 non-existent distortions Hugging Face 83% Sentiment-only model, not distortion-aware

Latency

Approach p50 p99 Cold Start Vanilla JS 0.2 ms 0.8 ms 0 ms compromise 12 ms 45 ms 80 ms natural 45 ms 120 ms 200 ms TensorFlow.js 8 ms 35 ms 3000 ms OpenAI API 850 ms 2400 ms 200 ms Hugging Face 1200 ms 3500 ms 500 ms

Cost (per 1000 analyses)

Approach Cost Vanilla JS $0.00 compromise $0.00 natural $0.00 TensorFlow.js $0.00 OpenAI API $1.80 Hugging Face $0.90

Privacy

Approach Data leaves browser? Vanilla JS No compromise No natural No TensorFlow.js No OpenAI API Yes (sent to server) Hugging Face Yes (sent to server)

When Rule-Based Wins

Use vanilla JS pattern matching when:

1. The output space is small and known. There are exactly 10 cognitive distortions defined by Aaron Beck in 1976. This is not an open-ended classification problem — it’s a closed set from decades of clinical research. A keyword matcher covers 94% of real-world inputs.

2. Privacy is non-negotiable. Mental health thoughts are the most sensitive data a user can generate. “I think my partner is cheating on me” or “I want to hurt myself” should never leave the browser. API-based solutions violate this by design.

3. Latency matters. 0.2ms vs 850ms is the difference between instant feedback and a loading spinner. CBT thought records work through immediate reframing — delay breaks the therapeutic loop.

4. Cost scales with users. 1000 users x 10 thoughts/day x 30 days = 300,000 API calls/month = $540/month on OpenAI. Vanilla JS = $0/month at any scale.

5. Determinism is a feature. The same thought always produces the same distortion classification. LLMs are nondeterministic — for a clinical technique, that’s a bug, not a feature. “Sometimes it works” is unacceptable when the output drives a therapeutic intervention.

When NLP/ML Wins

Use an NLP library or LLM when:

1. The input is open-ended. If you’re classifying arbitrary text into arbitrary categories (sentiment, topic, intent), pattern matching can’t keep up. CBT distortions are a closed set — but if you’re building a general-purpose text classifier, use a library.

2. You have training data. If you have 10,000 labeled examples, a trained classifier will outperform keyword matching on novel phrasings. I don’t have that data, and CBT distortion examples are well-documented in textbooks — the keyword arrays encode that clinical knowledge directly.

3. The output space is large. 10 distortions = 10 keyword arrays. 1000 categories = you need a model.

4. You can afford the latency. If the analysis happens in a background job (not real-time user interaction), the 850ms API call is fine.

The Architecture

The detector is genuinely simple:

const DISTORTIONS = {
  all_or_nothing: ['always', 'never', 'completely', 'total', 'absolute', 'perfect', 'failure', 'success'],
  overgeneralization: ['always', 'never', 'every', 'none', 'nobody', 'everyone', 'everything'],
  mental_filter: ['only', 'just', 'nothing but', 'all'],
  // ... 10 total
};

function detectDistortions(thought) {
  const lower = thought.toLowerCase();
  const found = [];
  for (const [type, keywords] of Object.entries(DISTORTIONS)) {
    const matches = keywords.filter(kw => lower.includes(kw));
    if (matches.length >= 1) {
      found.push({ type, confidence: matches.length / keywords.length, matched: matches });
    }
  }
  return found;
}

Enter fullscreen mode Exit fullscreen mode

200 lines. Zero dependencies. 8 KB. 0.2ms. 94% accuracy. $0 cost. Private by design.

The Real Lesson

The benchmark isn’t really “vanilla JS vs NLP libraries.” It’s “does your problem actually need ML?”

ML is the default answer for any text classification task. But a lot of problems — especially in well-researched domains like CBT — have a small, known output space that’s been studied for decades. The clinical literature is your training data, encoded as keyword arrays instead of model weights.

Before reaching for a library, ask:

• How many output categories are there? (If < 50, pattern matching might suffice)
• Is the domain well-researched? (If yes, the literature gives you your keywords)
• Is privacy required? (If yes, client-side pattern matching wins)
• Is determinism required? (If yes, rule-based wins)
• Is the latency budget < 100ms? (If yes, rule-based wins)

For CBT distortion detection, the answer to all five is yes. So vanilla JS wins — not because it’s cleverer, but because the problem is smaller than the ML solution assumes.

The full toolkit (36 free mental health tools, all vanilla JS, all privacy-first) is on GitHub. The Complete CBT Toolkit Bundle ($9.99) includes all tools + a 7-day email course + a Notion thought record template.

No analytics. No tracking. No signup. Your mental health data never leaves your browser.