TypeScript의 토큰 자카드 유사성: 간단한 텍스트 비교

Hey TypeScript devs! 👋

If you need a fast, lightweight way to measure how similar two pieces of text are (duplicate detection, content recommendations, search result grouping, or data cleaning), Token Jaccard Similarity is an excellent choice.

It’s simple to understand, works great in both browser and Node.js, and requires zero dependencies.

What is Jaccard Similarity?

The Jaccard index measures similarity between two sets:

J(A,B)=∣A∩B∣∣A∪B∣J(A, B) = frac{|A cap B|}{|A cup B|}

• Intersection = tokens present in both texts
• Union = all unique tokens from either text

Result ranges from 0 (completely different) to 1 (identical).

Token Jaccard = Jaccard applied to words

We tokenize the text into words, convert them into Sets (removing duplicates and order), then apply the formula.

Quick Example

Text A:

the quick brown fox jumps over the lazy dog

Text B:

a quick brown fox jumps over a lazy dog

Unique tokens:

• Text A: the, quick, brown, fox, jumps, over, lazy, dog
• Text B: a, quick, brown, fox, jumps, over, lazy, dog

Intersection: 7 tokens

Union: 9 tokens

Jaccard Similarity ≈ 0.78 (78% similar)

TypeScript Implementation

Here’s a clean, fully typed implementation:

function tokenize(text: string): Set<string> {
  // Matches words (alphanumeric), lowercased
  const matches = text.toLowerCase().match(/bw+b/g);
  return new Set(matches ?? []);
}

function jaccardSimilarity(text1: string, text2: string): number {
  const set1 = tokenize(text1);
  const set2 = tokenize(text2);

  if (set1.size === 0 && set2.size === 0) {
    return 1;
  }

  // Calculate intersection size
  let intersectionSize = 0;
  for (const token of set1) {
    if (set2.has(token)) {
      intersectionSize++;
    }
  }

  const unionSize = set1.size + set2.size - intersectionSize;

  return intersectionSize / unionSize;
}

// ======================
// Example usage
// ======================

const textA = "the quick brown fox jumps over the lazy dog";
const textB = "a quick brown fox jumps over a lazy dog";

const score = jaccardSimilarity(textA, textB);
console.log(`Jaccard Similarity: ${score.toFixed(4)}`); 
// → Jaccard Similarity: 0.7778

Enter fullscreen mode Exit fullscreen mode

Another Example

const textC = "Building a recommendation system using TypeScript and machine learning";
const textD = "How to create a recommendation engine with TypeScript and ML";

console.log(jaccardSimilarity(textC, textD).toFixed(2)); 
// → 0.50

Enter fullscreen mode Exit fullscreen mode

Real-World Use Cases

Use Case Why It Works Well in TypeScript Suggested Threshold Duplicate content detection Extremely fast even on large datasets > 0.85 Article / blog recommendations Good for topic similarity 0.4 – 0.65 Dataset deduplication Lightweight preprocessing step > 0.90 Search result clustering Quick fuzzy grouping > 0.70 Basic plagiarism detection Simple and explainable baseline > 0.60

Limitations & Quick Improvements

Token Jaccard is great but has trade-offs:

• Ignores word order and semantics
• Sensitive to common words (“the”, “a”, “is”)

Easy Upgrades

1. Remove stop words before tokenizing
2. Use n-grams (word or character shingles) instead of single words
3. Combine with other metrics (e.g. cosine similarity on embeddings)

But for most practical needs, plain Token Jaccard in TypeScript is already very effective and blazing fast.

When to Use It

• You want something simple and dependency-free
• You’re working in TypeScript / JavaScript (frontend, backend, or full-stack)
• You need explainable results
• You’re building a quick prototype or data pipeline

Conclusion

Token Jaccard Similarity remains one of the most practical tools for text comparison. With this TypeScript version, you can drop it into any project instantly — browser, Node.js, Deno, or even Edge functions.

Would you like me to show:

• A version with stop-word removal?
• N-gram (shingle) based Jaccard?
• How to use it efficiently at scale?

Just let me know in the comments!