String Comparators

Every matching rule in Kanoniv compares two field values and returns a similarity score between 0.0 (no match) and 1.0 (perfect match). This page explains each algorithm, when to use it, and how to configure it.

Decision Framework

Start here. Pick the algorithm based on your data type:

Data Type	Recommended	Why	Example
Email, SSN, phone	exact	Identifiers should match exactly after normalization	[email protected] = [email protected]
Person names	jaro_winkler	Weights prefix matches, handles typos	"Jon Smith" ≈ "John Smith"
Addresses, free text	levenshtein	Counts insertions/deletions/substitutions	"123 Main St" ≈ "123 Main Street"
Names with spelling variants	soundex	Fast phonetic encoding	"Smith" ≈ "Smyth"
International names	metaphone	Better phonetic rules for non-English	"Schmidt" ≈ "Smith"
Company names, descriptions	cosine	Token-overlap handles word reordering	"Acme Inc" ≈ "Inc Acme Corp"
Lat/long coordinates	haversine	Great-circle distance on Earth's surface	40.7128,-74.0060 ≈ 40.7130,-74.0058
Numeric ranges (age, revenue)	exact with tolerance	Exact match with configurable tolerance	32 ≈ 33 (within range)

Exact

How It Works

Binary comparison: returns 1.0 if two values are identical after normalization, 0.0 otherwise. No partial credit.

This is the fastest comparator and should be your first choice for structured identifiers — email addresses, phone numbers, tax IDs, and any field where a partial match is meaningless.

Example

Value A	Value B	Score
[email protected]	[email protected]	1.0
[email protected]	[email protected]	0.0
555-0100	555-0100	1.0
555-0100	555-0101	0.0

YAML Config

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0

Best for: Structured identifiers (email, phone, SSN, tax ID) where partial matches are meaningless.

Jaro-Winkler

How It Works

Jaro-Winkler is a two-step algorithm:

1. Jaro distance counts the number of matching characters (characters that appear in both strings within a window) and transpositions (matching characters in different order). The formula produces a score between 0 and 1.

2. Winkler modification boosts the score when the first 1-4 characters match exactly. The idea: if two strings share a prefix, they're more likely to be the same name with a typo in the middle.

This prefix bonus makes Jaro-Winkler ideal for person names, where the first few characters are almost always correct ("Jon" vs "John", "Catherine" vs "Katherine").

Scoring Behavior

• Range: 0.0 to 1.0 (continuous)
• Prefix bonus: up to +0.1 for matching prefixes
• Typical match threshold: 0.85 – 0.92

Example

CRM Name	Stripe Name	Score
John Smith	Jon Smith	0.96
John Smith	John Smyth	0.93
Catherine Lee	Katherine Lee	0.88
John Smith	Jane Doe	0.40
Acme Corporation	Acme Corp	0.91

YAML Config

rules:
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.88
    weight: 0.6

Best for: Person names, short strings where prefix accuracy matters.

Levenshtein

How It Works

Levenshtein distance counts the minimum number of single-character edits (insertions, deletions, substitutions) needed to transform one string into another. Kanoniv normalizes this to a similarity score:

score = 1 - (edit_distance / max(len(a), len(b)))

Unlike Jaro-Winkler, Levenshtein treats all positions equally — there's no prefix bonus. This makes it better for addresses and free-text fields where errors can occur anywhere in the string.

Scoring Behavior

• Range: 0.0 to 1.0 (continuous)
• Each edit (insert, delete, substitute) reduces the score proportionally
• Typical match threshold: 0.80 – 0.90

Example

CRM Address	Billing Address	Score
123 Main Street	123 Main St	0.81
123 Main Street	123 Main Street	1.00
456 Oak Avenue	456 Oak Ave	0.85
123 Main Street	789 Elm Road	0.25
San Francisco	San Fransisco	0.93

YAML Config

rules:
  - name: address_fuzzy
    type: similarity
    field: street
    algorithm: levenshtein
    threshold: 0.80
    weight: 0.4

Best for: Addresses, company names, any field where edits can occur anywhere in the string.

Soundex

How It Works

Soundex encodes a string into a four-character code based on how it sounds in English: the first letter is kept, and subsequent consonants are mapped to digits (B/F/P/V → 1, C/G/J/K/Q/S/X/Z → 2, etc.). Vowels and repeated codes are dropped.

Two strings match if their Soundex codes are identical. This makes Soundex a binary comparator — it returns 1.0 (same code) or 0.0 (different code), with no partial scores.

Soundex is fast and useful for catching phonetic variations, but it's coarse: "Smith" and "Smyth" both encode to S530, but so might unrelated names. Use it as a supplementary rule, not your primary comparator.

Scoring Behavior

• Range: binary (1.0 or 0.0)
• No partial scoring — codes either match or they don't
• Threshold: not applicable (always use 1.0)

Example

Value A	Value B	Soundex A	Soundex B	Score
Smith	Smyth	S530	S530	1.0
Robert	Rupert	R163	R163	1.0
John	Jon	J500	J500	1.0
Smith	Jones	S530	J520	0.0
Catherine	Katherine	C365	K365	0.0

YAML Config

rules:
  - name: name_phonetic
    type: similarity
    field: name
    algorithm: soundex
    threshold: 1.0
    weight: 0.3

TIP

Soundex misses Catherine / Katherine because the first letter differs. Use metaphone for better cross-initial phonetic matching.

Best for: Supplementary name matching in English-language datasets. Pair with jaro_winkler for best results.

Metaphone

How It Works

Metaphone (specifically Double Metaphone) generates one or two phonetic codes per string using rules that better model English pronunciation than Soundex. It handles silent letters, consonant clusters, and common spelling variations.

Key advantages over Soundex:

• Handles "ph" → "f", "ck" → "k", "wr" → "r" and similar patterns
• Generates two codes (primary and alternate) to handle words with multiple pronunciations
• Better accuracy for non-Anglo names

Like Soundex, it's a binary comparator: 1.0 if any code matches, 0.0 otherwise.

Scoring Behavior

• Range: binary (1.0 or 0.0)
• Matches if primary or alternate code of A equals primary or alternate code of B
• Threshold: not applicable (always use 1.0)

Example

Value A	Value B	Score	Why
Catherine	Katherine	1.0	Both produce code K0RN
Schmidt	Smith	1.0	Both produce code XMT/SMT
Wright	Right	1.0	"Wr" → "R"
Stephen	Steven	1.0	"ph" → "f" → same code
John	Jane	0.0	Different phonetic codes

YAML Config

rules:
  - name: name_phonetic
    type: similarity
    field: name
    algorithm: metaphone
    threshold: 1.0
    weight: 0.3

Best for: Name matching with international spelling variants, datasets with mixed-origin names.

Cosine

How It Works

Cosine similarity treats each string as a bag of tokens (words), builds a vector of token frequencies, and computes the cosine of the angle between the two vectors. The result measures token overlap regardless of word order.

This makes cosine ideal for company names and descriptions where:

• Words may appear in different order: "Acme Inc" vs "Inc Acme"
• Extra words may be added: "Acme" vs "Acme Corporation"
• Abbreviations change token count: "International Business Machines" vs "IBM" (low score — use exact for abbreviations)

Scoring Behavior

• Range: 0.0 to 1.0 (continuous)
• Order-independent — "A B C" and "C B A" score 1.0
• Sensitive to token overlap, not character-level edits
• Typical match threshold: 0.70 – 0.85

Example

CRM Company	Stripe Company	Score
Acme Inc	Acme Incorporated	0.71
Acme Inc	Inc Acme	1.00
Global Tech Solutions	Global Tech Solutions Inc	0.87
Acme Inc	Beta Corp	0.00
First National Bank	National First Bank	1.00

YAML Config

rules:
  - name: company_match
    type: similarity
    field: company_name
    algorithm: cosine
    threshold: 0.75
    weight: 0.5

Best for: Company names, product descriptions, any multi-word field where word order varies.

Haversine

How It Works

Haversine computes the great-circle distance between two points on Earth's surface given their latitude and longitude. Kanoniv converts this to a similarity score based on a configurable maximum distance:

distance = haversine(lat1, lon1, lat2, lon2)  // in km
score = max(0, 1 - distance / max_distance)

Points within max_distance km get a score > 0; points farther apart score 0. Points at the same location score 1.0.

Scoring Behavior

• Range: 0.0 to 1.0 (continuous, linear decay)
• 0.0 when distance >= max_distance
• 1.0 when distance = 0

Example

With max_distance = 10 km:

Location A	Location B	Distance	Score
40.7128, -74.0060 (NYC)	40.7130, -74.0058	0.03 km	0.997
40.7128, -74.0060 (NYC)	40.7580, -73.9855 (Central Park)	5.1 km	0.49
40.7128, -74.0060 (NYC)	34.0522, -118.2437 (LA)	3,944 km	0.00

YAML Config

rules:
  - name: location_match
    type: similarity
    field: coordinates
    algorithm: haversine
    threshold: 0.5
    weight: 0.3

Best for: Matching records with geographic coordinates — store locations, delivery addresses with geocoding, IoT device positions.

Comparison Table

Algorithm	Speed	Partial Scores	Order Sensitive	Best Data Type
exact	Fastest	No (binary)	Yes	Identifiers
jaro_winkler	Fast	Yes	Yes (prefix-weighted)	Person names
levenshtein	Medium	Yes	Yes	Addresses, free text
soundex	Fast	No (binary)	Yes	English names (supplementary)
metaphone	Fast	No (binary)	Yes	International names (supplementary)
cosine	Medium	Yes	No	Company names, descriptions
haversine	Fast	Yes	N/A	Geographic coordinates

Pre-Match Normalization

Before comparison, Kanoniv can normalize field values using blocking transformations. These run at the blocking stage but also improve comparison quality:

Normalizer	What It Does	Example
normalize_phone	Strips formatting, adds country code	(555) 010-0100 → +15550100100
normalize_email	Lowercases, strips dots/plus aliases	[email protected] → [email protected]
normalize_name	Lowercases, removes titles/suffixes	Dr. John Smith Jr. → john smith
normalize_domain	Extracts domain from URL/email	https://www.acme.com/about → acme.com

See Blocking Strategies for the full list of transformations.

Combining Comparators

Use composite rules to combine multiple comparators with AND/OR logic:

rules:
  - name: address_match
    type: composite
    operator: and
    children:
      - name: street_fuzzy
        type: similarity
        field: street
        algorithm: levenshtein
        threshold: 0.85
        weight: 0.4
      - name: zip_exact
        type: exact
        field: zip_code
        weight: 0.3

• AND: All sub-rules must meet their thresholds. Use for multi-field matching where every field must agree.
• OR: Any sub-rule meeting its threshold counts. Use when records may match on different fields.

Next Steps

• Blocking Strategies — Reduce comparison pairs before algorithms run
• Scoring Strategies — How rule scores combine into match decisions
• Survivorship — What happens after records match