Matching Strategy Guide

How to choose rules, algorithms, weights, and thresholds that produce accurate matches without drowning you in false positives or missed pairs.

Matching Is a Precision-Recall Trade-off

Every matching configuration makes a bet. Too aggressive and you merge records that should stay separate -- false merges that require manual splits and erode trust in your golden record. Too conservative and you miss obvious duplicates, leaving fragmented identities across your systems.

The asymmetry matters: false merges are expensive to undo, missed matches are cheap to fix later. A split operation rewrites downstream consumers, triggers audit events, and may violate data contracts. A missed match just means you run another reconciliation pass with looser rules.

Start conservative. Tighten the review window. Let the review queue teach you where to loosen up.

Choosing Rule Types by Field

Not every field deserves the same treatment. Strong identifiers should use exact matching with high weights. Fuzzy fields should contribute less and require corroboration.

Field Type	Rule Type	Algorithm	Threshold	Weight	Notes
Email	exact	--	--	0.9 - 1.0	Strongest single identifier for most B2C data
Phone	exact	--	--	0.8 - 0.9	Normalize format before matching (E.164)
Person name	similarity	jaro_winkler	0.85 - 0.92	0.5 - 0.7	Never trust name alone; combine with another signal
Company name	similarity	levenshtein	0.80	0.4 - 0.6	Abbreviations and legal suffixes cause noise
Address	composite	AND (street similarity + zip exact)	0.80 / --	0.5 - 0.7	Street similarity alone is unreliable
Date of birth	exact	--	--	0.3 - 0.5	Good corroborator, not a primary key
Transaction amount	range	--	tolerance: 0.01	0.3 - 0.5	Use tolerance for rounding differences
Account number / SSN	exact	--	--	1.0	Treat as a hard identifier when present

The pattern: fields that uniquely identify a person get high weights. Fields that corroborate identity get medium weights. Fields that are common across many records (e.g., country, state) get low weights or are used only inside composites.

Algorithm Selection

Kanoniv supports five similarity algorithms. Each has a sweet spot.

jaro_winkler

Best for person names. The algorithm is prefix-weighted, which means it tolerates typos at the end of a string better than at the beginning. "Robert" vs "Roberr" scores higher than "Xobert" vs "Robert." This matches how real-world typos occur in names.

• Threshold range: 0.85 - 0.92
• When to use: Short strings where the first few characters are reliable. First names, last names.
• When to avoid: Long strings, addresses, company names with common prefixes ("First National Bank of...").

levenshtein

Best for short structured strings -- company names, street addresses, product names. Counts the minimum number of single-character edits (insert, delete, replace) to transform one string into the other, normalized by length.

• Threshold range: 0.75 - 0.90
• When to use: General-purpose fuzzy matching where edit distance is meaningful.
• When to avoid: Strings where word order varies ("123 Main St Apt 4" vs "Apt 4, 123 Main St").

soundex

Binary phonetic match for English names that sound alike. "Smith" and "Smyth" produce the same code. Returns 1.0 (match) or 0.0 (no match) -- there is no partial score.

• Threshold: 1.0 (binary)
• When to use: Name matching where spelling variations are common and all names are English.
• When to avoid: Non-English names, anything that is not a proper noun.

metaphone

Improved phonetic algorithm that handles complex consonant clusters better than soundex. Produces more discriminating codes for names like "Wright" vs "Right" vs "Rite."

• Threshold: 1.0 (binary)
• When to use: Same use cases as soundex, but with better accuracy on complex English phonetics.
• When to avoid: Non-English phonetics, non-name fields.

cosine

Token-based similarity using TF-IDF vectors. Best for long strings where word order varies -- full addresses, company descriptions, product titles.

• Threshold range: 0.60 - 0.80
• When to use: Multi-word fields where the same words appear in different orders or with different delimiters.
• When to avoid: Short strings (single words). The tokenization adds overhead and loses positional information.

Algorithm Comparison

Algorithm	Input Type	Output	Speed	Handles Typos	Handles Reordering
jaro_winkler	Character	0.0 - 1.0	Fast	Yes (end-weighted)	No
levenshtein	Character	0.0 - 1.0	Medium	Yes (uniform)	No
soundex	Phonetic	0.0 or 1.0	Fast	Partial	No
metaphone	Phonetic	0.0 or 1.0	Fast	Partial	No
cosine	Token	0.0 - 1.0	Slow	No	Yes

Composite Rules for Complex Logic

Single-field rules are building blocks. Composite rules combine them with AND / OR operators to express real matching logic.

AND: Corroboration

"Both conditions must be true." Use AND when a single signal is not strong enough on its own.

rules:
  - name: name_and_zip
    type: composite
    operator: and
    children:
      - name: name_sim
        type: similarity
        field: name
        algorithm: jaro_winkler
        threshold: 0.88
      - name: zip_exact
        type: exact
        field: zip_code

This fires only when the name is similar and the zip code matches. Neither signal alone would justify a match.

OR: Multiple Paths

"Any one of these is enough." Use OR when you have several independent identifiers.

rules:
  - name: any_strong_id
    type: composite
    operator: or
    children:
      - name: email_match
        type: exact
        field: email
      - name: phone_match
        type: exact
        field: phone
      - name: name_zip_combo
        type: composite
        operator: and
        children:
          - name: name_sim
            type: similarity
            field: name
            algorithm: jaro_winkler
            threshold: 0.90
          - name: zip_exact
            type: exact
            field: zip_code

This matches if email matches, OR phone matches, OR (name + zip) both match. The nested composite gives you defense-in-depth: the weakest path still requires two corroborating signals.

Blocking Strategy

Without blocking, the engine compares every record against every other record -- O(n^2). At 100K records, that is 5 billion comparisons. Blocking partitions records into candidate groups so only records in the same block are compared.

Strategy	Mechanism	Best For	Recall	Speed
exact	Partition by exact key value	Clean data with a reliable key (email domain, zip)	Lowest	Fastest
phonetic	Partition by phonetic code of a name field	Name-based matching with spelling variations	Medium	Fast
ngram	Partition by overlapping character n-grams	Dirty data, no single reliable key	Highest	Slowest

Guidelines:

• Always include your strongest identifier as a blocking key. If you match on email, block on email domain or the full email.
• Multiple blocking keys produce a union of candidate sets. Two keys means a record can appear in either block -- this increases recall at the cost of more comparisons.
• Blocking is not matching. A blocking strategy can be broader than your matching rules. Its job is to avoid missing candidates, not to produce final decisions.

blocking:
  strategy: phonetic
  keys: [last_name, zip_code]

This creates candidate pairs where either the phonetic code of last_name matches or the zip_code is identical. It is a good balance for person-matching with medium data quality.

Threshold Tuning

Decision thresholds control the boundary between auto-match, review, and reject. The right thresholds depend on your data quality and your tolerance for manual review.

Starting Point

Start here and adjust based on what you see in the review queue:

decision:
  thresholds:
    match: 0.90
    review: 0.70

Adjustment Rules

Symptom	Action
Too many false merges	Raise match threshold (e.g., 0.90 -> 0.95)
Too many items in review	Narrow the review window: raise review or lower match
Missing obvious matches	Lower match threshold or add more rules to boost scores
Review queue is empty but data has known dupes	Lower review threshold to surface candidates

Thresholds by Data Quality

Data Quality	match	review	Characteristics
High (normalized, deduplicated sources)	0.85	0.60	Clean emails, standardized names, consistent formats
Medium (some nulls, mixed formats)	0.90	0.70	Typical SaaS data, partial records, minor inconsistencies
Low (free-text, legacy systems)	0.95	0.80	OCR data, manual entry, high null rates

Higher data quality lets you match at lower thresholds because the signals are more reliable. Dirty data needs higher thresholds to compensate for noise.

Scoring Method Selection

The scoring method determines how individual rule results combine into a final score.

Method	Tier	When to Use
weighted_sum	All	Default for most use cases. Score = sum(rule_score * weight) / sum(weights). Simple, interpretable, auditable.
ml_ensemble	Cloud	You have labeled training data (prior merge/reject decisions) and want the model to learn inter-rule relationships. Captures non-linear patterns that weighted sums miss.
custom	Cloud	You need conditional logic: "name similarity only counts if email also matched." Implemented as a scoring function in your spec.

Use weighted_sum unless you have a specific reason not to. It is transparent -- you can explain every score to a stakeholder by showing which rules fired and their weights. Switch to ml_ensemble only after you have collected enough review decisions to train on (typically 500+ labeled pairs).

Common Patterns by Use Case

Customer 360

Merge customer records across CRM, billing, and support systems.

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_fuzzy
    type: similarity
    field: full_name
    algorithm: jaro_winkler
    threshold: 0.88
    weight: 0.6
  - name: phone_exact
    type: exact
    field: phone
    weight: 0.8
  - name: address_composite
    type: composite
    operator: and
    children:
      - type: similarity
        field: street
        algorithm: levenshtein
        threshold: 0.80
      - type: exact
        field: zip_code
decision:
  scoring: weighted_sum
  thresholds:
    match: 0.90
    review: 0.70
  conflict_strategy: prefer_high_confidence
blocking:
  strategy: exact
  keys: [email, phone]

Payment Deduplication

Detect duplicate transactions across payment processors.

rules:
  - name: account_exact
    type: exact
    field: account_id
    weight: 1.0
  - name: amount_range
    type: range
    field: amount
    tolerance: 0.01
    weight: 0.5
  - name: date_range
    type: range
    field: transaction_date
    tolerance: 86400    # 1 day in seconds
    weight: 0.4
  - name: reference_exact
    type: exact
    field: reference_number
    weight: 1.0
decision:
  thresholds:
    match: 0.95
    review: 0.80
  conflict_strategy: prefer_recent
blocking:
  strategy: exact
  keys: [account_id]

High match threshold here because false-positive dedup on payments means lost revenue.

Patient Matching

Match patient records across clinical systems. HIPAA requires auditability and conservative defaults.

rules:
  - name: name_fuzzy
    type: similarity
    field: patient_name
    algorithm: jaro_winkler
    threshold: 0.92
    weight: 0.6
  - name: dob_exact
    type: exact
    field: date_of_birth
    weight: 0.5
  - name: ssn_exact
    type: exact
    field: ssn_last4
    weight: 1.0
  - name: address_composite
    type: composite
    operator: and
    children:
      - type: similarity
        field: street
        algorithm: levenshtein
        threshold: 0.80
      - type: exact
        field: zip_code
decision:
  thresholds:
    match: 0.95
    review: 0.75
  conflict_strategy: manual_review
blocking:
  strategy: phonetic
  keys: [patient_name, date_of_birth]

conflict_strategy: manual_review forces all conflicts through the review queue. In healthcare, no merge should happen without a human in the loop for edge cases.

Lead Deduplication

Deduplicate inbound leads across marketing and sales tools.

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: company_fuzzy
    type: similarity
    field: company_name
    algorithm: levenshtein
    threshold: 0.80
    weight: 0.5
  - name: name_fuzzy
    type: similarity
    field: contact_name
    algorithm: jaro_winkler
    threshold: 0.85
    weight: 0.5
decision:
  thresholds:
    match: 0.85
    review: 0.65
  conflict_strategy: prefer_high_confidence
blocking:
  strategy: exact
  keys: [email]

Lower thresholds are acceptable here because the cost of merging two leads incorrectly is low compared to missing a duplicate and wasting a sales rep's time.

Anti-Patterns

Do not set all weights to 1.0. Weights encode your domain knowledge about signal strength. If every rule has the same weight, a name match counts as much as an email match. That is almost never correct.

Do not skip blocking on large datasets. Without blocking, reconciliation time grows quadratically. At 50K records, you are looking at hours instead of seconds.

Do not use thresholds below 0.7 for auto-match. A 0.6 match threshold will merge records where less than two-thirds of the evidence agrees. That produces false merges at a rate that will overwhelm your review queue with split requests.

Do not ignore the review queue. The review queue is your feedback loop. Patterns in review decisions tell you exactly which rules need tuning and in which direction. If you are not reviewing, you are flying blind.

Do not use fuzzy matching on short codes. Zip codes, country codes, currency codes, and boolean flags should always use exact matching. A Levenshtein similarity of 0.80 between "10001" and "10002" is meaningless -- those are different zip codes.

Do not match on a single fuzzy field without corroboration. A jaro_winkler score of 0.88 on "John Smith" tells you very little. Combine it with at least one other signal (zip, email domain, phone area code) before treating it as a match.

Next Steps

• Tuning Match Quality: Diagnose false positives/negatives and iterate
• Writing Your First Spec: Build a complete spec from scratch
• Rules Reference: Full spec reference for rule types and parameters
• Decision Reference: Scoring methods, thresholds, and conflict strategies