Tuning Match Quality

Identity resolution is not set-and-forget. Your first spec will produce results, but those results will need refinement. This guide walks through a systematic workflow for diagnosing match quality issues and iterating toward better precision and recall.

The Iteration Loop

Every tuning cycle follows the same pattern:

Run -- Execute reconciliation with your current spec
Inspect -- Examine decisions, scores, and clusters
Adjust -- Change rules, weights, or thresholds based on what you found
Repeat -- Run again and compare against the previous results

python

from kanoniv import Spec, Source, validate, plan, diff, reconcile

spec = Spec.from_file("specs/customer.yaml")
validate(spec).raise_on_error()

sources = [
    Source.from_csv("test-dataset/stripe_customers.csv", system="stripe"),
    Source.from_csv("test-dataset/sf_contacts.csv", system="salesforce"),
]

result = reconcile(sources, spec)

print(f"Clusters: {result.cluster_count}")
print(f"Merge rate: {result.merge_rate:.2%}")
print(f"Decisions: {len(result.decisions)}")

Each decision in result.decisions includes the pair's score, the outcome (match, review, or reject), and the individual rule contributions that produced that score. This is the raw material for every diagnosis below.

Threshold Decision Tree

Use this decision tree after each reconciliation run to quickly diagnose whether your thresholds need adjustment:

Step 1 — Check Merge Rate

Merge Rate	Interpretation	Next Step
< 5%	Too strict — most true matches are being rejected	Go to "Diagnosing False Negatives"
5 – 30%	Normal range for cross-source reconciliation	Step 2
> 50%	Suspiciously high — likely over-merging	Go to "Diagnosing False Positives"

Step 2 — Spot-Check Auto-Matches

Sample 20 auto-merged pairs and manually verify them:

Wrong Pairs	Interpretation	Action
0 / 20	Thresholds are conservative — you may be missing true matches	Lower `match` threshold by 0.05
1 – 2 / 20	Acceptable false positive rate	Focus on improving recall
3+ / 20	Too many false positives	Raise `match` threshold or add corroboration rules

Step 3 — Check the Review Queue

Pattern	What It Means	Action
Mostly approvals	`match` threshold is too high	Lower `match` threshold toward the review range
Mostly rejections	`review` threshold is too low	Raise `review` threshold to reduce noise
Mixed (50/50)	Rules are not discriminative enough	Improve rules and weights, not thresholds

Diagnosing False Positives

False positives are records that were merged but should not have been. They erode trust in your identity graph.

Symptoms

Clusters contain records that clearly belong to different entities
Golden records have conflicting field values (two different addresses, unrelated companies)
Downstream consumers report incorrect enrichment

Investigation

Inspect the decisions that produced auto-merges and look at the score breakdown:

python

auto_matches = [d for d in result.decisions if d.outcome == "match"]

for decision in sorted(auto_matches, key=lambda d: d.score, reverse=True)[:10]:
    print(f"Score: {decision.score:.3f}")
    print(f"  Pair: {decision.record_a_id} <-> {decision.record_b_id}")
    for contrib in decision.contributions:
        print(f"  {contrib.rule_name}: {contrib.score:.3f} (weight: {contrib.weight})")
    print()

Common Causes and Fixes

Cause	What You See	Fix
Name-only matching without corroboration	High scores from `name_fuzzy` alone on common names	Add a composite AND rule requiring name + another signal
Low match threshold	Many auto-merges at 0.70-0.80 with thin evidence	Raise `match` threshold (e.g., 0.70 to 0.85)
High weight on low-discriminative fields	Generic emails like `info@` or shared phone numbers driving merges	Lower the weight on that rule, or exclude known-shared values
Missing blocking keys	Unrelated records compared and matched on weak signals	Add blocking keys to reduce the comparison space

Example Fix: Add Corroboration

If name matching alone is causing false positives, require a second signal:

yaml

rules:
  - name: name_and_email
    type: composite
    operator: and
    children:
      - field: name
        type: similarity
        algorithm: jaro_winkler
        threshold: 0.85
      - field: email
        type: exact

This rule only fires when both name and email match, eliminating merges based on name alone.

Diagnosing False Negatives

False negatives are records that should have matched but were rejected or never compared. They leave your identity graph fragmented.

Symptoms

Known duplicates appear as separate clusters
Merge rate is suspiciously low relative to your data overlap
Manual inspection reveals obvious matches that the engine missed

Investigation

Check rejected pairs and look for patterns:

python

rejects = [d for d in result.decisions if d.outcome == "reject"]

for decision in sorted(rejects, key=lambda d: d.score, reverse=True)[:10]:
    print(f"Score: {decision.score:.3f}")
    print(f"  Pair: {decision.record_a_id} <-> {decision.record_b_id}")
    for contrib in decision.contributions:
        print(f"  {contrib.rule_name}: {contrib.score:.3f} (weight: {contrib.weight})")
    print()

Also check whether pairs were never compared at all. If your blocking keys are too restrictive, some true matches will never enter the comparison stage.

Common Causes and Fixes

Cause	What You See	Fix
Blocking keys too restrictive	True matches with different email domains never compared	Add additional blocking keys (e.g., block on normalized phone OR email)
Threshold too high	Many rejects scoring 0.80-0.89 that are true matches	Lower `match` threshold
Missing rules for important signals	A discriminative field (e.g., account ID) is not used in any rule	Add a rule for that field
Data quality issues	Inconsistent formatting prevents exact matches	Normalize data before reconciliation (lowercase emails, strip phone formatting)
Misaligned name fields	Name rule scores 0.0 on every pair — one source has `first_name`, the other has `name`, mapped to different canonical fields	Map both to the same canonical field (see Sources: Attributes)
Denominator dilution	True matches with email+name scoring 0.55 (review) because unmatched phone rule drags down the weighted average	Lower the unmatched rule's weight, or lower the merge threshold

Using the Review Queue

The review queue holds pairs that scored between the review and match thresholds. It is your primary feedback mechanism for calibrating thresholds.

Reading the Signal

Pattern	What It Means	Action
You approve most reviews	Match threshold is too high	Lower `match` threshold toward the review range
You reject most reviews	Review threshold is too low	Raise `review` threshold to filter out noise
Approvals and rejections are split 50/50	Rules are not discriminative enough	Improve rules and weights, not thresholds

If the high-confidence bucket is large and you consistently approve those reviews, that is a strong signal to lower your match threshold by a few points.

Using Diff to Track Changes

Before deploying a spec change, always diff the old and new versions to understand exactly what changed:

python

from kanoniv import Spec, diff

old_spec = Spec.from_file("customer-spec-v1.yaml")
new_spec = Spec.from_file("customer-spec-v2.yaml")

changes = diff(old_spec, new_spec)
print(changes.summary())

The diff output shows added, removed, and modified rules, threshold changes, and blocking key adjustments. This serves two purposes:

Pre-deployment review -- Confirm you changed what you intended and nothing else
Audit trail -- Record what changed between versions for your team

Use plan() to verify the execution plan reflects your intent:

python

from kanoniv import plan

new_plan = plan(new_spec)
print(f"Plan hash: {new_plan.plan_hash}")
print(new_plan.summary())

for strategy in new_plan.match_strategies:
    print(f"  {strategy['name']}: weight={strategy.get('weight')}")

The plan_hash is a deterministic fingerprint of the execution plan. If two specs produce the same plan_hash, they will produce identical results. Use this to confirm your change actually affected the plan.

Comparing Runs

To measure the impact of a spec change, compare reconciliation results side by side:

python

from kanoniv import Spec, Source, reconcile

sources = [
    Source.from_csv("test-dataset/stripe_customers.csv", system="stripe"),
    Source.from_csv("test-dataset/sf_contacts.csv", system="salesforce"),
]

old_spec = Spec.from_file("customer-spec-v1.yaml")
new_spec = Spec.from_file("customer-spec-v2.yaml")

old_result = reconcile(sources, old_spec)
new_result = reconcile(sources, new_spec)

print(f"{'Metric':<20} {'v1':>10} {'v2':>10} {'Delta':>10}")
print("-" * 52)
print(f"{'Clusters':<20} {old_result.cluster_count:>10} {new_result.cluster_count:>10} "
      f"{new_result.cluster_count - old_result.cluster_count:>+10}")
print(f"{'Merge rate':<20} {old_result.merge_rate:>10.2%} {new_result.merge_rate:>10.2%} "
      f"{new_result.merge_rate - old_result.merge_rate:>+10.2%}")

A decrease in cluster count means more records are being merged. An increase means records that were previously merged are now separate. Neither is inherently good or bad -- the question is whether the change moves you toward ground truth.

For deeper analysis, compare the cluster membership between runs to find pairs that were merged in one version but not the other.

Weight Calibration

Scoring in Kanoniv is normalized: the final score is the sum of weighted rule contributions divided by the sum of all weights.

score = sum(rule_score_i * weight_i) / sum(weight_i)

This means weights are relative, not absolute. Doubling all weights changes nothing. What matters is the ratio between weights.

Systematic Approach

Review the score distribution. Plot or bucket the scores of your decisions.

python

from collections import Counter

buckets = Counter()
for d in result.decisions:
    bucket = round(d.score, 1)
    buckets[bucket] += 1

for score in sorted(buckets):
    print(f"  {score:.1f}: {'#' * buckets[score]} ({buckets[score]})")

Interpret the distribution.

Observation	Interpretation	Action
Most auto-matches score 0.95+	Thresholds may be too low; you are only catching obvious matches	Lower thresholds to capture more true matches
Most reviews score 0.85+	Review band contains mostly true matches	Raise items above 0.85 to auto-match
Scores cluster near 0.5	Rules are not discriminative	Increase weights on strong signals, decrease on weak ones

Adjust one weight at a time. Increase weight on fields that discriminate well (few false positives when they match). Decrease weight on fields that cause false positives (common values, low entropy).

Worked Example: Score Distribution

A well-tuned spec produces a bimodal score distribution: true matches cluster near 1.0 and true non-matches cluster near 0.0. The gap between is the "gray zone" — pairs where the engine is uncertain.

python

from collections import Counter

# After running reconcile()
buckets = Counter()
for d in result.decisions:
    bucket = round(d["confidence"], 1)
    buckets[bucket] += 1

print("Score Distribution:")
for score in sorted(buckets):
    bar = "#" * min(buckets[score], 60)
    print(f"  {score:.1f} | {bar} ({buckets[score]})")

What you want to see:

  0.0 | ############################################ (220)
  0.1 | ### (15)
  0.2 | # (5)
  0.3 |  (1)       <-- gap: very few pairs in the gray zone
  0.4 |  (0)
  0.5 |  (0)
  0.6 |  (2)
  0.7 | # (4)
  0.8 | ## (10)
  0.9 | ########################### (135)
  1.0 | ####################### (115)

What indicates problems:

No gap (scores spread evenly from 0 to 1): Rules are not discriminative — add stronger signals or increase weights on high-value fields.
Gray zone crowding (many scores at 0.4-0.7): Consider adding rules that push ambiguous pairs toward clear match/reject. Composite rules with corroboration help here.
Everything clustered at 0.9+: Thresholds are too low — you're only catching obvious matches. Lower the match threshold to capture the gray zone.

Narrowing the gray zone with weight adjustment:

If email matches are highly reliable but name matches are noisy, increase the email rule weight relative to name. This pushes true matches (which share email) higher and false matches (which only share name) lower, widening the gap.

Testing with Labeled Data

If you have ground truth -- a set of known matches and known non-matches -- use it as a regression test.

Setup

Create a labeled CSV with columns for record pairs and expected outcomes:

csv

record_a_id,record_b_id,expected
stripe_101,sf_201,match
stripe_102,sf_202,match
stripe_103,sf_203,no_match
stripe_104,sf_204,no_match

Evaluate

python

import csv
from kanoniv import Spec, Source, reconcile

spec = Spec.from_file("specs/customer.yaml")
sources = [
    Source.from_csv("test-dataset/stripe_customers.csv", system="stripe"),
    Source.from_csv("test-dataset/sf_contacts.csv", system="salesforce"),
]
result = reconcile(sources, spec)

# Build a lookup of actual decisions
actual = {}
for d in result.decisions:
    key = (d.record_a_id, d.record_b_id)
    actual[key] = d.outcome

# Compare against labeled data
tp, fp, tn, fn = 0, 0, 0, 0
with open("labeled_pairs.csv") as f:
    for row in csv.DictReader(f):
        key = (row["record_a_id"], row["record_b_id"])
        predicted = actual.get(key, "reject")
        expected = row["expected"]

        if expected == "match" and predicted == "match":
            tp += 1
        elif expected == "no_match" and predicted == "match":
            fp += 1
        elif expected == "no_match" and predicted != "match":
            tn += 1
        elif expected == "match" and predicted != "match":
            fn += 1

precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
print(f"Precision: {precision:.2%}")
print(f"Recall:    {recall:.2%}")
print(f"F1:        {2 * precision * recall / (precision + recall):.2%}" if (precision + recall) > 0 else "F1: 0%")

Run this after every spec change. If precision drops, you introduced false positives. If recall drops, you introduced false negatives.

Production Monitoring

Once your spec is deployed, ongoing monitoring catches regressions before they affect downstream systems.

Key Metrics to Track

Metric	What to Watch	Alert Condition
`merge_rate`	Sudden changes indicate data or rule drift	Change > 5% between consecutive runs
`cluster_count`	Large swings mean merge/split behavior shifted	Change > 10% between runs
Review queue depth	Growing queue means thresholds or rules need attention	Queue depth doubles week-over-week
Decision score distribution	Shift in score distribution signals data quality change	Median score moves > 0.05

Spec Validation on Update

Always validate before deploying a spec change:

python

from kanoniv import Spec, validate

spec = Spec.from_file("specs/customer.yaml")
result = validate(spec)
result.raise_on_error()

validate() catches field reference errors (referencing a field not defined in any source) and provides typo suggestions when a field name is close to a defined attribute.

Conflict Resolution

When multiple sources provide conflicting values for the same field, Kanoniv uses the conflict strategy defined in your spec's survivorship section:

Strategy	Behavior
`prefer_high_confidence`	Use the value from the source with the highest match confidence
`prefer_recent`	Use the most recently updated value
`manual_review`	Flag the conflict for human resolution

Monitor conflict rates alongside match quality. A spike in conflicts often indicates a new data source or a formatting change in an existing source.

Next Steps

Matching Strategy: Choose the right rules, algorithms, and thresholds
Writing Your First Spec: Build a complete spec from scratch
Spec Reference: Full YAML spec schema documentation
Python SDK (Core): Spec validation, planning, diffing, and local reconciliation

Tuning Match Quality ​

The Iteration Loop ​

Threshold Decision Tree ​

Diagnosing False Positives ​

Symptoms ​

Investigation ​

Common Causes and Fixes ​

Example Fix: Add Corroboration ​

Diagnosing False Negatives ​

Symptoms ​

Investigation ​

Common Causes and Fixes ​

Using the Review Queue ​

Reading the Signal ​

Using Diff to Track Changes ​

Comparing Runs ​

Weight Calibration ​

Systematic Approach ​

Worked Example: Score Distribution ​

Testing with Labeled Data ​

Setup ​

Evaluate ​

Production Monitoring ​

Key Metrics to Track ​

Spec Validation on Update ​

Conflict Resolution ​

Next Steps ​

Tuning Match Quality

The Iteration Loop

Threshold Decision Tree

Diagnosing False Positives

Symptoms

Investigation

Common Causes and Fixes

Example Fix: Add Corroboration

Diagnosing False Negatives

Symptoms

Investigation

Common Causes and Fixes

Using the Review Queue

Reading the Signal

Using Diff to Track Changes

Comparing Runs

Weight Calibration

Systematic Approach

Worked Example: Score Distribution

Testing with Labeled Data

Setup

Evaluate

Production Monitoring

Key Metrics to Track

Spec Validation on Update

Conflict Resolution

Next Steps