Kanoniv

Appearance

Tutorial: AutoTune a Spec ​

Start with a deliberately suboptimal identity spec and let AutoTune find improvements automatically. By the end you will understand how the optimizer explores mutations, scores candidates, and produces an improved spec with a full audit trail.

Time: 10 minutes Prerequisites: Python 3.9+, pip install kanoniv

What You Will Build ​

In this tutorial you will:

  1. Create two CSV sources with known overlapping records
  2. Write an intentionally weak spec (high threshold, no normalizers)
  3. Run autotune() to optimize the spec automatically
  4. Inspect the mutation log to understand what changed and why
  5. Compare before/after metrics and verify the improvement
  6. Save the mutation log for cross-dataset learning

Step 1: Create Sample Data ​

Create two CSV files with overlapping customers. The data has the kind of real-world messiness that identity resolution handles: name variations, case differences, and formatting inconsistencies.

crm.csv ​

csv
id,email,name,phone
c1,[email protected],Alice Smith,555-0001
c2,[email protected],Bob Jones,555-0002
c3,[email protected],Carol White,555-0003
c4,[email protected],Dave Brown,555-0004
c5,[email protected],Eve Davis,555-0005

orders.csv ​

csv
id,email,name,phone
o1,[email protected],Alice S.,555-0001
o2,[email protected],Robert Jones,555-0002
o3,[email protected],Frank Miller,555-0006
o4,[email protected],Carol W.,555-0003

Save both to your working directory. The overlaps:

  • Alice appears in both sources with the same email but different name formats
  • Bob has uppercase email in CRM, lowercase in orders, plus a name variation
  • Carol shares email and phone across both sources
  • Dave and Eve only appear in CRM; Frank only in orders

Step 2: Write a Deliberately Weak Spec ​

Create naive-spec.yaml with a high match threshold and no normalizers. This spec will under-merge because the threshold is too strict and there is no email normalization to handle the uppercase [email protected].

yaml
api_version: kanoniv/v2
identity_version: naive_v1.0
entity:
  name: customer
sources:
  - name: crm
    system: csv
    table: crm_export
    id: id
    attributes:
      email: email
      name: name
      phone: phone
  - name: orders
    system: csv
    table: order_export
    id: id
    attributes:
      email: email
      name: name
      phone: phone
blocking:
  strategy: composite
  keys:
    - [email]
rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
decision:
  thresholds:
    match: 0.99
    review: 0.5
  conflict_strategy: prefer_high_confidence

Problems with this spec:

  • match: 0.99 is too high - pairs need a near-perfect score to merge
  • No normalizer: email on the email rule - [email protected] will not match [email protected]
  • Only one blocking key - records that differ on email will never be compared
  • No phone rule at all

Step 3: Run the Baseline ​

Before running AutoTune, see what the naive spec produces:

python
from kanoniv import Spec, Source, reconcile

spec = Spec.from_file("naive-spec.yaml")
crm = Source.from_csv("crm", "crm.csv", primary_key="id")
orders = Source.from_csv("orders", "orders.csv", primary_key="id")

baseline = reconcile(sources=[crm, orders], spec=spec)
print(f"Clusters:   {baseline.cluster_count}")
print(f"Merge rate: {baseline.merge_rate:.1%}")

Expected output:

Clusters:   7
Merge rate: 22.2%

Only 2 of the 9 records merge (Alice and Carol match on email). Bob's uppercase email fails exact matching, and the 0.99 threshold is too strict for anything borderline.

Step 4: Run AutoTune ​

Now let AutoTune improve the spec:

python
from kanoniv import autotune, Source, Spec

spec = Spec.from_file("naive-spec.yaml")
crm = Source.from_csv("crm", "crm.csv", primary_key="id")
orders = Source.from_csv("orders", "orders.csv", primary_key="id")

result = autotune(
    sources=[crm, orders],
    spec=spec,
    max_iterations=30,
    max_conflict=0.05,
    verbose=True,
)

With verbose=True you will see AutoTune's progress through three phases:

Running baseline reconciliation...
  Baseline metrics: {'merge_rate': 0.2222, 'conflict_rate': 0.0, ...}

Phase 1: Tier 1 sweep (safe mutations)...
  Accepted: Add email normalizer to rule 'email_exact' (delta=0.1234)
  Accepted: Enable case-insensitive for rule 'email_exact' (delta=0.0567)
  Accepted: Add blocking key [phone] (delta=0.0890)

Phase 2: Tier 2 greedy (moderate mutations)...
  Accepted: Adjust match threshold 0.99 -> 0.94 (delta=0.0345)
  Accepted: Adjust match threshold 0.94 -> 0.89 (delta=0.0212)

Done. 5 mutations accepted in 18 iterations.

The three phases work as follows:

  1. Tier 1 (safe) - tries normalizers, case-insensitive flags, blocking keys, and graph clustering. Low risk, often high reward.
  2. Tier 2 (moderate) - nudges thresholds and weights by small amounts. Repeats until no single-step improvement is found.
  3. Tier 3 (risky) - only runs if Tier 2 made no progress. Tries larger threshold swings and algorithm switches.

Step 5: Inspect Results ​

Compare Metrics ​

python
print(result.summary())

This prints a before/after comparison:

AutoTune Results
==================================================

  Metrics Before -> After
  -----------------------
  conflict_rate: 0.0000 -> 0.0000 (0.0000)
  entropy: 0.0000 -> 0.0000 (0.0000)
  merge_rate: 0.2222 -> 0.4444 (+0.2222)
  stability: 1.0000 -> 0.6000 (-0.4000)

  Mutations accepted: 5/18
  Iterations used: 18

Key takeaways:

  • merge_rate doubled - more true matches found
  • conflict_rate stayed at 0 - no false merges introduced
  • stability decreased - expected, since the clusters changed from baseline

View the Diff ​

python
d = result.diff
if d.has_changes:
    print(d.summary)

This shows exactly what changed between the original and optimized spec.

Reconcile with the Optimized Spec ​

python
optimized = reconcile(sources=[crm, orders], spec=result.best_spec)
print(f"Clusters:   {optimized.cluster_count}")
print(f"Merge rate: {optimized.merge_rate:.1%}")

Step 6: Explore the Mutation Log ​

Every mutation AutoTune evaluates is recorded. This is the data you need for cross-dataset learning.

python
# Show accepted mutations
for entry in result.mutation_log:
    if entry.accepted:
        m = entry.mutation
        print(f"[Tier {m.tier}] {m.category}: {m.description}")
        print(f"  Score delta: {entry.score_delta:+.4f}")
        print()

Output:

[Tier 1] normalizer: Add email normalizer to rule 'email_exact'
  Score delta: +0.1234

[Tier 1] case_insensitive: Enable case-insensitive for rule 'email_exact'
  Score delta: +0.0567

[Tier 1] blocking: Add blocking key [phone]
  Score delta: +0.0890

[Tier 2] match_threshold: Adjust match threshold 0.99 -> 0.94
  Score delta: +0.0345

[Tier 2] match_threshold: Adjust match threshold 0.94 -> 0.89
  Score delta: +0.0212

Save the Log ​

python
result.save_log("autotune_log.json")

The JSON log contains every evaluation - accepted and rejected - with full metrics. Over time, patterns emerge across datasets:

  • "email normalizer almost always helps" - universal win
  • "phone blocking rarely hurts" - safe to apply broadly
  • "threshold below 0.7 increases conflict_rate" - dataset-dependent caution

Step 7: AutoTune with Labels ​

If you have labeled pairs (from active learning or manual review), AutoTune uses precision/recall/F1 instead of proxy metrics:

python
from kanoniv import autotune, FeedbackLabel, Source, Spec

spec = Spec.from_file("naive-spec.yaml")
crm = Source.from_csv("crm", "crm.csv", primary_key="id")
orders = Source.from_csv("orders", "orders.csv", primary_key="id")

labels = [
    FeedbackLabel("c1", "o1", "crm", "orders", "match"),   # Alice
    FeedbackLabel("c2", "o2", "crm", "orders", "match"),   # Bob
    FeedbackLabel("c3", "o4", "crm", "orders", "match"),   # Carol
    FeedbackLabel("c4", "o3", "crm", "orders", "no_match"),  # Dave != Frank
]

result = autotune(
    sources=[crm, orders],
    spec=spec,
    labels=labels,
    max_iterations=30,
)

print(result.summary())

When labels are provided, AutoTune automatically switches to supervised weights: f1=0.7, conflict_rate=0.2, stability=0.1. The optimizer maximizes F1 against your ground truth instead of relying on unsupervised proxies.

How the Scoring Works ​

AutoTune uses a composite score with z-score normalization to make weights dataset-independent:

score = w_merge * z(merge_rate)
      - w_conflict * z(conflict_rate)
      - w_entropy * z(entropy)
      + w_stability * z(stability)
  • merge_rate (higher is better) - fraction of records that were merged
  • conflict_rate (lower is better) - fraction of multi-member clusters with incompatible identifiers
  • entropy (lower is better) - Shannon entropy of identifier diversity within clusters
  • stability (higher is better) - Adjusted Rand Index comparing candidate clusters to the baseline

The max_conflict guardrail prevents any mutation that pushes conflict_rate above the threshold, regardless of score improvement. This approximates a precision floor without requiring labels.

Custom Weights ​

Override the default weights to fit your use case:

python
# Prioritize recall (more merges) over precision
result = autotune(
    sources=[crm, orders],
    spec=spec,
    weights={"merge_rate": 0.6, "conflict_rate": 0.2, "entropy": 0.1, "stability": 0.1},
    max_conflict=0.10,  # Allow higher conflict rate
)

# Prioritize precision (fewer false merges)
result = autotune(
    sources=[crm, orders],
    spec=spec,
    weights={"merge_rate": 0.1, "conflict_rate": 0.6, "entropy": 0.2, "stability": 0.1},
    max_conflict=0.02,  # Strict conflict ceiling
)

Next Steps ​

The identity and delegation layer for AI agents.

Copyright © 2026 Kanoniv