Migrating to Kanoniv

A practical guide for teams moving from Dedupe, Splink, or hand-rolled matching scripts to Kanoniv. This page covers the conceptual mapping between each tool and Kanoniv, side-by-side code comparisons, and a step-by-step migration checklist.

Why Migrate?

Record matching and entity resolution tools each make different trade-offs. Here is an honest comparison of the approaches and where Kanoniv fits.

Dimension	Hand-Rolled Scripts	Dedupe	Splink	Kanoniv
Config style	Ad-hoc Python/SQL	Python API + active learning	Python `SettingsCreator`	Declarative YAML spec
Scoring models	Whatever you write	Learned (active learning)	Fellegi-Sunter (probabilistic)	Exact, similarity, range, composite, ML ensemble
Survivorship / golden records	Manual priority code	Not included	Not included	Built-in strategies (source_priority, most_recent, most_complete, aggregate)
Blocking	Manual loops or SQL	Built-in predicates	`block_on()` rules	Configurable (exact, phonetic, ngram)
Review queue	Not included	Not included	Not included	Built-in with merge/reject/split
Audit trail	Not included	Not included	Not included	Every decision logged
Runtime dependency	Python only	Python only	Spark or DuckDB	Rust engine (in-process via PyO3), no external runtime
Spec version control	N/A	Training file	Python config	YAML file -- diffable, lintable, Git-friendly

Hand-rolled scripts offer maximum flexibility, but matching logic buried in application code is hard to audit, hard to test, and impossible to diff between releases. There is no standard way to express "what changed in our matching logic since last quarter."

Dedupe is a well-regarded Python library for deduplication. Its active-learning workflow is powerful when you have a human in the loop during model training. However, it does not produce golden records, has no survivorship layer, and the trained model is a binary artifact rather than a readable specification.

Splink is an excellent probabilistic record-linking library built on the Fellegi-Sunter model. It handles large-scale linking well, especially when paired with Spark or DuckDB. The trade-off is a hard dependency on one of those backends, a single statistical model (Fellegi-Sunter), and no survivorship or golden-record output.

Kanoniv takes a declarative, spec-driven approach. You define sources, matching rules, blocking, decision thresholds, and survivorship in a single YAML file. The reconciliation engine runs locally via a Rust core (no server required), or you can deploy to Kanoniv Cloud for a managed identity graph and real-time resolution API.

From Dedupe

Conceptual Mapping

Dedupe Concept	Kanoniv Equivalent
`dedupe.Dedupe(fields)`	`Spec.from_file("spec.yaml")` -- rules section
Field definitions (`'type': 'String'`)	Rule with `type: similarity`, `algorithm: jaro_winkler`
Field definitions (`'type': 'Exact'`)	Rule with `type: exact`
`deduper.prepare_training(data)` / `deduper.train()`	Not needed -- thresholds and weights are explicit in the spec
Active learning loop	Manual threshold tuning, or `type: ml` with pre-trained coefficients
`deduper.partition(data, threshold)`	`reconcile(sources, spec)`
Cluster output	`result.clusters` + golden records via survivorship

Side-by-Side Code

Dedupe:

python

import dedupe

fields = [
    {'field': 'name', 'type': 'String'},
    {'field': 'address', 'type': 'String'},
    {'field': 'email', 'type': 'Exact'},
]
deduper = dedupe.Dedupe(fields)
deduper.prepare_training(data)
# ... interactive training loop ...
deduper.train()
clusters = deduper.partition(data, threshold=0.5)

Kanoniv:

python

from kanoniv import Spec, Source, reconcile

spec = Spec.from_file("customer-spec.yaml")
sources = [Source.from_csv("crm", "contacts.csv")]
result = reconcile(sources, spec)
print(result.clusters)

With the spec file (customer-spec.yaml):

yaml

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.88
    weight: 0.6
  - name: address_fuzzy
    type: similarity
    field: address
    algorithm: levenshtein
    threshold: 0.80
    weight: 0.4

Key Differences

No training loop. Dedupe requires an interactive session to label pairs before it can score records. Kanoniv uses explicit rules and weights, so results are deterministic from the first run. If you prefer a learned model, you can use type: ml rules with pre-trained coefficients instead.
Deterministic, auditable results. The YAML spec is version-controlled. You can diff it, review it in a pull request, and know exactly what changed between runs.
Survivorship included. Dedupe outputs clusters but does not pick winning field values. Kanoniv produces golden records using configurable survivorship strategies (source_priority, most_recent, most_complete, aggregate).
No Python-only lock-in. The Kanoniv reconciliation engine is written in Rust and exposed to Python via PyO3. It runs in-process with no external runtime or server dependency.

From Splink

Conceptual Mapping

Splink Concept	Kanoniv Equivalent
`SettingsCreator(comparisons=[...])`	Spec YAML `rules:` section
`cl.ExactMatch("email")`	`type: exact`, `fields: [email]`
`cl.JaroWinklerAtThresholds("name", [0.9, 0.7])`	`type: similarity`, `algorithm: jaro_winkler`, `threshold: 0.88`
`cl.LevenshteinAtThresholds("address", [1, 2])`	`type: similarity`, `algorithm: levenshtein`, `threshold: 0.80`
`block_on("email")`	`blocking:` section with `strategy: exact`, `keys: [email]`
`linker.training.estimate_u_using_random_sampling()`	Not needed -- weights are set explicitly
`linker.inference.predict(threshold_match_probability=0.9)`	`reconcile(sources, spec)` with `decision.thresholds.match: 0.9`
`Linker(df, settings, db_api=DuckDBAPI())`	`Source.from_pandas()` or `Source.from_csv()` -- no DuckDB/Spark required

Side-by-Side Code

Splink:

python

from splink import Linker, SettingsCreator, block_on
import splink.comparison_library as cl

settings = SettingsCreator(
    link_type="dedupe_only",
    comparisons=[
        cl.ExactMatch("email"),
        cl.JaroWinklerAtThresholds("name", [0.9, 0.7]),
        cl.LevenshteinAtThresholds("address", [1, 2]),
    ],
    blocking_rules_to_generate_predictions=[
        block_on("email"),
        block_on("name"),
    ],
)
linker = Linker(df, settings, db_api=DuckDBAPI())
linker.training.estimate_u_using_random_sampling(max_pairs=1e6)
results = linker.inference.predict(threshold_match_probability=0.9)

Kanoniv:

python

from kanoniv import Spec, Source, reconcile

spec = Spec.from_file("customer-spec.yaml")
sources = [Source.from_pandas("crm", df)]
result = reconcile(sources, spec)
print(result.clusters)

With the spec file:

yaml

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.88
    weight: 0.6
  - name: address_fuzzy
    type: similarity
    field: address
    algorithm: levenshtein
    threshold: 0.80
    weight: 0.4
blocking:
  strategy: exact
  keys: [email]
decision:
  thresholds:
    match: 0.9
    review: 0.7
survivorship:
  default: most_recent

Key Differences

No u/m parameter estimation. Splink requires a training step to estimate the Fellegi-Sunter u and m parameters via random sampling and expectation maximization. Kanoniv uses explicit weights -- you set them directly based on domain knowledge or tune them empirically.
No Spark or DuckDB dependency. Splink delegates computation to an external SQL engine. Kanoniv runs entirely in-process via its Rust engine. For larger datasets, you can scale horizontally with Kanoniv Cloud.
Survivorship built in. Splink produces linked record pairs but does not generate golden records. Kanoniv's survivorship layer selects the best value for each field across matched records.
Single YAML file vs. Python config. Splink's configuration lives in Python code. Kanoniv's spec is a standalone YAML file that can be validated (validate(spec)), planned (plan(spec)), and diffed (diff(old_spec, new_spec)) independently of any application code.
Multiple scoring methods. Splink uses Fellegi-Sunter exclusively. Kanoniv supports exact matching, multiple similarity algorithms (Jaro-Winkler, Levenshtein, Soundex, Metaphone, cosine), range matching, composite rules, and ML ensemble scoring in a single spec.

From Hand-Rolled Scripts

If your matching logic lives in application code -- nested if statements, fuzzywuzzy calls, or SQL self-joins -- Kanoniv gives you a structured replacement with the same flexibility.

Common Patterns and Their Kanoniv Equivalents

Exact field comparison:

python

# Before
if a["email"].lower() == b["email"].lower():
    mark_match(a, b)

yaml

# After
rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0

Fuzzy string matching:

python

# Before
from fuzzywuzzy import fuzz
if fuzz.ratio(a["name"], b["name"]) > 85:
    mark_match(a, b)

yaml

# After
rules:
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.85
    weight: 0.6

Nested conditional logic:

python

# Before
if a["email"] == b["email"]:
    mark_match(a, b)
elif fuzz.ratio(a["name"], b["name"]) > 90 and a["zip"] == b["zip"]:
    mark_match(a, b)
elif fuzz.ratio(a["name"], b["name"]) > 80:
    mark_review(a, b)

yaml

# After
rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_and_zip
    type: composite
    operator: and
    children:
      - type: similarity
        field: name
        algorithm: jaro_winkler
        threshold: 0.90
      - type: exact
        field: zip
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.80
    weight: 0.4
decision:
  thresholds:
    match: 0.9
    review: 0.7

Manual priority picking:

python

# Before
def pick_best(records):
    for r in records:
        if r["source"] == "salesforce":
            return r
    return records[0]

yaml

# After
survivorship:
  default: most_recent
  overrides:
    - fields: [name, email]
      strategy: source_priority
      priority: [salesforce, stripe, hubspot]

Why This Is Better

Auditable. The spec is a single artifact you can attach to a change request. Reviewers see exactly what matching logic is in production.
Testable. Run validate(spec) in CI to catch configuration errors before deployment. Run plan(spec) to preview the execution strategy.
Diffable. Use diff(old_spec, new_spec) to see exactly what changed between two versions of your matching logic -- which rules were added, which thresholds moved, which survivorship strategies changed.
Separates logic from code. Matching rules change more often than application code. A YAML spec lets data engineers iterate on rules without redeploying your application.

Migration Checklist

Use this checklist to structure your migration, regardless of which tool you are migrating from.

[ ] Inventory your matching fields. List every field used in your current matching logic (email, name, phone, address, account ID, etc.).
[ ] Map each comparison to a Kanoniv rule type. Exact equality becomes type: exact. String similarity becomes type: similarity with the appropriate algorithm. Compound conditions become type: composite.
[ ] Set initial weights based on field discriminative power. Strong identifiers (email, SSN) get weights near 1.0. Weaker signals (name, city) get 0.3-0.5. See the Matching Strategy guide for weight recommendations.
[ ] Configure blocking keys from your existing blocking logic. If you currently block on email prefix or zip code, translate that to the blocking: section with strategy: exact, phonetic, or ngram.
[ ] Set conservative decision thresholds. Start with match: 0.9 and review: 0.7. It is easier to lower thresholds later than to undo incorrect merges.
[ ] Add survivorship rules for golden record field selection. Decide which source wins for each field, or use most_recent or most_complete as a default strategy.
[ ] Run on a sample dataset and compare against previous results. Use Source.from_csv() or Source.from_pandas() to load a sample, then compare Kanoniv's clusters to your existing output.
[ ] Iterate on thresholds and weights. Adjust based on false-positive and false-negative rates. Use plan(spec) to preview the execution strategy after each change.
[ ] Deploy to Kanoniv Cloud for production (optional). If you need a managed identity graph, real-time resolution API, or multi-tenant isolation, deploy your spec to Kanoniv Cloud with from kanoniv import Client.

Loading Your Data

Kanoniv's source adapters make it straightforward to bring data in from wherever it lives today:

python

from kanoniv import Source

# From CSV files
crm = Source.from_csv("crm", "contacts.csv")

# From pandas DataFrames (useful during migration testing)
import pandas as pd
df = pd.read_csv("legacy_export.csv")
legacy = Source.from_pandas("legacy", df)

# From a data warehouse
warehouse = Source.from_warehouse("analytics", connection_string="...")

# From dbt models
dbt_source = Source.from_dbt("dbt_customers", manifest_path="target/manifest.json")

During migration, Source.from_pandas() is especially useful: load your existing tool's output into a DataFrame, run it through Kanoniv, and compare the results side by side.

Next Steps

Writing Your First Spec -- End-to-end walkthrough of creating and running a spec
Matching Strategy Guide -- Deep dive into rule types, weights, and threshold tuning
Python SDK -- Full reference for Spec, Source, validate, plan, reconcile
Kanoniv Cloud -- Managed identity graph and real-time resolution API
Source Adapters -- CSV, pandas, warehouse, and dbt adapter reference

Migrating to Kanoniv ​

Why Migrate? ​

From Dedupe ​

Conceptual Mapping ​

Side-by-Side Code ​

Key Differences ​

From Splink ​

Conceptual Mapping ​

Side-by-Side Code ​

Key Differences ​

From Hand-Rolled Scripts ​

Common Patterns and Their Kanoniv Equivalents ​

Why This Is Better ​

Migration Checklist ​

Loading Your Data ​

Next Steps ​

Migrating to Kanoniv

Why Migrate?

From Dedupe

Conceptual Mapping

Side-by-Side Code

Key Differences

From Splink

Conceptual Mapping

Side-by-Side Code

Key Differences

From Hand-Rolled Scripts

Common Patterns and Their Kanoniv Equivalents

Why This Is Better

Migration Checklist

Loading Your Data

Next Steps