Kanoniv vs Splink
Bottom line: Splink is an excellent probabilistic record linkage library for data scientists who need statistical rigor and model interpretability. Kanoniv is a declarative identity resolution platform that includes golden record creation, real-time APIs, and multi-tenant cloud deployment. Choose Splink for statistical record linkage research; choose Kanoniv for production identity resolution with survivorship and serving.
At a Glance
| Kanoniv | Splink | |
|---|---|---|
| Type | Identity resolution platform | Record linkage library |
| Approach | Declarative rules (YAML spec) | Probabilistic (Fellegi-Sunter + EM) |
| Language | Python SDK (Rust engine) | Python (SQL generation) |
| License | Free SDK + Cloud | MIT |
| GitHub Stars | -- | ~1,900 |
| Configuration | YAML file | Python code |
| Backends | Local (PyO3) + Cloud API | DuckDB, Spark, Athena, PostgreSQL, SQLite |
| Golden Records | Yes (survivorship strategies) | No |
| Real-time API | Yes (sub-ms) | No |
| Multi-tenant | Yes (RLS isolation) | No |
| Built by | Kanoniv | UK Ministry of Justice |
Feature Comparison
| Feature | Kanoniv | Splink |
|---|---|---|
| Deterministic matching | Yes | Yes (via blocking rules) |
| Fuzzy matching | Yes (Jaro-Winkler, Levenshtein, phonetic) | Yes (Jaro-Winkler, Jaro, Levenshtein, date comparisons) |
| Probabilistic matching | Yes (Fellegi-Sunter with EM training) | Yes (core feature -- Fellegi-Sunter) |
| Survivorship / golden records | Yes (source priority, recency, aggregation) | No |
| Identity graph | Yes (persistent, queryable) | No (outputs clusters only) |
| Real-time resolution API | Yes (sub-millisecond) | No (library only) |
| Batch reconciliation | Yes | Yes |
| Incremental matching | Yes | Partial (find_matches_to_new_records) |
| Multi-tenant isolation | Yes (row-level security) | No |
| Audit logs | Yes (immutable, with reason codes) | No |
| HIPAA compliance features | Yes (PII masking, retention policies) | No |
| Visualization / diagnostics | Run health dashboard | Waterfall charts, match weight histograms |
| Model persistence | Spec is the model (YAML) | Settings serialized to JSON |
| Warehouse integration | Snowflake, dbt adapters | DuckDB, Spark, Athena, PostgreSQL |
| Training data required | No | No (EM estimates parameters) |
Code Comparison
Kanoniv: Define rules in YAML, reconcile in Python
# customer-spec.yaml
entity:
name: customer
sources:
- name: crm
adapter: csv
location: contacts.csv
primary_key: id
- name: billing
adapter: csv
location: stripe.csv
primary_key: id
rules:
- name: email_exact
type: exact
field: email
weight: 1.0
- name: name_fuzzy
type: jaro_winkler
field: name
threshold: 0.9
weight: 0.8
survivorship:
strategy: source_priority
priority: [crm, billing]
decision:
thresholds:
match: 0.85from kanoniv import Spec, Source, reconcile, validate
spec = Spec.from_file("customer-spec.yaml")
validate(spec).raise_on_error()
sources = [
Source.from_csv("crm", "contacts.csv"),
Source.from_csv("billing", "stripe.csv"),
]
result = reconcile(sources, spec)
print(f"Golden records: {len(result.golden_records)}")
print(f"Merge rate: {result.merge_rate:.1%}")Splink: Define comparisons in Python, train and predict
import splink.comparison_library as cl
from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets
db_api = DuckDBAPI()
df = splink_datasets.fake_1000
settings = SettingsCreator(
link_type="dedupe_only",
comparisons=[
cl.JaroWinklerAtThresholds("first_name", [0.9, 0.7]),
cl.JaroAtThresholds("surname", [0.9, 0.7]),
cl.DateOfBirthComparison("dob", input_is_string=True),
cl.ExactMatch("city").configure(term_frequency_adjustments=True),
cl.EmailComparison("email"),
],
blocking_rules_to_generate_predictions=[
block_on("first_name"),
block_on("surname"),
],
)
linker = Linker(df, settings, db_api)
# Train the model
linker.training.estimate_u_using_random_sampling(max_pairs=1e6)
linker.training.estimate_parameters_using_expectation_maximisation(
block_on("first_name", "surname"), fix_u_probabilities=False
)
# Predict and cluster
results = linker.inference.predict(threshold_match_probability=0.9)
clusters = linker.clustering.cluster_pairwise_predictions_at_threshold(
results, threshold_match_probability=0.95
)
# Returns clusters -- but no golden records or merged outputWhen to Choose Splink
- You need probabilistic record linkage with statistical interpretability (m/u probabilities, Fellegi-Sunter model)
- You're doing research or analysis where the matching model itself is the deliverable
- You want multiple SQL backends (especially Spark for 100M+ records or Athena for serverless)
- Your team has data science expertise and is comfortable with the Fellegi-Sunter framework
- You need rich diagnostics like waterfall charts and match weight distributions
- You're working in government or public sector where Splink has proven adoption (UK ONS, NHS, MoD)
When to Choose Kanoniv
- You need golden records with survivorship strategies, not just clusters
- You need a real-time resolution API for application integration
- You want declarative configuration in YAML instead of Python code
- You need multi-tenant isolation for a SaaS product or multi-team deployment
- You need audit logs and compliance features (HIPAA, GDPR, immutable audit trail)
- You want to reconcile locally during development and deploy to cloud for production
- You need dbt and Snowflake integration for warehouse-native identity resolution
Key Differences Explained
Probabilistic Matching
Both Splink and Kanoniv implement Fellegi-Sunter probabilistic matching with EM parameter estimation. Splink has this as its core and only matching paradigm, with deep diagnostics (waterfall charts, match weight histograms).
Kanoniv supports both deterministic rules and Fellegi-Sunter probabilistic matching in the same spec. You can mix exact-match rules with EM-trained probabilistic scoring, and every match decision carries an explicit reason code. The zero-config AutoTune pipeline can bootstrap and optimize Fellegi-Sunter parameters automatically from your data.
Library vs Platform
Splink is a Python library. You import it, run it, and get results. It doesn't persist anything, doesn't serve an API, and doesn't manage entities over time.
Kanoniv is a platform. The Python SDK handles local development and validation, but Kanoniv Cloud persists the identity graph, serves real-time resolution, monitors run health, and manages multi-tenant access. The spec travels with you from laptop to production.
Post-Matching
Splink stops at clustering. Once you have clusters of matched records, you're on your own for merging, survivorship, golden record creation, and downstream integration.
Kanoniv includes survivorship as a first-class concept in the spec. You declare how to pick the "winning" value for each field (source priority, most recent, aggregation), and the engine produces golden records automatically.
