Full Pipeline: Zero to Resolved Entities

This tutorial walks through the flagship Kanoniv workflow: discover your data, bootstrap a matching spec, auto-tune thresholds, and reconcile identities - all in 4 CLI commands.

No YAML to write. No weights to hand-tune. No intermediate models to maintain.

Cloud feature

The discover/bootstrap/autotune pipeline uses the Kanoniv Cloud API. You need a Cloud account and API key. If you prefer to work entirely locally, see Writing Your First Spec to create a YAML spec manually and reconcile with the free SDK.

Prerequisites

pip install kanoniv[cloud]
kanoniv login

Requirements: Python 3.9+

Step 1: Ingest Your Data

Start with one or more data files. In this example we use CSV files from different systems:

kanoniv ingest ./data/ --entity-type person

This uploads all CSV/JSON files in the data/ directory. Works with any supported format.

You can also ingest individual files:

kanoniv ingest data/crm_contacts.csv --entity-type person
kanoniv ingest data/billing_customers.csv --entity-type person
kanoniv ingest data/support_tickets.csv --entity-type person

Step 2: Discover + Bootstrap

The autodetect --bootstrap command profiles every table, detects identity-relevant columns, classifies them by type (email, phone, name, etc.), and generates a complete identity spec:

kanoniv autodetect --bootstrap --entity-type person

What happens under the hood:

1. Samples up to 1,000 rows from each source
2. Detects column types by inspecting actual data values: email, phone, name, domain, identifier, date, sku, upc, mrn, and more
3. Computes uniqueness ratios, null rates, and pattern distributions
4. Identifies primary key candidates and entity-relevant fields
5. Infers the entity type from detected signals - one of person, company, product, transaction, or healthcare (defaults to person if ambiguous)
6. Generates source definitions, blocking keys, matching rules, scoring configuration, survivorship rules, and decision thresholds - all calibrated to the data distribution

What it generates:

• Source definitions with attribute mappings
• Blocking keys selected from the inferred entity type's templates (e.g. [email] for person, [sku] for product, [mrn] for healthcare)
• Matching rules with appropriate algorithms (exact for email/SKU, Jaro-Winkler for names, etc.)
• Fellegi-Sunter scoring configuration with weights calibrated per entity type
• Survivorship rules (most_complete for names, most_frequent for email)
• Decision thresholds calibrated to the data distribution

Step 3: AutoTune (Optional)

AutoTune refines the bootstrapped spec by running mini-reconciliations and optimizing thresholds:

kanoniv autotune --entity-type person

AutoTune:

1. Runs a sample reconciliation
2. Identifies uncertain pairs near the decision boundary
3. Tests threshold adjustments and normalizer combinations
4. Accepts changes that improve precision without sacrificing recall
5. Returns the optimized spec (or the original if no improvements found)

TIP

If the bootstrapped spec is already well-calibrated (common with clean data), AutoTune may find zero improvements. This is expected.

Step 4: Reconcile

Run the full identity resolution pipeline:

kanoniv reconcile --wait --entity-type person

Inspect the results

After reconciliation completes, check your stats:

kanoniv stats --entity-type person

Example output:

Entity type:  person
Records:      6,539
Entities:     2,896
Merge rate:   55.7%

Export golden records

Use the Cloud API or CLI to export resolved entities:

# Export entities as JSON
kanoniv export entities --entity-type person --output golden_records.json

# Or use the Python SDK with the Cloud client
from kanoniv import Client
client = Client(api_key="your-key")
export = client.entities.export(entity_type="person")

The Complete Pipeline

Here it is - the entire workflow in 4 commands:

pip install kanoniv[cloud]
kanoniv login

kanoniv ingest ./data/ --entity-type person             # Upload data
kanoniv autodetect --bootstrap --entity-type person      # Profile + generate spec
kanoniv autotune --entity-type person                    # Optimize thresholds
kanoniv reconcile --wait --entity-type person             # Run identity resolution
# Entities: 2896, Merge rate: 55.7%

What Comes Next

After your first reconciliation:

• Iterate on the spec: Edit the generated YAML to add custom rules or adjust thresholds
• Add feedback: Review uncertain pairs in the dashboard or via the API and label them as match/no_match
• Diff changes: Compare spec versions with kanoniv diff
• Go to production: Deploy to Kanoniv Cloud for real-time resolution, audit logs, and monitoring

CLI Reference

Command	Description
kanoniv ingest ./data/ --entity-type TYPE	Upload data files to Cloud
kanoniv autodetect --bootstrap --entity-type TYPE	Profile sources, detect identity columns, generate spec
kanoniv autotune --entity-type TYPE	Optimize spec thresholds and normalizers
kanoniv reconcile --wait --entity-type TYPE	Run full identity resolution

See CLI Reference for all available commands and options.

DiscoveryResult

Property	Type	Description
profiles	list[TableProfile]	Per-source profiling results

TableProfile

Property	Type	Description
source_name	str	Source name
row_count	int	Number of rows sampled
columns	list[ColumnProfile]	Column-level profiling

ColumnProfile

Property	Type	Description
name	str	Column name
identity_type	str | None	Detected type: email, phone, first_name, last_name, full_name, company_name, domain, tax_id, sku, upc, product_name, brand, category, transaction_id, amount, currency, mrn, npi, patient_name, date_of_birth, identifier, date, or None
uniqueness	float	Ratio of unique values (0.0 - 1.0)
null_rate	float	Ratio of null/empty values (0.0 - 1.0)