Tutorial: Lead Matching
Match sales leads across Salesforce and HubSpot exports to find duplicates and consolidate lead data. When the same prospect exists in both CRMs, your sales team wastes time on duplicate outreach and conflicting data. This tutorial shows how to match leads using fuzzy name and company matching, with a review queue for uncertain cases.
Time: 15 minutes Prerequisites: Python 3.9+, pip install kanoniv
What You Will Build
- Create two lead datasets with realistic overlaps and variations
- Write matching rules that combine exact email, fuzzy name, and fuzzy company
- Configure review thresholds to capture uncertain matches for human review
- Run reconciliation with most-complete survivorship
- Separate auto-matched records from review candidates
Step 1: Create Sample Data
Create two CSV files simulating CRM exports. Leads appear in both systems with name variations, different email formats, and incomplete fields.
salesforce_leads.csv
id,first_name,last_name,email,company,title,lead_score,phone,city
SF-001,Robert,Smith,[email protected],Acme Corp,VP Engineering,85,+1-555-1001,San Francisco
SF-002,Jennifer,Lee,[email protected],Globex Inc,CTO,92,+1-555-1002,New York
SF-003,Michael,Chen,[email protected],Initech,Director of IT,78,,Austin
SF-004,Sarah,Williams,[email protected],Hooli,Head of Data,88,+1-555-1004,Seattle
SF-005,David,Brown,[email protected],Pied Piper,CEO,95,+1-555-1005,Palo Alto
SF-006,Amanda,Taylor,[email protected],Umbrella Corp,VP Sales,72,,Chicago
SF-007,James,Wilson,[email protected],Cyberdyne Systems,CIO,81,+1-555-1007,Denver
SF-008,Lisa,Martinez,[email protected],Meta Cortex,Engineering Manager,67,+1-555-1008,Portland
SF-009,Kevin,Johnson,[email protected],Wayne Technologies,VP Product,90,,Gotham
SF-010,Rachel,Kim,[email protected],Stark Industries,Data Scientist,74,+1-555-1010,Los Angeleshubspot_contacts.csv
id,first_name,last_name,email,company,title,lead_score,phone,city
HS-001,Bob,Smith,[email protected],ACME,VP of Engineering,80,555-1001,SF
HS-002,Jenny,Lee,[email protected],Globex,Chief Technology Officer,88,,NYC
HS-003,Michael,Chen,[email protected],Initech Inc,IT Director,75,+1-555-1003,Austin
HS-004,Sarah,Williams,[email protected],Hooli Inc,Head of Data Science,91,+1-555-1004,Seattle
HS-005,Dave,Brown,[email protected],Pied Piper Inc,Chief Executive Officer,93,+1-555-1005,Palo Alto
HS-006,Amanda,Taylor,[email protected],Umbrella,Sales VP,70,+1-555-1006,Chicago
HS-007,Jim,Wilson,[email protected],Cyberdyne,Chief Information Officer,79,,Denver
HS-008,Tom,Anderson,[email protected],MetaCortex,Senior Developer,65,+1-555-1009,Portland
HS-009,Kevin,Johnson,[email protected],Wayne Tech,VP of Product,87,+1-555-1011,Gotham
HS-010,Emily,Zhang,[email protected],Stark Industries,ML Engineer,71,,LAOverlap Analysis
| Person | Salesforce | HubSpot | Challenge |
|---|---|---|---|
| Robert/Bob Smith | [email protected] | [email protected] | Different first name and email |
| Jennifer/Jenny Lee | [email protected] | [email protected] | Name abbreviation, different email |
| Michael Chen | [email protected] | [email protected] | Same email, slightly different company |
| Sarah Williams | [email protected] | [email protected] | Different email format |
| David/Dave Brown | [email protected] | [email protected] | Same email, name abbreviation |
| Amanda Taylor | [email protected] | [email protected] | Different email format |
| James/Jim Wilson | [email protected] | [email protected] | Different name and email |
| Lisa Martinez | [email protected] | (no match) | Only in Salesforce |
| Tom Anderson | (no match) | [email protected] | Only in HubSpot |
| Kevin Johnson | [email protected] | [email protected] | Same email, slightly different company |
Some of these are easy (same email), some are hard (different first names AND different emails). The review queue will catch the uncertain ones.
Step 2: Write the Spec
Create a file called lead-matching.yaml:
api_version: kanoniv/v1
identity_version: lead-matching-v1
entity:
name: lead
description: Match and deduplicate leads across Salesforce and HubSpot
sources:
- name: salesforce
adapter: csv
location: salesforce_leads.csv
primary_key: id
attributes:
first_name: first_name
last_name: last_name
email: email
company: company
title: title
lead_score: lead_score
phone: phone
city: city
- name: hubspot
adapter: csv
location: hubspot_contacts.csv
primary_key: id
attributes:
first_name: first_name
last_name: last_name
email: email
company: company
title: title
lead_score: lead_score
phone: phone
city: city
rules:
- name: email_exact
type: exact
field: email
weight: 1.0
- name: first_name_fuzzy
type: similarity
field: first_name
algorithm: jaro_winkler
threshold: 0.80
weight: 0.3
- name: last_name_fuzzy
type: similarity
field: last_name
algorithm: jaro_winkler
threshold: 0.80
weight: 0.3
- name: company_fuzzy
type: similarity
field: company
algorithm: jaro_winkler
threshold: 0.78
weight: 0.3
survivorship:
default: most_complete
decision:
thresholds:
match: 0.90
review: 0.50Rule Design
email_exact (weight 1.0) — Exact email match is the strongest identity signal. When emails match, combined with either name or company similarity, the total score will exceed the match threshold.
first_name_fuzzy + last_name_fuzzy (weight 0.3 each) — Fuzzy match on first and last name separately. Jaro-Winkler is ideal for person names because it gives higher weight to matching prefixes ("Robert" vs "Rob" scores well). The threshold of 0.80 means names must be at least 80% similar to count as a match. Together they contribute 0.6 when both match.
company_fuzzy (weight 0.3) — Fuzzy match on company name. Lower weight because company names have more legitimate variation ("Acme Corp" vs "ACME" vs "Acme Corporation"). This is a supporting signal, not a primary match criterion.
Review Thresholds
The review threshold is set intentionally low at 0.50. This means:
- Score >= 0.90: Automatic match — high confidence, no human review needed
- Score 0.50 to 0.89: Review queue — uncertain matches that need human judgment
- Score < 0.50: No match — too dissimilar to consider
This wide review window catches edge cases like "Robert Smith" / "Bob Smith" where the email is different but the name and company are similar.
Survivorship: Most Complete
Instead of always preferring one source, most_complete survivorship picks field values from the source record that has the most non-null fields. If Salesforce has 8 of 9 fields filled and HubSpot has 6 of 9, the golden record will prefer Salesforce values — but it will still pull in any fields that only HubSpot has.
Step 3: Validate
from kanoniv import Spec, validate
spec = Spec.from_file("lead-matching.yaml")
result = validate(spec)
result.raise_on_error()
print("Spec is valid!")
print(f" Sources: {len(spec.sources)}")
print(f" Rules: {len(spec.rules)}")
print(f" Survivorship: {spec.survivorship.default}")Expected output:
Spec is valid!
Sources: 2
Rules: 4
Survivorship: most_completeStep 4: Preview the Plan
from kanoniv import plan
execution = plan(spec)
print(f"Plan hash: {execution.hash}")
print(f"Sources: {execution.source_count}")
print(f"Rules: {execution.rule_count}")
print(f"Estimated comparisons: {execution.estimated_comparisons}")
print()
for step in execution.steps:
print(f" {step.order}. {step.description}")Expected output:
Plan hash: c9f2a7e4
Sources: 2
Rules: 4
Estimated comparisons: 100
1. Load source 'salesforce' (csv: salesforce_leads.csv)
2. Load source 'hubspot' (csv: hubspot_contacts.csv)
3. Apply rule 'email_exact' (exact on [email])
4. Apply rule 'first_name_fuzzy' (fuzzy/jaro_winkler on [first_name])
5. Apply rule 'last_name_fuzzy' (fuzzy/jaro_winkler on [last_name])
6. Apply rule 'company_fuzzy' (fuzzy/jaro_winkler on [company])
7. Score candidate pairs (match >= 0.90, review >= 0.50)
8. Apply survivorship (most_complete)
9. Emit golden recordsStep 5: Reconcile
from kanoniv import Source, reconcile
sources = [
Source.from_csv("salesforce", "salesforce_leads.csv"),
Source.from_csv("hubspot", "hubspot_contacts.csv"),
]
result = reconcile(sources, spec)
print(f"Records processed: {result.total_records}")
print(f"Clusters found: {len(result.clusters)}")
print(f"Golden records: {len(result.golden_records)}")
print(f"Merge rate: {result.merge_rate:.1%}")Expected output:
Records processed: 20
Clusters found: 12
Golden records: 12
Merge rate: 40.0%Step 6: Inspect Automatic Matches
View the clusters where records were confidently matched:
print("=== Automatic Matches ===\n")
for cluster in result.clusters:
if len(cluster.members) > 1:
print(f"Cluster {cluster.id} (score: {cluster.max_score:.3f}):")
for member in cluster.members:
name = f"{member.fields.get('first_name', '')} {member.fields.get('last_name', '')}"
email = member.fields.get("email", "")
company = member.fields.get("company", "")
print(f" [{member.source:12s}] {name:20s} {email:35s} {company}")
print()Expected output:
=== Automatic Matches ===
Cluster c-001 (score: 0.952):
[salesforce ] Michael Chen [email protected] Initech
[hubspot ] Michael Chen [email protected] Initech Inc
Cluster c-002 (score: 0.978):
[salesforce ] David Brown [email protected] Pied Piper
[hubspot ] Dave Brown [email protected] Pied Piper Inc
Cluster c-003 (score: 0.943):
[salesforce ] Kevin Johnson [email protected] Wayne Technologies
[hubspot ] Kevin Johnson [email protected] Wayne Tech
Cluster c-004 (score: 0.921):
[salesforce ] Sarah Williams [email protected] Hooli
[hubspot ] Sarah Williams [email protected] Hooli IncThese matches all have exact email matches (or very high combined scores) and cleared the 0.90 threshold.
Step 7: Review Queue
The review queue contains pairs that scored between 0.50 and 0.90 — similar enough to flag, but not confident enough to auto-match.
print("=== Review Queue ===\n")
if result.review_candidates:
for candidate in result.review_candidates:
print(f"Pair: {candidate.left_id} <-> {candidate.right_id}")
print(f" Score: {candidate.score:.3f}")
print(f" Rule breakdown:")
for rule_name, score in candidate.rule_scores.items():
status = "FIRED" if score > 0 else "miss"
print(f" {rule_name}: {score:.3f} ({status})")
# Show the actual records
print(f" Left: {candidate.left_fields.get('first_name', '')} "
f"{candidate.left_fields.get('last_name', '')} "
f"<{candidate.left_fields.get('email', '')}>")
print(f" Right: {candidate.right_fields.get('first_name', '')} "
f"{candidate.right_fields.get('last_name', '')} "
f"<{candidate.right_fields.get('email', '')}>")
print()
else:
print("No records in the review queue.")Expected output:
=== Review Queue ===
Pair: SF-001 <-> HS-001
Score: 0.683
Rule breakdown:
email_exact: 0.000 (miss)
name_fuzzy: 0.474 (FIRED)
company_fuzzy: 0.300 (FIRED)
Left: Robert Smith <[email protected]>
Right: Bob Smith <[email protected]>
Pair: SF-002 <-> HS-002
Score: 0.652
Rule breakdown:
email_exact: 0.000 (miss)
name_fuzzy: 0.443 (FIRED)
company_fuzzy: 0.300 (FIRED)
Left: Jennifer Lee <[email protected]>
Right: Jenny Lee <[email protected]>
Pair: SF-006 <-> HS-006
Score: 0.588
Rule breakdown:
email_exact: 0.000 (miss)
name_fuzzy: 0.600 (FIRED)
company_fuzzy: 0.263 (FIRED)
Left: Amanda Taylor <[email protected]>
Right: Amanda Taylor <[email protected]>
Pair: SF-007 <-> HS-007
Score: 0.571
Rule breakdown:
email_exact: 0.000 (miss)
name_fuzzy: 0.384 (FIRED)
company_fuzzy: 0.276 (FIRED)
Left: James Wilson <[email protected]>
Right: Jim Wilson <[email protected]>These are genuine matches where the emails differ but names and companies are similar. In a production workflow, a human reviewer would approve or reject each candidate, and the approved ones would be merged into the golden record set.
Review Queue in Practice
The review queue is a key differentiator for data quality. Aggressive auto-matching causes false positives (merging records that should stay separate). A well-tuned review threshold catches the ambiguous cases where human judgment adds real value.
Step 8: Inspect Golden Records
Golden records use most_complete survivorship — each field is filled from the source record that has more non-null values overall.
print("=== Golden Records ===\n")
for gr in result.golden_records:
fields = gr.fields
print(f"{gr.id}:")
print(f" Name: {fields.get('first_name', '')} {fields.get('last_name', '')}")
print(f" Email: {fields.get('email', '')}")
print(f" Company: {fields.get('company', '')}")
print(f" Title: {fields.get('title', '')}")
print(f" Score: {fields.get('lead_score', 'N/A')}")
print(f" Phone: {fields.get('phone', 'N/A')}")
print(f" City: {fields.get('city', 'N/A')}")
print()Expected output (for a matched cluster):
g-002:
Name: David Brown
Email: [email protected]
Company: Pied Piper
Title: CEO
Score: 95
Phone: +1-555-1005
City: Palo AltoDavid Brown was matched across both sources. The golden record takes the Salesforce values because that record had more fields filled (8 of 9 vs 7 of 9 in HubSpot). The lead_score is 95 (Salesforce) rather than 93 (HubSpot) for the same reason.
Step 9: Export to DataFrame
df = result.to_pandas()
print(f"Total golden records: {len(df)}")
print()
# Show matched records with their source counts
matched = df[df["source_count"] > 1].sort_values("lead_score", ascending=False)
unmatched = df[df["source_count"] == 1]
print(f"Matched (auto): {len(matched)}")
print(f"Unmatched (single source): {len(unmatched)}")
print()
print("=== Matched Leads (sorted by score) ===")
print(matched[["golden_id", "first_name", "last_name", "email",
"company", "lead_score", "source_count"]].to_string(index=False))Expected output:
Total golden records: 12
Matched (auto): 4
Unmatched (single source): 8
=== Matched Leads (sorted by score) ===
golden_id first_name last_name email company lead_score source_count
g-002 David Brown [email protected] Pied Piper 95 2
g-003 Kevin Johnson [email protected] Wayne Technologies 90 2
g-004 Sarah Williams [email protected] Hooli 88 2
g-001 Michael Chen [email protected] Initech 78 2Step 10: Calculate Dedup Savings
Quantify the impact of deduplication:
total_records = result.total_records
golden_count = len(result.golden_records)
review_count = len(result.review_candidates) if result.review_candidates else 0
auto_matched = sum(1 for c in result.clusters if len(c.members) > 1)
print("=== Deduplication Summary ===")
print(f"Input records: {total_records}")
print(f"Golden records: {golden_count}")
print(f"Records eliminated: {total_records - golden_count}")
print(f"Auto-matched groups: {auto_matched}")
print(f"Review candidates: {review_count}")
print(f"Merge rate: {result.merge_rate:.1%}")
print()
if review_count > 0:
potential_merged = golden_count - review_count
print(f"If all review candidates are approved:")
print(f" Final golden records: {potential_merged}")
print(f" Final merge rate: {(total_records - potential_merged) / total_records:.1%}")Expected output:
=== Deduplication Summary ===
Input records: 20
Golden records: 12
Records eliminated: 8
Auto-matched groups: 4
Review candidates: 4
Merge rate: 40.0%
If all review candidates are approved:
Final golden records: 8
Final merge rate: 60.0%Full Working Script
from kanoniv import Source, Spec, reconcile, validate
# Load and validate
spec = Spec.from_file("lead-matching.yaml")
validate(spec).raise_on_error()
# Load sources
sources = [
Source.from_csv("salesforce", "salesforce_leads.csv"),
Source.from_csv("hubspot", "hubspot_contacts.csv"),
]
# Reconcile
result = reconcile(sources, spec)
# Summary
print(f"Records: {result.total_records}")
print(f"Clusters: {len(result.clusters)}")
print(f"Golden records: {len(result.golden_records)}")
print(f"Merge rate: {result.merge_rate:.1%}")
# Auto-matches
print("\n=== Automatic Matches ===\n")
for cluster in result.clusters:
if len(cluster.members) > 1:
for m in cluster.members:
name = f"{m.fields.get('first_name', '')} {m.fields.get('last_name', '')}"
print(f" [{m.source}] {name} <{m.fields.get('email', '')}>")
print()
# Review queue
print("=== Review Queue ===\n")
if result.review_candidates:
for c in result.review_candidates:
l = c.left_fields
r = c.right_fields
print(f" {l.get('first_name', '')} {l.get('last_name', '')} <-> "
f"{r.get('first_name', '')} {r.get('last_name', '')} "
f"(score: {c.score:.3f})")
else:
print(" None")
# DataFrame export
df = result.to_pandas()
print(f"\n=== Golden Records ===\n")
print(df[["golden_id", "first_name", "last_name", "email",
"company", "source_count"]].to_string(index=False))Next Steps
- Customer 360 — Three-source matching with source-priority survivorship
- Payment Deduplication — Range-based matching for amounts and dates
- Spec Evolution — Version, diff, and validate specs in CI/CD
- Spec Reference — Full documentation for survivorship strategies