Scoring Strategies

After comparison rules produce individual scores, a scoring strategy combines them into a single composite score for each record pair. This score drives the match/review/reject decision.

Comparison Table

Strategy	Complexity	Training Data	Interpretability	Best For
weighted_sum	Low	None	High	Most use cases — start here
fellegi_sunter	Medium	Optional (EM)	High	Calibrated probabilities, regulatory
ml_ensemble	High	Required	Medium	Large labeled datasets
custom	Varies	None	Depends	Domain-specific formulas

Weighted Sum

The default scoring strategy. Simple, interpretable, and effective for most use cases.

Formula

composite_score = sum(score_i × weight_i) / sum(weight_i)

Each rule contributes its score multiplied by its weight. The result is normalized by the total weight, producing a score between 0.0 and 1.0.

Worked Example

Consider matching CRM and Stripe records for "John Smith":

Rules and weights:

Rule	Fields	Algorithm	Weight
email_exact	email	exact	1.0
name_fuzzy	name	jaro_winkler	0.6
phone_exact	phone	exact	0.85

Pair: CRM record vs Stripe record

Rule	CRM Value	Stripe Value	Score
email_exact	[email protected]	[email protected]	1.0
name_fuzzy	John Smith	Jon Smith	0.96
phone_exact	555-0100	555-0101	0.0

Calculation:

composite = (1.0 × 1.0 + 0.96 × 0.6 + 0.0 × 0.85) / (1.0 + 0.6 + 0.85)
          = (1.0 + 0.576 + 0.0) / 2.45
          = 1.576 / 2.45
          = 0.643

With thresholds match: 0.9, review: 0.6, this pair scores 0.643 — a review candidate. The matching email and name are strong signals, but the mismatched phone pulled the score down.

YAML Config

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.88
    weight: 0.6
  - name: phone_exact
    type: exact
    field: phone
    weight: 0.85

decision:
  scoring: weighted_sum
  thresholds:
    match: 0.9
    review: 0.6
    reject: 0.3

Tips

• High weight = high importance. Give weight 1.0 to your strongest identifier.
• Don't over-weight fuzzy rules. A name match alone shouldn't push a pair over the match threshold.
• Weights are relative. Only the ratios matter — weights of [2.0, 1.2, 1.7] produce the same result as [1.0, 0.6, 0.85].

Fellegi-Sunter

The gold standard for probabilistic record linkage, developed by Fellegi and Sunter in 1969. Instead of ad-hoc weights, it uses statistical probabilities to compute a log-likelihood ratio that a pair is a match.

Core Concepts

For each comparison field, two probabilities are estimated:

• m-probability: P(field agrees | records are a true match). "How often does this field match when two records really are the same person?"
• u-probability: P(field agrees | records are not a match). "How often does this field match by coincidence?"

A field is discriminating when m is high and u is low. Email with m=0.95, u=0.01 is highly discriminating — matches almost always agree, and coincidental agreement is rare. First name with m=0.90, u=0.15 is less discriminating — many non-matches share a first name.

Typical m/u Values

Field Type	m-probability	u-probability	Discriminating Power
Email	0.95	0.01	Very high
Phone	0.90	0.02	Very high
SSN/Tax ID	0.98	0.001	Extremely high
Last name	0.85	0.05	High
First name	0.90	0.15	Medium
City	0.80	0.10	Medium
State/Province	0.85	0.30	Low
Gender	0.98	0.50	Very low
Country	0.95	0.60	Very low

Log-Likelihood Ratios

For each field, Fellegi-Sunter computes a weight based on agreement or disagreement:

w_agree    = ln(m / u)           # positive: evidence FOR match
w_disagree = ln((1 - m) / (1 - u))  # negative: evidence AGAINST match

The composite score is the sum of per-field weights.

Worked Example

Setup: 4-field comparison — email, name, phone, city.

Field	m	u	w_agree = ln(m/u)	w_disagree = ln((1-m)/(1-u))
email	0.95	0.01	ln(95) = 4.55	ln(0.05/0.99) = -2.98
name	0.90	0.10	ln(9) = 2.20	ln(0.10/0.90) = -2.20
phone	0.90	0.02	ln(45) = 3.81	ln(0.10/0.98) = -2.28
city	0.80	0.10	ln(8) = 2.08	ln(0.20/0.90) = -1.50

Pair: Jon Smith (CRM) vs John Smith (Stripe)

Field	CRM	Stripe	Agrees?	Weight
email	[email protected]	[email protected]	Yes	+4.55
name	John Smith	Jon Smith	Yes (JW=0.96)	+2.20
phone	555-0100	555-0101	No	-2.28
city	San Francisco	San Francisco	Yes	+2.08
			Composite	+6.55

Comparison with weighted sum:

The same pair scored 0.643 with weighted sum (a "review"). With Fellegi-Sunter, the composite of +6.55 reflects that three strong agreements far outweigh one phone mismatch. The calibrated probabilities give more credit to the email match (extremely unlikely to agree by chance) than a simple weight of 1.0 would.

Posterior Probability

The raw composite score can be converted to a posterior probability — the probability that this pair is a true match given the observed agreements:

P(match | data) = sigmoid(composite) = 1 / (1 + e^(-composite))

For our example: P = 1 / (1 + e^(-6.55)) = 0.9986 — a 99.86% probability of being a true match.

This is useful for:

• Setting thresholds in probability terms ("match if P > 0.95")
• Regulatory environments where you need to justify match decisions
• Comparing match quality across different specs

EM Training

If you don't know the m/u probabilities for your data, Expectation-Maximization (EM) can estimate them automatically from unlabeled data.

How EM works:

1. Initialize: Start with reasonable guesses for m and u (e.g., m=0.9, u=0.1 for all fields)
2. E-step: For each pair, compute the probability of being a match given current m/u values
3. M-step: Re-estimate m/u values based on the match probabilities
4. Repeat: Until m/u values converge (typically 5-20 iterations)

EM finds the m/u values that best explain the observed agreement patterns in your data — without needing labeled training data.

Unbiased u-Probability Estimation

Standard EM estimates u-probabilities from blocked pairs — but blocked pairs always agree on the blocking field, which inflates u for that field. For example, if you block on email, every candidate pair shares an email, so EM thinks email agreement is common among non-matches.

The fix is a two-phase approach:

1. Phase 1: Estimate u-probabilities from random (unblocked) pairs via estimate_u: random_sampling. Random pairs represent the true non-match population, so the agreement rate gives an unbiased u estimate.
2. Phase 2: Run EM with u frozen (fix_u_probabilities=true) to only learn m-probabilities from blocked candidate pairs.

This is the same approach used by Splink and recommended in the record linkage literature.

YAML Config with EM + random u estimation (recommended):

decision:
  scoring:
    strategy: fellegi_sunter
    training:
      method: em
      estimate_u: random_sampling  # Phase 1: unbiased u from random pairs
      max_random_pairs: 1000000    # Sample up to 1M random pairs for u
      max_iterations: 25           # Phase 2: EM iterations for m
      convergence_threshold: 0.001
    fields:
      - name: email
        comparator: exact
        normalizer: email          # Gmail dots, plus-addressing
        m_probability: 0.95        # initial estimate (EM will refine)
        u_probability: 0.01
      - name: first_name
        comparator: jaro_winkler
        normalizer: nickname       # Bob → robert before compare
        m_probability: 0.90
        u_probability: 0.10
      - name: phone
        comparator: exact
        normalizer: phone          # E.164 normalization
        m_probability: 0.90
        u_probability: 0.02
      - name: city
        comparator: exact
        m_probability: 0.80
        u_probability: 0.10
    thresholds:
      match: 8.0          # log-likelihood composite
      possible: 4.0       # needs human review
      non_match: -5.0
  thresholds:
    match: 0.95
    review: 0.70

YAML Config without EM (fixed probabilities):

decision:
  scoring:
    strategy: fellegi_sunter
    fields:
      - name: email
        comparator: exact
        normalizer: email
        m_probability: 0.95
        u_probability: 0.01
      - name: name
        comparator: jaro_winkler
        normalizer: name
        m_probability: 0.90
        u_probability: 0.10
      - name: phone
        comparator: exact
        normalizer: phone
        m_probability: 0.90
        u_probability: 0.02
    thresholds:
      match: 6.0        # log-likelihood composite
      possible: 2.0
      non_match: -3.0
  thresholds:
    match: 0.9
    review: 0.7

Each field supports an optional normalizer that runs before comparison — both during EM training and scoring. See Matching Rules > Pre-Match Normalization for the full list.

Sensitivity Analysis

Understanding which fields drive your match decisions:

Field elasticity — How much does removing a field change the composite score?

Field Removed	Avg. Composite Change	Impact
email	-4.55 (agree) / +2.98 (disagree)	Critical — removing email drops most matches below threshold
phone	-3.81 / +2.28	High — phone is the second strongest signal
name	-2.20 / +2.20	Medium — contributes but doesn't dominate
city	-2.08 / +1.50	Low — confirmatory but not decisive

Threshold sensitivity — Small changes in the match threshold can significantly change match counts:

Match Threshold	Pairs Matched	Precision	Recall
8.0	2,340	99.8%	82%
6.0	3,180	99.2%	91%
4.0	4,520	97.1%	96%
2.0	6,890	91.3%	99%

The right threshold depends on your tolerance for false positives vs. false negatives. Start with a high threshold and lower it until recall is acceptable.

ML Ensemble

Use a logistic regression model with trained coefficients to score pairs. This requires labeled training data but can capture non-linear relationships between fields.

How It Works

score = sigmoid(bias + sum(coefficient_i × rule_score_i))
      = 1 / (1 + e^(-(bias + sum(coefficient_i × rule_score_i))))

The coefficients are trained on labeled match/non-match pairs. The output is a probability between 0 and 1.

YAML Config

decision:
  scoring:
    strategy: ml_ensemble
    bias: -2.5
    coefficients:
      email_exact: 4.2
      name_fuzzy: 1.8
      phone_exact: 3.1
      address_fuzzy: 0.9
  thresholds:
    match: 0.90
    review: 0.60

When to Use

• You have 1,000+ labeled match/non-match pairs
• Simple weighted sum isn't capturing the patterns in your data
• You've tried Fellegi-Sunter and want to compare performance

When Not to Use

• You don't have labeled training data (use weighted sum or Fellegi-Sunter with EM instead)
• You need to explain individual match decisions to regulators (coefficients are less interpretable than m/u probabilities)

Custom Expression

Define your own scoring formula using arithmetic operators and rule scores as variables.

How It Works

Write an expression using rule names as variables. Available operators: +, -, *, /, min(), max().

YAML Config

rules:
  - name: email_exact
    type: exact
    field: email
    weight: 1.0
  - name: name_fuzzy
    type: similarity
    field: name
    algorithm: jaro_winkler
    threshold: 0.85
    weight: 0.6
  - name: phone_exact
    type: exact
    field: phone
    weight: 0.85

decision:
  scoring:
    strategy: custom
    expression: "max(email_exact, (name_fuzzy * 0.7 + phone_exact * 0.3))"
  thresholds:
    match: 0.9
    review: 0.6

This scores a pair as the maximum of:

1. The email score alone (an exact email match = automatic 1.0)
2. A weighted combination of name and phone

When to Use

• You need domain-specific logic that weighted sum can't express
• You want to implement "if email matches, always match regardless of other fields"
• You're migrating from a legacy system with a known scoring formula

Choosing a Strategy

                    Do you have labeled training data?
                           /              \
                         Yes               No
                         /                   \
                   ML Ensemble          Do you need calibrated
                                        probabilities?
                                         /          \
                                       Yes           No
                                       /               \
                              Fellegi-Sunter       Weighted Sum
                              (with or without EM)   (default)

Start with weighted_sum. It's the simplest, requires no training data, and works well for 80% of use cases. Switch to Fellegi-Sunter when you need calibrated probabilities or regulatory justification. Use ML Ensemble only when you have labeled data and weighted sum isn't performing well enough.

Next Steps

• String Comparators — The algorithms that produce individual rule scores
• Blocking Strategies — Reducing pairs before scoring runs
• Survivorship — What happens after pairs are scored and classified