How To Measure Identity Resolution Accuracy

Identity resolution accuracy should be measured across three dimensions: precision, recall, and cluster coherence. Precision asks whether the links the system made are correct and detects false merges. Recall asks whether the links that should exist were found and detects false splits. Cluster coherence asks whether each resulting customer profile actually represents one person.

Match rate is not accuracy. Match rate measures coverage: the share of records, events, or sessions that were linked to a known identity. It does not tell you whether those links are correct or whether valid links were missed.

A CDP can report a 91 percent match rate while still merging different household members into one profile and splitting loyal customers across several profiles.

That is why match rate alone should never be used as an identity resolution validation gate.

There is another complication: “match rate” does not have one universal unit. One team may report the percentage of input records matched. Another may report the percentage of sessions associated with a known user. Another may report the percentage of events resolved to a profile.

Those are different metrics.

Every identity report should therefore state exactly what is being counted before anyone compares the number with another system, vendor, or previous period.

Match Rate Is Not Accuracy: The Five Numbers That Describe Identity Quality

Identity resolution quality cannot be reduced to one percentage. Five measurements answer different questions.

Match Rate Measures Coverage

Match rate answers: How much of the data was linked to a known identity?

The basic calculation is:

Matched units ÷ total units

But the unit must always be stated.

That could mean:

• Records in an input file
• Customer events
• Website sessions

A low match rate can indicate missing identifiers, normalization problems, incomplete source coverage, or failed stitching.

A high match rate proves only that the system linked a large share of the data. It does not prove that those links are correct.

Match rate is the only one of the five core measurements that does not require ground truth.

Precision Measures False Merges

Precision answers: Of the identity links the system created, how many are correct?

In a formal pairwise analysis, precision is calculated as correct predicted links divided by all predicted links.

Operationally, enterprise teams can review sampled merged profiles and calculate the share that actually represent one person.

The corresponding false merge rate is:

1 minus precision

Low precision means different people are being combined. That can create wrong recommendations, incorrect loyalty tiers, contaminated customer histories, inaccurate predictive scores, and privacy exposure.

One terminology point matters here.

In a graph containing millions of customer records, nearly every possible pair of records is a non-match. The textbook false positive rate, calculated against all non-matching pairs, can therefore appear vanishingly small even when many customer profiles are wrong.

For practical CDP measurement, “false merge rate” should mean the share of reviewed merges or profiles that were incorrect.

Recall Measures False Splits

Recall answers: Of the identity links that should exist, how many did the system find?

Formally, pairwise recall is correct predicted links divided by all true links.

Operationally, teams can begin with customers whose identity is independently verified across multiple systems and determine whether all of their records resolve to one canonical profile.

The false split rate is:

1 minus recall

Low recall means customers are fragmented.

That can produce duplicate messages, repeat recommendations, incomplete lifetime value, failed suppression, and disconnected cross-channel journeys.

Recall is harder to measure than precision because the ground truth must include relationships the identity system failed to create.

Cluster Coherence Detects Over-Merged Profiles

Precision and recall are not enough.

Pairwise metrics can look healthy even when a small number of very large customer clusters are badly wrong. That is particularly important in CDPs because household over-merges and identity explosion often appear in the largest clusters.

Cluster coherence asks: Does each canonical profile represent exactly one person?

Practical checks include:

• Reviewing the share of sampled profiles containing exactly one person
• Examining cluster size distribution
• Reviewing every exceptionally large cluster
• Using cluster or B-cubed precision and recall where more formal measurement is required

A long tail of oversized clusters can reveal a serious graph problem even when average pairwise metrics look acceptable.

Confidence Calibration Tests Probabilistic Matching

For probabilistic identity, teams also need to ask:

When the system says it is 90 percent confident, is it actually correct about 90 percent of the time?

Group probabilistic matches into confidence bands, sample records in each band, and calculate the percentage that reviewers confirm as correct.

If a 90 to 95 confidence band is correct only 80 percent of the time, the score is poorly calibrated for that use case.

Confidence thresholds should therefore be based on measured performance against your own data, not solely on the number shown by the vendor.

The Six-Step Identity Resolution Accuracy Measurement Procedure

A reliable accuracy test requires ground truth, a defensible sample, and clear reporting of uncertainty.

Step 1: Define The Unit And Match Rate

Start by choosing exactly what match rate means in your analysis.

For example:

• Percentage of input customer records matched
• Percentage of events resolved to known profiles
• Percentage of sessions associated with known users

Record the source systems, reporting period, and unit every time the metric is reported.

Never compare two match rates that use different units.

Step 2: Build A Ground-Truth Sample

Full ground truth rarely exists for an enterprise identity graph. Creating it for every customer would also be prohibitively expensive.

Instead, build a labeled sample.

Three practical sources of ground truth are especially useful.

Use Verified Anchor Customers

Start with customers whose identities can be independently confirmed across channels.

Examples include repeat buyers with verified records in CRM, ecommerce, loyalty, and POS systems.

These customers are particularly useful for measuring recall because the team already knows which records should resolve together.

Hide Strong Deterministic Links To Test Probabilistic Matching

Where a deterministic identifier is genuinely unique to one person, the team can temporarily withhold that link and test whether the probabilistic identity layer reconstructs the relationship.

This method is only valid when the deterministic identifier is trustworthy.

A shared household email, reused device ID, or generic company address should not be treated as ground truth simply because it is deterministic.

Use Clerical Review

Trained reviewers can inspect sampled profiles or candidate record pairs against the underlying source systems and decide whether the records belong to the same person.

Review requires written rules for ambiguous cases such as name changes, shared household emails, and shared devices.

Because reviewers may access personal customer data, access should be restricted and approved through the appropriate privacy process before labeling begins.

Design The Sample Around Risk

For precision and cluster coherence, sample by profile.

Stratify the sample where appropriate by:

• Cluster size
• Deterministic versus probabilistic matching
• Source-system pair
• Confidence-score band

Very large clusters should be reviewed in full.

For recall, start with verified multi-channel customers and determine whether all of their records resolve to one profile. Add likely unmerged candidates with similar names, addresses, devices, or other plausible links rather than relying only on random record pairs.

Step 3: Label The Sample

Reviewers then classify the sampled records or profiles using source-system evidence.

At least a subset should be reviewed by a second person so the team can measure agreement and identify ambiguous labeling rules. A practical Stable Kernel recommendation is to double-label approximately 10 to 20 percent of the sample.

Ground truth itself can contain errors, so disagreements should be documented rather than silently resolved.

Step 4: Calculate Precision, Recall, Coherence, And Calibration

Once the sample is labeled:

• Calculate precision and false merge rate from reviewed merged profiles.
• Calculate recall and person-level split rate from verified anchor customers.
• Calculate cluster coherence from the profile sample.
• Review the largest clusters separately.
• For probabilistic identity, calculate observed accuracy within each confidence band.

Open-source evaluation tooling can help teams calculate pairwise, cluster, and B-cubed metrics when a more formal analysis is needed.

Step 5: Report The Margin Of Error

A percentage calculated from a sample is an estimate, not an exact population value.

Every sampled result should therefore include:

• Sample size
• Observed rate
• Approximate confidence interval
• Sampling method

This is what allows a program manager to distinguish a meaningful failure from ordinary sampling variation.

For formal reporting, an analyst should confirm the sampling design and interval method, particularly when stratified or clustered sampling is used.

Step 6: Compare Results Against Use-Case Tolerances

The final step is not to compare your identity accuracy against an arbitrary industry benchmark.

Compare it with the tolerance defined for the business use case.

Record the result as:

• Pass
• Conditional pass with remediation
• Block

Store the results so identity quality can be trended over time.

How Many Profiles Should You Review?

There is no single correct sample size.

For simple random samples from large populations, standard statistical calculations provide useful planning estimates.

For A Rough Estimate

If you have no idea what the underlying error rate is and want approximately a ±5 percentage-point margin at 95 percent confidence, roughly 385 profiles is a common planning figure.

For approximately ±3 points, the sample increases to around 1,070 profiles.

For A Rate Near 2 Percent

If you expect the actual error rate to be near 2 percent, estimating that rate within approximately ±1 percentage point requires around 750 profiles under simple random sampling assumptions.

Tightening the margin to roughly ±0.5 point can require around 3,000 profiles.

When You Observe Zero Errors

The “rule of three” provides a useful approximation.

If zero errors are found in 200 randomly sampled profiles, the approximate upper 95 percent bound is about:

3 ÷ 200 = 1.5 percent

If zero errors are found across 500 profiles, the approximate upper bound falls to about 0.6 percent.

This highlights an important limitation in smaller audits.

A sample of 100 to 200 profiles can catch gross failures. It cannot estimate a rate near 2 percent with a particularly narrow margin.

Stratified sampling changes the arithmetic, so sample design should be reviewed by an analyst before formal publication or executive reporting.

An Illustrative Identity Accuracy Example

The following numbers are illustrative only. They are not benchmarks or Stable Kernel client results.

Assume an identity system processes 100,000 customer events.

Match Rate

Suppose 91,000 of the 100,000 events resolve to a profile.

The match rate is:

91 percent of events

Coverage looks strong.

But nothing about that number tells us whether the matches are correct.

False Merge Rate

Now suppose the team reviews 400 canonical profiles and finds that 12 contain records belonging to more than one person.

The illustrative false merge rate is:

12 ÷ 400 = 3 percent

The approximate margin is roughly ±1.7 percentage points using a normal approximation.

If the applicable validation gate were 2 percent, this illustrative result would not pass simply because the point estimate is close.

False Split Rate

Next, suppose the team reviews 300 verified multi-channel customers and discovers that 42 are represented by two or more canonical profiles.

The illustrative person-level split rate is:

42 ÷ 300 = 14 percent

That means roughly one in seven of the verified customers in this illustrative sample is fragmented.

Largest-Cluster Review

Finally, suppose the team reviews every cluster with more than 50 linked identifiers. There are nine such clusters, and six contain more than one actual person.

That indicates a potentially serious over-merge or identity explosion problem.

The point of the example is simple: the 91 percent match rate would have looked healthy while precision, recall, and cluster analysis revealed material defects.

How Accurate Is Accurate Enough?

There is no universal identity resolution accuracy benchmark.

Published vendor ranges vary substantially, and many do not explain the measurement method, unit, sampling process, or ground truth behind the percentage.

Stable Kernel recommends setting tolerances according to the consequence of an incorrect identity decision.

Regulated, Account-Specific, And AI Agent Actions

False merges are the primary concern.

Prioritize precision and cluster coherence. Use deterministic identity for the action path and treat a confirmed false merge as a blocking defect.

Large clusters should be reviewed directly rather than relying only on a sampled average.

Consent And Suppression

Both false merges and false splits matter.

A false split can leave an opt-out attached to only one fragment of the customer. A false merge can suppress the wrong person.

Use precision and recall together, and validate actual suppression behavior with designated test profiles.

Loyalty And Lifecycle Messaging

False merges can expose the wrong tier, balance, status, or benefit.

Stable Kernel's existing pre-production guidance uses an under-2-percent false merge gate as a baseline for general identity validation, but the tolerance should become stricter when customer-specific balances, points, or other sensitive status information is displayed.

Personalization To Known Customers

Prioritize precision first, then recall.

Probabilistic matches should be allowed into customer-facing activation only above confidence bands that have been calibrated on labeled data.

The threshold should reflect the consequence of a wrong personalization decision rather than a vendor's generic accuracy claim.

Audience Expansion And Aggregate Analytics

Recall and coverage become relatively more important.

An occasional missed match may reduce reach, while an overly restrictive identity strategy may make the audience less useful.

The acceptable tolerance should be documented based on the business cost of inaccurate reach or distorted aggregates.

How To Calibrate Probabilistic Confidence Scores

Do not assume a confidence score is calibrated simply because the platform outputs one.

Group matches into score bands such as 70 to 79, 80 to 89, and 90 to 100. Review a sample from each band and calculate the share that is actually correct.

Then select the lowest band whose measured precision meets the use case's required tolerance.

A rough review of about 100 matches per band can provide an initial read, but more observations are appropriate when the threshold is close to a decision boundary.

Keep Measuring Identity Accuracy After Go-Live

Identity accuracy is not a one-time implementation gate.

Identifiers change. Customers change devices and email addresses. New sources enter the graph. Matching rules change. Vendors update algorithms.

The measurement cadence should therefore combine inexpensive automated proxies with periodic ground-truth audits.

Weekly Automated Monitoring

Monitor indicators that can reveal drift:

• Match rate by source, with the unit explicitly stated
• Anonymous-to-known stitch rate
• Duplicate canonical profile rate
• Largest cluster size
• Merge and unmerge volume

These indicators raise alarms. They do not prove accuracy.

Monthly Cluster Monitoring

Review cluster-size distribution and probabilistic confidence-band distribution.

The purpose is to detect gradual graph growth, identity explosion, or shifts in matching behavior before they become visible in customer-facing systems.

Quarterly Precision And Coherence Audit

Repeat the labeled sampling process for precision, false merge rate, cluster coherence, and confidence calibration.

Reuse the same labeling rules where possible so results remain comparable over time.

Twice-Yearly Recall Audit

Recall is more expensive to measure because teams must establish what links should have been created.

Run a person-level split audit against the verified anchor population once or twice per year and after material identity changes.

Event-Driven Revalidation

Re-run the relevant audit when:

• A new source is connected
• An identity threshold changes
• A match rule changes
• The vendor changes its algorithm
• The organization enters a new market
• The consent or privacy environment changes

A launch-time result does not remain valid indefinitely.

Five Measurement Mistakes That Produce Reassuring But Wrong Numbers

The most common identity measurement failures are methodological, not technical:

• Reporting match rate as accuracy: Coverage and correctness are different.

• Mixing match-rate definitions: Records, events, and sessions are not interchangeable units.
• Measuring only pairwise accuracy: Pair averages can hide a few very large, badly merged clusters.
• Treating shared identifiers as ground truth: A household email or reused device ID can contaminate the answer key itself.
• Accepting a vendor benchmark instead of testing your data: Ask for precision, recall, ground-truth methodology, and sample design, then reproduce the measurement on your own records.

How Stable Kernel Measures Identity Resolution Accuracy

Stable Kernel treats identity accuracy measurement as part of the CDP implementation and operating model, not as a one-time vendor acceptance statistic.

Define The Measurement Method Before The Gate

Stable Kernel can help enterprise teams build an identity accuracy measurement plan that defines the match-rate unit, ground-truth sources, sample design, reviewer process, precision and recall calculations, cluster review requirements, and tolerance by use case.

That same methodology can then support the pre-production identity validation gate and establish a repeatable baseline for post-launch monitoring.

Turn Identity Accuracy Into An Ongoing Control

After go-live, the measurement process should not disappear.

Stable Kernel helps teams establish the weekly proxies, quarterly precision and cluster audits, recall reviews, and event-driven revalidation needed to detect identity drift before inaccurate profiles begin affecting personalization, suppression, loyalty, attribution, or AI-driven activation.

If your CDP currently reports match rate but cannot show how many of those matches are actually correct, how many true relationships it missed, or whether large customer clusters still represent one person, Stable Kernel can help turn identity accuracy from a vendor claim into a measurable operating metric.

Request an identity accuracy assessment from Stable Kernel to build the ground-truth sample, measure false merges and false splits, evaluate cluster coherence, calibrate probabilistic confidence bands, and define the accuracy thresholds your highest-value use cases actually require.

FAQ

How Do You Measure Identity Resolution Accuracy?

Measure identity resolution against ground truth using precision, recall, and cluster coherence. Build a labeled sample from verified customer anchors and reviewed profiles, calculate each metric, and report the result with sample size and a confidence interval. Match rate alone does not measure accuracy.

Is Match Rate The Same As Identity Resolution Accuracy?

No. Match rate measures coverage, such as the share of records, events, or sessions linked to a known identity. It does not show whether those links are correct or whether valid links were missed, and the unit must always be stated.

What Are Precision And Recall In Identity Resolution?

Precision measures the share of identity links the system made that are correct, so low precision indicates false merges. Recall measures the share of true links the system found, so low recall indicates false splits. Both should be supplemented with cluster-level analysis because pairwise averages can hide large over-merged profiles.

How Do You Calculate A False Merge Rate?

Review a labeled sample of merged profiles and count how many contain records belonging to more than one person. Divide incorrect profiles by profiles reviewed; at the profile level, that is equivalent to 1 minus precision. Report the result with the sample size and an appropriate confidence interval.

How Do You Get Ground Truth For Identity Resolution?

Ground truth usually comes from verified anchor customers, strong deterministic links that are genuinely unique to one person, and clerical review of sampled profiles. Reviewers should use documented labeling rules, and a subset should be double-labeled to check agreement.

How Many Profiles Do I Need To Sample?

For a simple random sample, roughly 385 profiles provides about a ±5-point margin at 95 percent confidence when the underlying rate is unknown. Estimating a rate near 2 percent within roughly ±1 point requires about 750 profiles, while observing zero errors in 200 profiles gives an approximate upper 95 percent bound near 1.5 percent. Stratified sampling changes the calculation, so an analyst should confirm the design.

What Is A Good Identity Resolution Accuracy?

There is no universal benchmark. Set the tolerance according to the consequence of an incorrect match, using much tighter requirements for regulated, account-specific, consent, loyalty, or AI-driven actions than for aggregate analytics or audience expansion. Then measure that tolerance against your own data.

How Often Should Identity Resolution Accuracy Be Re-Measured?

Run automated drift indicators weekly, cluster and confidence distributions monthly, sampled precision and coherence audits quarterly, and recall audits once or twice per year. Re-run the relevant audit whenever identity rules, sources, thresholds, markets, or vendor algorithms materially change.

Can Stable Kernel Help Measure Identity Resolution Accuracy?

Yes. Stable Kernel helps enterprise teams define the measurement method, build ground-truth samples, calculate precision and recall, audit cluster coherence, calibrate probabilistic matching thresholds, establish use-case tolerances, and create the ongoing measurement cadence required after go-live.