Eliminating False Positives on International and Healthcare Identifiers

If your DLP pipeline blocks every government identifier it encounters — even when it appears in a template, a test fixture, or a regulatory FAQ — your security team stops trusting the system. They turn it off, whitelist the entity type, or route around it entirely. At that point, you have no protection at all.

This is what we set out to fix with our latest contextual validator.

The Problem: Legitimate IDs, Wrong Context

The Arbitex DLP pipeline uses a multi-layer content inspection pipeline. Tier 1 catches structured patterns with checksums. Tier 2 uses named entity recognition to identify sensitive data in context. Tier 3 — the contextual validator — uses AI-powered contextual analysis to answer a harder question: is this detected entity actually sensitive in this specific context?

The contextual analysis approach works by evaluating the surrounding text alongside a hypothesis about the candidate entity. The model scores whether the context supports or contradicts that hypothesis. High confidence means enforce. High contextual rejection means allow.

The previous generation of the contextual validator handled most entity types well. But a set of international and healthcare identifier types had persistent false positive problems. These identifiers share a property: their formats overlap heavily with non-sensitive numbers that appear routinely in enterprise communications — invoice IDs, configuration values, test data, and reference codes.

What Changed

We retrained the contextual validator with targeted expansions in five areas:

• ID disambiguation: Training examples where government-issued identifiers appear alongside non-sensitive numbers in similar formats, forcing the model to rely on contextual signals rather than format alone.
• Medical context: Clinical, administrative, and insurance scenarios where healthcare identifiers appear in both genuine patient records and reference documentation.
• Infrastructure logs: System outputs containing numeric strings that match ID formats but carry no PII — configuration hashes, port numbers, process IDs, health check values.
• Code and credentials: Developer contexts where test fixtures, mock data, and example values use realistic-looking identifiers that are not real.
• Chat messaging: Conversational patterns where users reference IDs in ways that differ from formal documents — shorthand, partial context, follow-up messages.

The result is a model that distinguishes between a real government identifier in a customer service record and a test value in a developer’s unit test.

Results: Problem Entity Types Resolved

The entity types that previously had the highest false positive rates — including international government identifiers and healthcare identifiers — now achieve accurate detection with near-zero false positives. Most previously-problematic types now produce perfect detection scores. The remaining types show improvements that reduce manual review burden to negligible levels.

Across 40+ entity types including international identifiers and healthcare identifiers, the contextual validator materially reduced false positives — the failure mode that causes security teams to disable a DLP system rather than tune it.

We also evaluate against a dedicated adversarial test suite across multiple attack categories: code fixtures containing realistic test data, credentials embedded in documentation, fictional names in realistic contexts, format-obfuscated identifiers, and structured data with embedded entities. These are the categories where naive pattern matching fails most often, and they are where contextual validation earns its place in the pipeline.

Why This Matters for Enterprise Deployment

A DLP system with a high false positive rate on international identifiers creates a specific operational problem: it makes the system unusable for organizations with international employees, customers, or compliance obligations.

If your security team has to whitelist identifier types because the DLP pipeline blocks every occurrence, you have no detection for those types at all — not just reduced detection, but zero. This applies equally to international government IDs, healthcare provider identifiers, employer identification numbers, and other identifiers that appear across globally operating enterprises.

The current contextual validator means these entity types work in production without tuning, without whitelists, and without your security team learning to ignore alerts. Detection fires when it should and stays quiet when it should not.

For more on how the multi-layer content inspection pipeline works, read the full pipeline series. For our accuracy measurement methodology, see how we measure and publish DLP accuracy.

Book a demo to see contextual validation in action on your AI traffic.