Category: False Positive Reduction

  • Years ago, I worked with a system that had a very extensive global synonym data. Every once in a while, we caught it making matches it really shouldn’t have, so we got the vendor to turn some off – since our customers wanted an explanation that made sense to them.

    From that, I wondered, if AI/LLM based systems (my “One Funnel” model) wouldn’t make things worse.

    So, my prompt:

    I am familiar with 2 basic kinds of false positive reduction for sanctions screening systems. One model relies on sophisticated name matching algorithms rather than whitelisting or rules-based processing to minimize false positive matches. I call that a “one funnel” model.

    In contrast, a “two funnel system” uses a more basic set of matching algorithms, combined with user-built and maintained false positive reduction rules, multi-element score boarding, and/or white listing tools.

    Assume, as a starting point, that this is a valid way to categorize the overwhelming bulk of systems offered by solutions providers.

    Now, assume that third party data is in the Latin character set, but is global in nature (e.g. could represent parties from any country on Earth).

    Assuming that tokens that appear on sanctions lists also have alternative, benign meanings in other languages (e.g. “PRIMO” can mean “best” or “first”) but, due to the low incidence of single-token sanctioned party names, the actual incidence where such collisions occur is very small.

    Assume, too, that the system must be tuned with a significant bias toward minimizing false negatives, although there needs to be the capability to also reduce false positives in a significant way.

    How do these considerations affect the choice of a false positive reduction model and how it is designed, configured, implemented and maintained?

    And the response I got:

    Based on your scenario—specifically the need to handle global data, the presence of rare “dictionary word” collisions, and a strict requirement to miss zero sanctions hits—the considerations heavily favor a “Two Funnel” system architecture.

    Here is the analysis of how your assumptions dictate the design, configuration, implementation, and maintenance, with key industry terms explained for clarity.


    1. Choice of Model: The “Two Funnel” Imperative

    While “One Funnel” systems (often AI-driven models that try to decide “match or no match” in a single step) are efficient, your requirement to minimize False Negatives makes them risky.

    • The Problem with One Funnel: To ensure you don’t miss any bad actors in a “One Funnel” model, you have to tell the system to be very paranoid. This lowers the confidence threshold (the minimum similarity score required to trigger an alert). Without a secondary filter, this “paranoid” setting floods you with False Positives (innocent parties wrongly flagged as sanctioned) that the algorithm can’t easily explain away.
    • The Strength of Two Funnel: This architecture separates Recall from Precision.
      • Recall (The ability of the system to catch every possible bad actor): You handle this in Funnel 1 by casting a very wide net.
      • Precision (The ability of the system to filter out the noise and only present true risks): You handle this in Funnel 2 using specific logic rules.

    Verdict: Because you cannot afford to miss a hit, you need the Two Funnel approach. It allows you to be “loose” in the initial scan to catch everything, and “strict” in the second phase to clean up the mess.


    2. Design Implications

    Designing the system under these assumptions requires a “Wide Net, Fine Mesh” architecture.

    • Funnel 1 Design (The Wide Net):
      • Algorithm Selection: Since your data is global but uses the Latin alphabet (A-Z), simple “sounds-like” algorithms (phonetics) aren’t enough. You need algorithms focused on Edit Distance (calculating how many typos or character changes it takes to turn one name into another). This catches typos and variations in transliteration (the process of converting names from scripts like Arabic or Cyrillic into English characters).
      • Segmentation: You should design the system to treat people and companies differently. A collision with the word “PRIMO” is fine for a company (e.g., “Primo Logistics”) but suspicious for a person. The design must allow different rules for different entity types.
    • Funnel 2 Design (The Fine Mesh):
      • The “Weak Alias” Logic: Because you noted that single-word collisions (like “PRIMO”) are rare but possible, you should design a “Weak Alias” or “Stop List” module. This prevents the system from generating alerts solely on common dictionary words unless there is more corroborating data (like a matching country or date of birth).

    3. Configuration & Tuning

    The global nature of your data and the “Zero False Negative” goal drive specific configuration choices.

    • Fuzzy Matching Thresholds:
      • To minimize False Negatives (missed hits), you must configure your matching thresholds loosely (e.g., flagging names that are only 80% similar). This accounts for the messy reality of global data.
      • Effect: This guarantees the system catches variations like “Mohammed,” “Mohamad,” and “Muhamad.”
    • Handling the “PRIMO” Assumption:
      • Since actual collisions with dictionary words are statistically rare, you do not need complex AI to figure out context.
      • Configuration: Instead, implement a Token Whitelist Rule.
        • Rule Logic: “If the matched token (the specific word or part of the name) is ‘PRIMO’, AND it is the onlyword that matches, THEN suppress the alert.”
        • This effectively neutralizes the issue without over-engineering the system.
    • Multi-Element Score-boarding:
      • Score-boarding is a method where the system assigns “points” to a match based on various factors.
      • Since the data is global, you cannot rely on name matching alone. You must configure geographic weighting. If the customer is in “Canada” and the sanctioned party is in “Iran,” the system should deduct points from the risk score, helping automate the dismissal of the alert.

    4. Implementation Strategy

    Implementing this system requires a focus on cleaning data before the matching even begins.

    • Pre-Processing (The Hidden Funnel):
      • Because the data is global, implementation must include a normalization layer. This means stripping out accents, umlauts, and special characters (converting ñ to nç to c) so that the basic matching algorithms in Funnel 1 can compare “apples to apples.”
    • Testing for “Zero Miss” Bias:
      • You cannot just test random names. You must implement a “Sanctions Testing Deck”—a dataset of known sanctioned entities with intentionally corrupted spellings (e.g., testing “Usama bin Ladin” to see if it matches “Osama bin Laden”). The system is only ready for launch if Funnel 1 catches 100% of these variations.

    5. Maintenance & Governance

    The choice of a Two Funnel system shifts the operational burden from monitoring a “Black Box” AI to managing a set of transparent rules.

    • Rule Decay:
      • The “PRIMO” rule works today. But if the government sanctions a vessel named “PRIMO” tomorrow, your rule essentially creates a False Negative (a blind spot).
      • Maintenance Protocol: You must implement a “List Management” process. Every time a sanctions list is updated, an automated script must check if any new sanctioned names conflict with your existing “Good Guy” rules.
    • False Negative Tuning:
      • To ensure you aren’t missing things, you must perform “Below the Line” testing.
      • Process: Periodically look at customers who did not generate an alert. Run them against a secondary, more aggressive screening tool. If you find a missed match, you know you need to loosen the settings in Funnel 1.

    Summary Table

    FeatureEffect of your Assumptions
    Model ChoiceTwo Funnel. It allows you to keep the “Screening” loose (to catch everything) while using “Rules” to manage the resulting noise.
    AlgorithmsMust focus on Edit Distance (fuzzy matching) rather than phonetics to handle global spelling differences.
    Linguistic NoiseAddressed via exclusion lists (lists of words to ignore) due to the low frequency of these collisions.
    Risk BiasFalse Negative intolerance dictates loose matching thresholds in Funnel 1 to ensure high Recall.
    MaintenanceFocus shifts to Rule Hygiene (ensuring your rules don’t accidentally hide new sanctions).

    Was my prompt exactly what I wanted to say? Nah, but it was pretty close – yes, in our experience, we had very few collisions between tokens that had meanings in multiple languages. But I think it was more about that matching is, largely, a multiple token matching business – with the exception of cargo vessels, of course. So, even if you hit one of those colliding tokens, usually you’re going to have to match something else in addition – and that reduces the likelihood you’ve got a match to review.

    It also highlights a design consideration for matching systems. Do you need synonyms, semantic equivalents, etc for an extensive set of common terms – or only for tokens in sanctions listings? I suspect it’s the latter, which might call into question risk/reward of LLMs in cases where the source data is more heterogeneous.

  • These evaluation questions are designed to cut through the marketing fluff. Vendors often use terms like “AI-powered” loosely; these questions will force them to reveal whether they are truly building a “One Funnel” (precision) architecture or simply patching a “Two Funnel” (filtering) system.

    The evaluation framework has been restructured to distinguish between universal “Funnel” mechanics (applicable to both worlds) and those features that primarily drive efficiency in Static Data (KYC/Customer) screening.

    Part 1: Universal Design Considerations

    These factors are critical regardless of whether you are screening a static client database or a live wire transfer. They address the fundamental “One Funnel vs. Two Funnel” architecture of the engine itself.

    1. The Matching Engine Mathematics (The “Funnel” Type)

    Whether screening a customer name or a payment string, the core algorithm dictates the baseline efficiency.

    • Question: “Is the matching engine based on deterministic/probabilistic algorithms (e.g., Levenshtein, Jaro-Winkler) or Semantic AI (Vector/Neural Networks)?”
    • Why it’s Universal: Both transaction and static screening suffer from basic fuzzy matching errors (e.g., over-flagging “Main Street”). A “One Funnel” (Semantic) engine reduces noise in both environments by understanding context and meaning rather than just character overlap.

    2. Synonym & Variation Handling

    • Question: “Does the system identify synonyms (e.g., Bill = William, Ltd = Limited) via a hardcoded dictionary or learned semantic associations?”
    • Why it’s Universal: A terrorist or sanctioned entity can disguise their name in a payment instruction just as easily as they can in account opening documents. If the system relies on a manual dictionary (Two Funnel), you have to maintain that dictionary for both workflows.

    3. Data Normalization & Cleaning (Pre-Computation)

    • Question: “How does the system handle dirty or concatenated strings (e.g., ‘IBM_CORP_NY’ or ‘PaymentRef:Inv#1234’)?”
    • Why it’s Universal:
      • In Static: Bad data entry exists in legacy systems.
      • In Transactions: SWIFT/ISO messages often cram names, addresses, and references into single free-text fields.
      • Note: The system must be able to parse and clean these strings before matching, or the “One Funnel” engine will fail.

    4. Model Governance & Explainability

    • Question: “Can you generate a report explaining why a specific variation was not flagged?”
    • Why it’s Universal: Regulators (OFAC/OFSI) require validation for both systems. If you cannot explain why a wire transfer wasn’t stopped (False Negative), or why a customer wasn’t flagged, the regulatory penalty is the same.

    Part 2: Static Data-Specific Considerations

    These considerations rely on rich, structured data (Dates of Birth, Citizenship, full addresses). They are highly effective for “Two Funnel” reduction in Static Data screening but are less applicable or difficult to implement in Transaction Screening because payment messages are often ephemeral, unstructured, and lack these specific data fields.

    1. Multi-Dimensional Scoring (The “Tie-Breaker”)

    • The Capability: Weighting the match score based on non-name attributes. (e.g., “If Name matches 100% but Year of Birth is >10 years apart, reduce score by 50%”).
    • Why it’s Static-Dominant:
      • Static Data: You almost always have the KYC data (DOB, Country of Citizenship) to compare against the sanctions list.
      • Transaction Screening: A standard SWIFT MT103 or ISO 20022 message often does not contain the Date of Birth or Citizenship of the beneficiary. Therefore, a scoring model capable of weighing these factors is useless for the vast majority of payments.

    2. Entity-Based Whitelisting (“Golden Record” Suppression)

    • The Capability: Permanently suppressing a match for a specific customer entity ID after a human reviews it (e.g., “Client ID 12345 is NOT the terrorist John Smith. Never flag him again.”).
    • Why it’s Static-Dominant:
      • Static Data: You screen the same unique “Customer ID” periodically. Once cleared, the “Two Funnel” whitelist prevents re-alerting on that ID.
      • Transaction Screening: You are screening a string of text in a message, not a “Customer ID.” The text might change slightly (“J. Smith” vs “Mr. John Smith”). You cannot reliably “whitelist” a text string without risking that a bad actor might use that same string later.

    3. Delta Screening (Trigger-Based Scanning)

    • The Capability: Only screening records that have changed or when the sanctions list updates, rather than re-screening the whole database every day.
    • Why it’s Static-Dominant:
      • Static Data: This is the primary efficiency driver for customer databases.
      • Transaction Screening: Every transaction is a “new” event. You cannot “Delta screen” a wire transfer; you must screen the whole message every time it occurs.

    Summary of Differences

    FeatureImportance in Static ScreeningImportance in Transaction Screening
    Semantic AI EngineHigh (Reduces review volume)High (Reduces stopped payments)
    DOB/Nationality WeightingCritical (Major false positive reducer)Low (Data rarely exists in message)
    Entity WhitelistingCritical (prevents “Groundhog Day” reviews)Low (Too risky to whitelist text strings)
    Address Fuzzy MatchingHigh (Matches client address to sanctioned city)Medium (Payment addresses are often unstructured
  • I have always looked at False Positive Reduction (FPR) in sanctions screening in a certain way – that there were systems that relied on matching algorithms and others that relied more on user-built tools like scorecards and FPR rules.

    What did Gemini think of my way of looking at things?

    First, my prompt:

    I am familiar with 2 basic kinds of false positive reduction for sanctions screening systems. One model relies on sophisticated name matching algorithms rather than whitelisting or rules-based processing to minimize false positive matches. I call that a “one funnel” model. In contrast, a “two funnel system” uses a more basic set of matching algorithms, combined with user-built and maintained false positive reduction rules, multi-element score boarding, and/or white listing tools. IS that an appropriate way to categorize the false positive reduction capabilities of these systems? If not, are there other ways of categorizing these systems? What other considerations have I not considered here?

    Here’s the response I got:

    Your “One Funnel vs. Two Funnel” analogy is an astute and highly effective way to conceptualize the philosophical divide in modern sanctions screening. You are essentially distinguishing between upstream optimization (preventing the false positive from being generated) and downstream optimization (suppressing the false positive after generation).

    Here is a breakdown of why your categorization works, alternative ways to view the market, and the critical considerations you may have overlooked.


    1. Validating Your “Funnel” Model

    Your categorization is appropriate because it accurately describes where the “intelligence” of the system resides.

    • The “One Funnel” (Precision-First Model):
      • Philosophy: “Get it right the first time.”
      • Mechanism: Relies on heavy computational linguistics, AI, and fuzzy logic during the matching process to ensure that only high-probability matches are generated.
      • Pros: Fewer alerts to manage; cleaner data flow.
      • Cons: “Black box” risk (hard to explain to regulators why a name didn’t match); computationally expensive; difficult to tune without data science expertise.
    • The “Two Funnel” (Filter-First Model):
      • Philosophy: “Cast a wide net, then filter the catch.”
      • Mechanism: Uses a “looser” matching engine (often deterministic or basic fuzzy) to grab many potential matches, then applies a secondary logic layer (whitelist, scoring, secondary field checks) to discard the noise.
      • Pros: Transparent and explainable (you can see exactly which rule suppressed the alert); easier for compliance officers to manually tune; often safer for risk-averse institutions.
      • Cons: High initial volume; relies heavily on the maintenance of complex rule sets which can become unmanageable “spaghetti logic” over time.

    2. Alternative Categorizations

    While your funnel model focuses on process flow, the industry often categorizes these systems based on technology generation or data usage.

    A. Deterministic vs. Probabilistic vs. Semantic

    • Deterministic (Legacy): Relies on exact matches or simple wildcards. Zero intelligence. High false negatives (misses risks) or massive false positives.
    • Probabilistic (Fuzzy): Uses algorithms like Levenshtein Distance or Soundex to measure similarity scores (e.g., “Match if name is 85% similar”). This is the standard “Two Funnel” base.
    • Semantic/Vector (AI-Native): Uses Neural Networks to understand that “Bill” and “William” are the same entity without sharing letters. This typically aligns with your “One Funnel” model.

    B. The “Three-Stage” Lifecycle

    Rather than just funnels, many vendors categorize by where the reduction happens in the lifecycle:

    1. Pre-Computation (Data prep): Cleaning and normalizing data before it hits the engine.
    2. In-Memory (The Match): The algorithms you described.
    3. Post-Process (Decisioning): The “score boarding” and “rules” you described.

    3. Critical Considerations You May Have Missed

    Your model focuses heavily on Name Matching algorithms and Rules. However, modern false positive reduction relies on several other pillars that don’t fit neatly into either bucket.

    A. The “Data Hygiene” Paradox

    Neither funnel works if the input data is garbage.

    • Consideration: A system might have the world’s best “One Funnel” AI, but if your internal customer data has “N/A” listed as a last name, you will get false positives.
    • Correction: You should consider Pre-processing/Normalization as a distinct capability. Does the system strip special characters, standardize “Street” vs “St”, and parse concatenated names before matching?

    B. Secondary Attribute Matching (The “Tie-Breaker”)

    You focused on name matching algorithms. However, the biggest driver of False Positive reduction is not the name, but the secondary identifiers.

    • The Gap: A “One Funnel” system might correctly identify that “John Smith” looks like “John Smyth”. But does it automatically penalize the score if the Dates of Birth are 20 years apart?
    • Multidimensional Scoring: Sophisticated systems don’t just match names; they match Name + Country + DoB as a single weighted vector. This is often more effective than name matching alone.

    C. Explainability (XAI) vs. Performance

    • The Regulatory Trap: The “One Funnel” AI model is dangerous if it cannot be explained. If a regulator asks, “Why did you not flag this variation of a terrorist’s name?”, answering “Our neural network didn’t feel it was close enough” is often unacceptable.
    • Consideration: Does the system provide a “white box” explanation for its AI decisions?

    D. Predictive Adjudication (The “Third Funnel”)

    There is a new category emerging that sits after your “Two Funnel” concept.

    • AI Analyst Agents: Instead of rules (IF/THEN), these systems observe human analysts. If humans dismiss “partial matches on common Spanish surnames” 1,000 times, the AI learns to auto-dismiss the 1,001st instance. This is Hibernation/Auto-closure based on behavioral learning, not just static rules.

    Summary Table

    Capability“One Funnel” (Your Model)“Two Funnel” (Your Model)The Missing Piece (Holistic)
    Primary FilterSophisticated Matching AIBasic Matching + Post-RulesData Normalization + Contextual Vectoring
    Logic“Smart” Pattern Recognition“Brute Force” Logic/WhitelistsBehavioral Learning (Predictive)
    StrengthPrecisionAuditability/ControlEfficiency & Context
    WeaknessExplainability (Black Box)Maintenance OverheadData Dependency