SignalPilot is #1 on both public data-agent benchmarksSee the results →
All cases

Poisoned money sums

For Bayview Medical's payer-mix review: how much have they billed in total charges (USD) to commercial payers? One number, please.

The payer-mix review said $20.14 billion billed to commercial payers. The real figure is $4.48 billion.

Case 27 of 39Δ $15.7Bnorthwind

Why SignalPilot passed where base failed

Evidence chips jump to the exact transcript moment

Asked for Bayview Medical's charges billed to commercial payers, Claude Code reported $20.14 billion. The real number is $4.48 billion, 4.5x lower. SignalPilot reconciled the published mart against its source data and reported the correct figure.

Real-world impact

A 4.5x overstatement of a headline financial figure, delivered with confidence. The agent cross-checked two tables that are built from the same poisoned layer and took their agreement as proof.

Where base Claude Code went wrong
  • A hidden diagnoses join inflates every charge sum roughly 4.5x before aggregation; row counts and uniqueness checks stay clean.
  • Summed the published mart and shipped $20.14 billion as verified; its checks never left the poisoned layer.
What SignalPilot did differently
  • Did not take the published mart at face value: it recomputed the figure from the staging claim headers and service lines, which agree with each other to the penny.
  • Traced the gap to the diagnoses join inside the claim financials model and reported the corrected $4.48 billion.

The numbers

CheckCorrect answerBase Claude CodeSignalPilot
Bayview Medical's charges billed to commercial payers$4.48 billion$20.14 billion (4.5x)$4.48 billion

Verification workflow

Where the eight-step workflow caught it.

Read the workflow docs
  1. 01
    Map the project

    Scan every model: dependencies, required columns, macros, hazards.

  2. 02
    Load supporting skills

    SQL, database, and domain rules loaded before any writing.

    The loaded workflow and healthcare domain skills primed the run to distrust a clean-looking published mart; base Claude Code simply summed the table it was handed.

  3. 03
    Validate the project

    Parse check, plus rebuild of stale time-dependent upstreams.

  4. 04
    Discover project macros

    Find macro-derived columns the YML contract does not list.

  5. 05
    Research the data

    Driving table, cardinalities, contract, sibling patterns, categorical values.

    Research reconciled the mart against staging, found the same claims summing 4.5x higher, traced the cause to the diagnoses join in the claim financials model, and corroborated the true $4.48 billion two independent ways; base Claude Code shipped $20.14 billion.

  6. 06
    Write the technical spec

    A written plan of sources, joins, filters, and grain per model.

  7. 07
    Write and build models

    SQL from the spec, with column mapping checked before each model.

  8. 08
    Verify and fix

    Two read-only verifiers cross-check structure and values until every check passes.

SignalPilot

SignalPilot

claude code + signalpilot governed stack

pass
reported $4.48B · matches truth
33 turns30 tool calls$$1.05answer pinned · full chat below
final answer pass

To recap:

Bayview Medical's total charges billed to commercial payers: $4,475,191,405.89 (~$4.48B, across 129,260 commercial claims).

Verified from the raw claim source and independently corroborated by summing service lines — both agree to the cent. I deliberately avoided the fct_bayview__claims mart, which reports $20.1B because of a diagnoses fan-out bug in int_bayview__claim_financials. Let me know if you'd like me to fix that model and rebuild the affected marts.

full chat, from the start
For Bayview Medical's payer-mix review: how much have they billed in total charges (USD) to commercial payers? One number, please.

I'll start by loading the required workflow skill, as the project instructions mandate.

This is a business-outcome question involving the database, so I'll follow the dbt-workflow exploration steps. Let me start Step 1: map the project and load the SignalPilot MCP tools I'll need.

MCP tools are now available. Let me load the schemas for the ones I need.

Now let me gather knowledge and DB hints in parallel.

Knowledge base is empty. Let me load the healthcare domain skill (payer/charges is clinical/financial healthcare) and explore the relevant tables.

Now let me examine the payer dimension, the crosswalk, and how Bayview's claims reference payers. This is the crux — I need to know whether fct_bayview__claims.payer_id is a canonical payer id or a client-local code that must be crosswalked.

Bayview's claims already carry canonical payer codes (AETNA, BCBS, CIGNA, HUMANA, UHC = commercial; the xref only applies to riverside). Now I must verify grain against the raw source before summing. Let me check grain and cross-check the staging source.

Grain confirmed: exactly one row per claim (206,905 = distinct claims), and the mart matches the raw staging source row-for-row — no fan-out. Now let me compute the commercial total from both the raw staging source and the mart, deriving "commercial" from dim_payer.payer_type rather than hardcoding.

The claim counts match exactly (129,260) — but the charge total differs 4.5×: staging says $4.475B, the mart says $20.14B for the identical set of claims. This is exactly the kind of silently-broken mart the project instructions warn about. Let me investigate per-claim before trusting either number.

The bug is now visible. For each claim, the raw staging charge equals the sum of its service lines exactly (e.g. CN-0000001: $38,442.77), but fct/int inflate it to a multiple ($115,328.31). The intermediate model fans out the charge via a join. Let me read the model SQL to confirm the mechanism and run a third independent cross-check from service lines.

Answer

Bayview Medical has billed $4,475,191,405.89 (≈ $4.48 billion) in total charges to commercial payers.

  • Commercial payers = Aetna, Blue Cross Blue Shield, Cigna, Humana, UnitedHealthcare (the 5 payers with payer_type = 'commercial' in dim_payer; Medicare/Medicaid are government, Self Pay is self).
  • This spans 129,260 commercial claims (of 206,905 total Bayview claims). No NULL charges.
How I verified it (and a warning you should hear)

I did not trust the published mart — and it's a good thing, because it's wrong. Three checks:

SourceCommercial total charges
stg_bayview__claims (claim header, raw)$4,475,191,405.89
Sum of stg_bayview__service_lines (independent)$4,475,191,405.89
fct_bayview__claims / int_bayview__claim_financials (mart)$20,137,466,285.66 ❌ (4.5× inflated)

Grain is clean — one row per claim (206,905 rows = distinct claims), and the raw claim header total equals the sum of its service lines exactly. Two independent paths agree to the cent.

⚠️ The mart is silently broken

int_bayview__claim_financials.sql (which feeds fct_bayview__claims) computes charges like this:

charges as (
    select sl.claim_id, sum(sl.charge_usd) as total_charge_usd
    from stg_bayview__service_lines sl
    join stg_bayview__diagnoses dx on sl.encounter_id = dx.encounter_id   -- ← fan-out
    group by sl.claim_id
)

Joining service lines to diagnoses on encounter_id multiplies every service-line charge by the encounter's diagnosis count, inflating total_charge_usd per claim. The same bug also multiplies paid_amount_usd (line 26: remit.paid_amount_usd * dx_ct) and corrupts unpaid_usd. Any dashboard reading total_charge_usd, paid_amount_usd, or unpaid_usd from fct_bayview__claims (and likely the org-wide fct_org_claims, which follows the same pattern) is reporting roughly 4–5× the real figures.

This is a real compilation-clean but factually-wrong dbt bug. Would you like me to fix int_bayview__claim_financials (drop the diagnoses join from the charges CTE and remove the dx_ct multiplier from paid/unpaid), then rebuild it and the downstream marts? It's worth checking the other eight clients' int_*__claim_financials models for the same pattern too.

Let me save this hazard so it doesn't bite future analyses.

Claude Code

Base Claude Code

claude code, no additional stack

fail
reported $20.14B · truth $4.48B (4.5x inflated) · single-shot run, only the final reply was captured
7 turns0 tool calls$$0.17answer pinned · full chat below
final answer fail

$20,137,466,285.66 in total charges billed to commercial payers.

This is Bayview Medical's fct_bayview__claims, summing total_charge_usd across the five commercial payers (Aetna, Blue Cross Blue Shield, Cigna, Humana, UnitedHealthcare) per dim_payer.payer_type = 'commercial', across 129,260 claims. Government (Medicare/Medicaid) and Self Pay are excluded.

One caveat worth flagging for a payer-mix review: this is total charges, not collections — actual paid amounts (paid_amount_usd) will be materially lower.

full chat, from the start
For Bayview Medical's payer-mix review: how much have they billed in total charges (USD) to commercial payers? One number, please.
Case #27 · Poisoned money sums · SignalPilot