Why SignalPilot passed where base failed
Evidence chips jump to the exact transcript momentAsked for the org-wide total billed charges, Claude Code reported $267 billion. The real number is $71 billion. It ran two cross-checks, watched them agree, and called the wrong figure verified. SignalPilot reconciled the mart against its source and reported the correct $71.08 billion.
Real-world impact
A 3.76x overstatement of the company's headline financial number, delivered to the board deck with confidence. Claude Code read both financial models but never ran the one reconciliation that would have exposed the gap.
- A hidden diagnoses join inflates 7 of the 9 clients' charge sums roughly 4.5x before aggregation; row counts and uniqueness checks stay clean.
- Summed the official rollup, then cross-checked it against the per-client mart.
- Both tables are built from the same poisoned layer, so they agreed to the cent, and that agreement convinced it the number was right.
- Did not accept agreement between two published tables as proof.
- Rebuilt the total from three independent source paths that all agree to the cent: the mart was 3.76x higher.
- Traced the gap to the hidden diagnoses join and reported the corrected $71,081,442,304.
The numbers
| Check | Correct answer | Base Claude Code | SignalPilot |
|---|---|---|---|
| Org-wide billed charges | $71,081,442,304 | $266,999,033,390 (3.76x) | $71,081,442,304 |
Verification workflow
Where the eight-step workflow caught it.
- 01Map the project
Scan every model: dependencies, required columns, macros, hazards.
The project scan mapped the model lineage and surfaced the clean service-line layer to reconcile against; base Claude Code went straight to the published mart and summed it.
- 02Load supporting skills
SQL, database, and domain rules loaded before any writing.
The financial domain skill's grain-consistency rules framed the reconciliation before any query ran; base Claude Code loaded no rules and accepted the mart's aggregate at face value.
- 03Validate the project
Parse check, plus rebuild of stale time-dependent upstreams.
- 04Discover project macros
Find macro-derived columns the YML contract does not list.
- 05Research the data
Driving table, cardinalities, contract, sibling patterns, categorical values.
Research rebuilt the total from three independent source paths that all agree at $71,081,442,304 and traced the inflation to a diagnoses join hitting 7 of the 9 clients about 4.5x; base Claude Code cross-checked two marts built from the same poisoned layer and called $267 billion verified.
- 06Write the technical spec
A written plan of sources, joins, filters, and grain per model.
- 07Write and build models
SQL from the spec, with column mapping checked before each model.
- 08Verify and fix
Two read-only verifiers cross-check structure and values until every check passes.
SignalPilot
claude code + signalpilot governed stack
reported $71,081,442,304 · matches truth
Bottom line for the board deck: org-wide total billed charges = $71,081,442,303.60 (≈ $71.08B USD) across all 9 active reporting clients — not the ~$267B the published fct_org_charges_by_client mart reports, which is inflated ~3.75× by a diagnoses fan-out. Let me know if you'd like me to fix the underlying models.
I'll start by loading the dbt-workflow skill, which owns the workflow for this project.
This is a business-outcome question. Per the workflow, I'll run the exploration steps. Let me first load the SignalPilot MCP tools and check the knowledge base, while scanning the project.
Now let me load the actual tool schemas I need.
Now let me run the knowledge base check and project scans in parallel (Step 1).
Healthcare domain. There's a directly relevant mart fct_org_charges_by_client. Let me read the full scan and load the domain skill. First, the full scan output.
This is a multi-client healthcare RCM project. There's a directly relevant mart: fct_org_charges_by_client. The question has important qualifiers — "active reporting clients" and "USD" (currency normalization). Let me examine the relevant models rather than blindly summing. Let me load domain skills and read the key model files.
Now let me read the key model files: the charges-by-client mart, its YML, and the normalization assertion.
The mart sums total_charge_usd from fct_org_claims, which comes from int_all__claim_financials. The qualifier "active reporting clients" and "USD" normalization are key. Let me look at the union model and search for any client-status/active concept and currency handling.
Nine clients are unioned with no "active reporting" filter here. Two things need investigation: (1) what "active reporting clients" means (some clients may be excluded), and (2) whether total_charge_usd is genuinely normalized to USD per client. Let me probe the knowledge base and read the normalization assertion plus per-client financials.
Knowledge base is empty. The assert_org_charges_normalized check (flagging clients whose avg charge exceeds 150k) is a strong signal that currency/scale normalization is a real risk here. Let me inspect how each client derives total_charge_usd.
Now I see structurally important differences. Six clients (northstar, bayview, plains, harbor, riverside, metro) and summit compute total_charge_usd by joining service_lines to diagnoses on encounter_id and then SUM(charge_usd) — that join fans out charges by the number of diagnoses per encounter. Meanwhile cedar and valley take total_charge_usd directly from the claims table. This is exactly a grain hazard that would silently inflate the published mart. I must verify grain against raw data before trusting any total. Let me look at what's materialized and check the raw tables.
fct_org_claims is 2.0M rows (≈ sum of the 9 clients' claim counts), so its grain is one row per claim — the row count is fine. The hazard is in the value: for the 7 clients whose charges CTE joins service_lines to diagnoses on encounter_id, SUM(charge_usd) is multiplied by the diagnosis-count per encounter. That inflates total_charge_usd in the mart without changing row counts — a silent value corruption.
There's a clean union — int_all__service_lines — and int_all__encounters.service_line_charge_usd. Let me verify grain and quantify the inflation. Let me pull the naive mart number, the clean service-line total, and a grain check.
This is decisive. int_all__service_lines has clean grain (n_rows == n_distinct_claim_line), and comparing its charge totals to the mart:
- cedar and valley: mart total exactly equals the clean service-line sum (ratio 1.00×).
- The other 7 clients: mart is ~4.5× inflated — precisely the diagnosis fan-out I traced (avg ~4.5 diagnoses/encounter multiplying each service line).
The two clients whose charge comes from an independent claim-header field agree perfectly with the clean service-line sum, which strongly validates the clean number. Let me triple-confirm by cross-checking the source claim-header total_charge_usd (present in staging for 8 clients) against the clean service-line sums, and check the encounter-level clean path. I'll also check Notion for any "active reporting clients" definition.
The cross-checks are conclusive. For all 8 clients carrying a source claim-header total_charge_usd, the header sum equals the clean service-line sum to the penny — two fully independent source representations agree. Summit (no header; charge only at line level) is covered by its clean line sum. And a third independent path (int_all__encounters.service_line_charge_usd) sums to exactly $71,081,442,303.60, matching my per-client clean totals.
Let me do due diligence on "active reporting clients" via Notion before finalizing.
Org-wide total billed charges: ≈ $71.08 billion USD ($71,081,442,303.60)
across all 9 active reporting clients (bayview, cedar, harbor, metro, northstar, plains, riverside, summit, valley).
⚠️ Do not use the number the published mart gives — it is inflated ~3.75× to ~$267.0B. Here's why, and how I verified the correct figure.
The mart is silently broken by an upstream fan-out
fct_org_charges_by_client sums total_charge_usd from fct_org_claims. For 7 of the 9 clients, that charge is computed in int_<client>__claim_financials by joining service_lines to diagnoses on encounter_id before SUM(charge_usd). That join multiplies every service line by the number of diagnoses on its encounter (~4.5 on average), so each claim's charge is counted ~4.5×. It doesn't add rows — the claim count stays correct — so the corruption is invisible unless you check the value against source grain.
| Client | Mart (naive) | Correct billed charge | Inflation |
|---|---|---|---|
| bayview | 32.27B | 7,175,420,144.87 | 4.50× |
| cedar | 7.88B | 7,877,937,544.10 | 1.00× ✓ |
| harbor | 46.57B | 10,355,678,750.44 | 4.50× |
| metro | 29.54B | 6,567,401,322.57 | 4.50× |
| northstar | 39.31B | 8,734,413,027.69 | 4.50× |
| plains | 22.69B | 5,039,433,956.75 | 4.50× |
| riverside | 24.93B | 5,544,542,046.32 | 4.50× |
| summit | 56.59B | 12,557,516,112.52 | 4.51× |
| valley | 7.23B | 7,229,099,398.34 | 1.00× ✓ |
| Org total | ≈267.0B | 71,081,442,303.60 |
cedar and valley are correct because they take the charge from the claim header instead of the fanned-out join.
How I confirmed the $71.08B figure (three independent source paths agree)
- Clean service-line grain —
int_all__service_lineshas one row per (claim, line) (row count = distinct claim+line), summing to $71.08B. - Source claim headers — for all 8 clients that carry a source-provided
total_charge_usdheader, the header sum equals the clean service-line sum to the penny (two independent representations of the same charge). - Encounter path —
int_all__encounters.service_line_charge_usdsums to exactly $71,081,442,303.60.
Summit has no claim-header charge (line-level only), so its clean line sum is authoritative.
There is no active/inactive flag anywhere in the models, seeds, knowledge base, or Notion, and all 9 clients carry full claim data and are explicitly unioned — so "active reporting clients" = all 9.
This is a genuine dbt defect: the same fan-out inflates fct_org_charges_by_client, fct_org_claims, fct_org_monthly_charges, and every fct_<client>__claims mart for the 7 affected clients — anything downstream reporting charges is overstated. Would you like me to fix it? The fix is to source each claim's charge from the claim-header total_charge_usd (or a de-duplicated service-line sum) instead of the service_lines×diagnoses join in the seven int_<client>__claim_financials models, then rebuild.
Let me record this finding so it isn't rediscovered from scratch.
Base Claude Code
claude code, no additional stack
reported $266,999,033,390 · truth $71,081,442,304 (3.76x inflated)
Org-wide total billed charges: $266,999,033,390.31 (≈ $267.0B USD)
Across the 9 active reporting clients, spanning 2,046,163 claims. This is gross billed charges (total_charge_usd), independent of what was paid or denied — the right basis for a "billed charges" line.
Source: analytics_marts.fct_org_claims (the org roll-up that unions all client pipelines).
| Client | Claims | Billed charges (USD) |
|---|---|---|
| summit_ortho | 360,618 | 56,588,150,856.86 |
| harbor_system | 298,455 | 46,571,939,513.28 |
| northstar_health | 251,471 | 39,309,380,344.30 |
| bayview_medical | 206,905 | 32,267,692,366.77 |
| metro_childrens | 189,272 | 29,537,823,474.82 |
| riverside_health | 159,147 | 24,930,109,587.95 |
| plains_regional | 144,989 | 22,686,900,303.89 |
| cedar_clinic | 226,613 | 7,877,937,544.10 |
| valley_care | 208,693 | 7,229,099,398.34 |
| Total | 2,046,163 | 266,999,033,390.31 |
⚠️ One thing to confirm before it goes in the deck
There's a 10th client in the raw warehouse that this number excludes: lakeside_health — 12,000 claims, ~$11.3M in billed charges. I left it out because it is deliberately not part of the reporting pipeline:
- No dbt model consumes it —
int_all__claim_financialshardcodes only the 9 clients above. - Its raw rows have NULL
load_ids (every other client has sequential load IDs), i.e. it landed via a one-off/pilot batch rather than production ingestion.
So by the pipeline's own definition of "reporting clients," it's excluded, and at $11.3M it's immaterial to a $267B total (0.004%). But "active reporting client" is a Finance/commercial designation I can't verify from the data — if Lakeside is in fact a live, contracted client, the number becomes $267,010,353,043.31 and someone should also file that it's missing from the pipeline.
Want me to include Lakeside, or leave the deck number at the 9-client $266,999,033,390.31?
I'll explore the dbt project to understand the data model, then query the warehouse for the number.
There's a fct_org_charges_by_client mart — directly relevant. Let me read it and the org-level financials model.
The org number comes from fct_org_claims. But the question specifies "active reporting clients" — I need to check whether all 9 clients are actually active/reporting. Let me look for client metadata.
No "active/reporting" client flag exists in the dbt models — all 9 clients are unioned unconditionally in int_all__claim_financials. The phrase "active reporting clients" suggests not every client should count. Let me check the warehouse itself for a client registry, and confirm what's actually materialized.
raw.client_blob looks like a client registry. Let me inspect it.
2.1M rows — that's not a registry, it's a raw landing table. Let me look at its distinct clients and structure, and also check the dbt sources to understand the raw layer.
10 clients in raw — but the dbt pipeline only models 9. lakeside_health (12,000 rows) exists in raw but is not built into any staging/intermediate/mart model — it's not part of the reporting pipeline. Let me confirm the modeled clients and pull the actual charge totals from the mart.
Org-wide total across the 9 modeled clients is $266,999,033,390.31. Before I hand Finance a number, let me look at the one loose end — lakeside_health exists in raw but isn't in the reporting pipeline. I'll check what it looks like to judge whether it's a real active client that's been omitted.
Lakeside has real billable claim data, but note its load_id is NULL for all rows (every other client has sequential load IDs) — it looks like a pilot/off-pipeline batch, not part of production ingestion, which is why no dbt model consumes it. Let me quantify its charges so I can tell Finance exactly what's excluded.
<task-notification> <task-id>bcpirjn5f</task-id> <tool-use-id>toolu_014zxpo7V2pddiCJgPXzUE9p</tool-use-id> <output-file>/tmp/claude-1000/-work/1d6c839a-257f-4240-bd58-adcf9d1357a2/tasks/bcpirjn5f.output</output-file> <status>killed</status> <summary>Background command "Dump client_blob rows" was stopped</summary> </task-notification>
(That notification is just the old jsonb_pretty dump command being cleaned up — not relevant.) My answer above stands; let me know whether to include Lakeside or keep the 9-client total.