Why SignalPilot passed where base failed
Evidence chips jump to the exact transcript momentAsked for drift_labs's total experiment enrollments, Claude Code reported 89,944. The raw landing table holds 185,216: 51% of the rows had silently vanished below the mart. SignalPilot reconciled the count down to the raw data and recovered the full population.
Real-world impact
Capacity planning, billing, and experiment power calculations built on this count all start from a number 51% too low, and nothing in the pipeline flags it: the shrunken table has clean grain, clean uniqueness, and an authoritative name.
- Below the published mart, a filter drops every enrollment whose variant label is missing, discarding over half the population.
- Row counts and uniqueness checks still pass; only the total is wrong.
- Counted the authoritative-looking table and reported 89,944 with confidence; the raw landing table was never consulted.
- Started from the client's own staging and raw landing data instead of the published rollup, counting the population at every layer.
- Reported 185,216 from the source, matching the independently derived truth exactly.
The numbers
| Check | Correct answer | Base Claude Code | SignalPilot |
|---|---|---|---|
| Drift_labs's total experiment enrollments | 185,216 | 89,944 (51% missing) | 185,216 |
Verification workflow
Where the eight-step workflow caught it.
- 01Map the project
Scan every model: dependencies, required columns, macros, hazards.
The scan mapped lineage from raw landing data up through the marts, steering the count to the client's own staging model instead of the org rollup; base Claude Code counted the rollup.
- 02Load supporting skills
SQL, database, and domain rules loaded before any writing.
- 03Validate the project
Parse check, plus rebuild of stale time-dependent upstreams.
- 04Discover project macros
Find macro-derived columns the YML contract does not list.
- 05Research the data
Driving table, cardinalities, contract, sibling patterns, categorical values.
Cardinality research pinned drift_labs at 185,216 clean one-per-enrollment rows and reconciled staging against the raw landing table; base Claude Code reported the rollup's 89,944 and never consulted raw.
- 06Write the technical spec
A written plan of sources, joins, filters, and grain per model.
- 07Write and build models
SQL from the spec, with column mapping checked before each model.
- 08Verify and fix
Two read-only verifiers cross-check structure and values until every check passes.
SignalPilot
claude code + signalpilot governed stack
reported 185,216 · matches truth
Verified end-to-end. The raw source reconciles exactly with staging, so the number is trustworthy.
Answer: 185,216 enrollments for drift_labs
That's the count of enrollment records (one row per enrollment_id), verified from the raw landing layer (raw.client_blob) straight through to staging — both agree at 185,216 with zero fan-out and zero null keys.
Grain detail (one definitional fork worth knowing)
An enrollment_id is a single subject↔experiment assignment event. There's a small gap between assignment events and distinct subject-experiment pairings:
| Definition | Count |
|---|---|
Enrollment records (enrollment_id grain) | 185,216 |
| Distinct (subject, experiment) pairs | 181,671 |
| Difference (repeat assignments of same subject into same experiment) | 3,545 |
So ~3,545 enrollments are re-assignments of a subject that was already in that experiment. If your capacity planning cares about assignment rows (storage, event volume), use 185,216. If it cares about unique subjects-in-experiments, use 181,671.
For context, these enrollments span 12 experiments and 148,284 distinct subjects.
How I verified (not trusting any single number)
client_idbehind thedriftsource is literallydrift_labs(rawsource_client = 'drift_labs').raw.client_blob: 185,216 rows → 185,216 distinctenrollment_id, 0 nulls — raw grain is one enrollment per blob.staging.stg_drift__enrollments: 185,216 rows = 185,216 distinctenrollment_id— no dedup needed, staging faithfully mirrors raw.
One caution: dim_drift__subjects.enrollments and the various fct_*/org marts also carry enrollment figures, but those are aggregated/rolled-up columns. The figure above is re-derived at the true grain from source, which is the one to rely on. If you'd instead like the count filtered to external subjects only (excluding is_internal traffic), or per-experiment, say the word.
I'll start by loading the required skills, since this is a warehouse/data question.
This is a capacity/business question about warehouse data. Per project instructions, I'll explore the dbt project and verify grain rather than blindly counting. Let me load the SignalPilot MCP tools and run the initial project scan + KB probes in parallel.
Now let me load the specific tools I need and run the initial exploration.
The schema is clear: this is a multi-tenant A/B experimentation warehouse, one source system per client (drift, ember, fjord, …). "drift_labs" is almost certainly the client_id value behind the drift source. Enrollments = subject↔experiment assignments, and the candidate source is staging.stg_drift__enrollments (185K rows).
Per the grain rule, I won't trust that row count until I've verified it. Let me confirm the client_id and check grain against the raw source.
Confirmed client_id = drift_labs, and the staging table is exactly one row per enrollment_id (185,216 rows = 185,216 distinct enrollment_ids — no fan-out at enrollment grain). But note a definitional fork: there are only 181,671 distinct (subject, experiment) pairs, so ~3,545 enrollment records are repeat assignments of the same subject into the same experiment.
Now, per the grain rule, staging is itself derived from raw.client_blob — I need to confirm staging faithfully reflects raw before reporting. Let me inspect the raw layer.
The raw structure is clear: each raw.client_blob row for drift_labs is one enrollment record — a single assignment object with one enrollment_id, one subject, one experiment. Raw count (185,216) equals staging row count. Let me do the final raw-grain verification to confirm staging didn't drop or duplicate anything.
Base Claude Code
claude code, no additional stack
reported 89,944 · truth 185,216 (51% of rows silently dropped)
drift_labs has 89,944 enrollments in the warehouse (from org.int_all__enrollments).
One small nuance worth flagging: that's 89,944 enrollment rows, each with a unique enrollment_id, but they collapse to 89,067 distinct (experiment, subject) pairs. So ~877 subjects have more than one enrollment row for the same experiment (likely re-assignments/re-enrollments).
- If you want raw enrollment records → 89,944
- If you want unique subject-experiment assignments → 89,067
Want me to dig into what those ~877 duplicates are (e.g. variant changes vs. reshuffles)?
I'll query the warehouse to find enrollment data for drift_labs. Let me first explore the schema.
There's an int_all__enrollments table. Let me inspect its structure and how it identifies clients.
The int_all__enrollments table is keyed exactly on subject-experiment assignments. Let me do a quick sanity check that rows aren't duplicated.