Skip to main content

​
Agent Orchestration Loop

SignalPilot’s agent loop is the core engine that transforms single questions into comprehensive data investigations. Unlike single-shot AI completions, this loop maintains context, executes multi-step plans, and keeps analysts in control throughout the process.

​
How the Loop Works

The agent orchestration loop follows a 7-step process that repeats until the investigation is complete: 
┌─────────────────────────────────────────────────────────────────┐
│                    AGENT ORCHESTRATION LOOP                      │
├─────────────────────────────────────────────────────────────────┤
│  1. CONTEXT RESOLUTION (Parallel)                                │
│     └─ Fetch schemas, lineage, Slack threads, query history     │
│                                                                  │
│  2. SYSTEM PROMPT CONSTRUCTION                                   │
│     └─ Build prompt with full organizational context             │
│                                                                  │
│  3. PLAN GENERATION                                              │
│     └─ AI proposes multi-step investigation approach             │
│                                                                  │
│  4. 👤 ANALYST APPROVAL CHECKPOINT                               │
│     └─ Review, modify, or reject before execution                │
│                                                                  │
│  5. TOOL EXECUTION (Parallel)                                    │
│     └─ Run queries, update cells, generate plots                 │
│                                                                  │
│  6. COMPLETION CHECK                                             │
│     └─ Evaluate if investigation goals are met                   │
│                                                                  │
│  7. MEMORY PERSISTENCE                                           │
│     └─ Save findings and patterns for future investigations      │
│                                                                  │
│  ↺ Loop continues until complete or analyst stops                │
└─────────────────────────────────────────────────────────────────┘

​
Step-by-Step Breakdown

​
Step 1: Context Resolution (Parallel)

When you ask a question, SignalPilot immediately fetches relevant context from multiple sources in parallel:

Internal Context Sources

  • Kernel State: Current variables, dataframes, and their schemas
  • Database Schemas: Table structures, column types, relationships
  • Query History: Recent queries and their results
  • Notebook Content: Code cells, markdown, and outputs
  • Local Files: CSVs, configs, and related notebooks
  • dbt: Model lineage, documentation, test results
  • Slack: Recent discussions about relevant data or metrics
  • Jira: Related tickets, deployment history
  • Notion/GDocs: Design docs, runbooks, data dictionaries
Context resolution happens in parallel to minimize latency. A typical resolution takes 1-3 seconds regardless of how many sources are queried.

​
Step 2: System Prompt Construction

SignalPilot builds a comprehensive system prompt that includes:
  1. Base Instructions: How to investigate data problems
  2. Available Tools: What actions the agent can take
  3. Resolved Context: All fetched organizational knowledge
  4. Memory Recall: Relevant findings from past investigations
  5. Active Rules: Team-specific constraints and standards
The system prompt is dynamically constructed based on the question type. A revenue question includes financial schemas and business logic; an ML question includes model metadata and feature definitions.

​
Step 3: Plan Generation

The AI analyzes your question and generates a structured investigation plan: 
## Investigation Plan: "Why did conversion drop 8% last week?"

### Phase 1: Data Discovery
- [ ] Query conversion_events table for last 14 days
- [ ] Compare week-over-week conversion rates by segment

### Phase 2: Hypothesis Testing
- [ ] Check for A/B tests deployed last week
- [ ] Analyze traffic source distribution changes
- [ ] Look for tracking pixel issues

### Phase 3: Root Cause Analysis
- [ ] Deep dive into identified anomaly
- [ ] Generate before/after comparison

### Phase 4: Reporting
- [ ] Create visualization of findings
- [ ] Document root cause and recommendation
Plans are structured with phases to break complex investigations into manageable steps. Each phase has clear objectives that can be reviewed and modified.

​
Step 4: Analyst Approval Checkpoint

This is what makes SignalPilot “analyst-in-the-loop” rather than fully autonomous. Before executing any plan, SignalPilot presents it for your approval:
  • ✅ Approve: Execute the plan as proposed
  • ✏️ Modify: Adjust steps, add constraints, change approach
  • ❌ Reject: Provide new direction or ask a different question
The approval checkpoint is mandatory in Agent Mode. This ensures you maintain control over what queries run, what data is accessed, and what actions are taken.

​
Step 5: Tool Execution (Parallel)

Once approved, SignalPilot executes plan steps using available tools:
ToolPurposeExample
execute_codeRun Python/SQL in notebook cellsQuery database, transform data
write_cellCreate or update notebook cellsAdd visualizations, documentation
read_schemaIntrospect database structureDiscover tables and relationships
search_contextQuery MCP sourcesFind Slack discussions, Jira tickets
create_plotGenerate visualizationsCharts, graphs, dashboards
Independent operations execute in parallel. For example, querying two different tables runs simultaneously, reducing investigation time.

​
Step 6: Completion Check

After each execution phase, SignalPilot evaluates:
  1. Goal Achievement: Did we answer the original question?
  2. Data Quality: Are results statistically significant?
  3. Completeness: Are there unexplored hypotheses?
  4. Confidence Level: How certain is the conclusion?
If the investigation isn’t complete, the loop returns to Step 3 to generate the next phase plan.

​
Step 7: Memory Persistence

When the investigation completes, SignalPilot saves:
  • Findings: What was discovered and concluded
  • Validated Assumptions: Business logic that was confirmed
  • Data Quirks: Anomalies or gotchas discovered
  • Solution Patterns: Approaches that worked for this type of question

Learn More: Multi-Session Memory

See how memory makes future investigations faster and more accurate

​
Why “Long-Running” Matters

Traditional AI interactions are single-shot: you ask, AI responds, done. This fails for data investigations because:
Single-Shot LimitationsLong-Running Solution
Loses context between queriesMaintains full investigation state
Can’t adapt based on findingsAdjusts approach as data reveals patterns
No memory of what was triedTracks hypotheses tested and ruled out
Manual context managementAutomatic context orchestration
Real example: When investigating a conversion drop, you might discover an A/B test was running. A single-shot AI would need you to manually provide this context. SignalPilot’s long-running loop automatically incorporates this finding and adjusts the investigation plan.

​
Performance Characteristics

MetricTypical ValueNotes
Context resolution1-3 secondsParallel fetching from all sources
Plan generation2-5 secondsDepends on question complexity
Tool executionVariesDatabase queries, code execution
Completion check<1 secondLightweight evaluation
Full investigation2-10 minutesComplex questions with multiple phases

​
Best Practices

1

Ask Clear Questions

Specific questions lead to focused investigation plans. “Why did MRR drop?” is better than “Something seems wrong with revenue.”
2

Review Plans Carefully

Take time at the approval checkpoint to ensure the plan addresses your actual question. Modify if needed.
3

Let the Loop Complete

Resist the urge to interrupt mid-investigation. The loop is designed to reach conclusions, not just surface data.
4

Provide Feedback

If a plan doesn’t work, tell SignalPilot why. This improves future investigations through memory.

Context Aggregation

How MCP enables multi-source context resolution

Multi-Session Memory

What gets stored and how it improves over time

Hooks System

Custom constraints and validations

How It Works

Overview of all 4 foundational systems