Benchmarks
SignalPilot is officially the #1 ranked AI agent on the Spider 2.0-DBT leaderboard — resolving 42 of 64 tasks on the world's hardest data-engineering benchmark.
Full task-by-task evaluation results below.
Read the full story: How We Beat JetBrains to #1| Task | Status |
|---|---|
| activity001 | Pass |
| airbnb001 | Pass |
| airport001 | Pass |
| app_reporting001 | Pass |
| app_reporting002 | Pass |
| apple_store001 | Pass |
| asana001 | Pass |
| asset001 | Pass |
| chinook001 | Pass |
| divvy001 | Pass |
| f1002 | Pass |
| f1003 | Pass |
| google_play001 | Pass |
| google_play002 | Pass |
| greenhouse001 | Pass |
| hive001 | Pass |
| hubspot001 | Pass |
| intercom001 | Pass |
| lever001 | Pass |
| marketo001 | Pass |
| maturity001 | Pass |
| mrr001 | Pass |
| mrr002 | Pass |
| pendo001 | Pass |
| playbook001 | Pass |
| qualtrics001 | Pass |
| quickbooks002 | Pass |
| quickbooks003 | Pass |
| recharge002 | Pass |
| reddit001 | Pass |
| retail001 | Pass |
| salesforce001 | Pass |
| shopify_holistic_reporting001 | Pass |
| shopify001 | Pass |
| shopify002 | Pass |
| superstore001 | Pass |
| tickit001 | Pass |
| tpch001 | Pass |
| twilio001 | Pass |
| workday001 | Pass |
| workday002 | Pass |
| zuora001 | Pass |
| analytics_engineering001 | Fail |
| atp_tour001 | Fail |
| f1001 | Fail |
| flicks001 | Fail |
| inzight001 | Fail |
| jira001 | Fail |
| movie_recomm001 | Fail |
| nba001 | Fail |
| netflix001 | Fail |
| playbook002 | Fail |
| provider001 | Fail |
| quickbooks001 | Fail |
| recharge001 | Fail |
| sap001 | Fail |
| scd001 | Fail |
| social_media001 | Fail |
| synthea001 | Fail |
| tickit002 | Fail |
| tpch002 | Fail |
| xero_new001 | Fail |
| xero_new002 | Fail |
| xero001 | Fail |
Run ID: dbt-run9 · Suite: spider2-dbt · Spider2 GitHub