Skip to main content

​
Databricks Connection

Connect your Databricks workspace to enable AI-powered analysis across your lakehouse data with full catalog, schema, and table introspection.
What you’ll need: Workspace URL, authentication (access token or service principal), SQL warehouse HTTP path, and catalog/schema names.

​
Quick Setup

1

Get connection details from SQL Warehouse

In Databricks:
  1. Go to SQL Warehouses → Select your warehouse
  2. Click Connection Details tab
  3. Copy Server hostname and HTTP path
2

Generate authentication

Access Token: Settings → Developer → Access tokens → Generate new tokenOr use Service Principal (see Authentication section)
3

Connect in SignalPilot

Database icon → Add Connection → DatabricksEnter URL, auth, HTTP path, catalog → Test Connection
SignalPilot discovers all accessible catalogs, schemas, and tables.

​
Connection Fields

FieldRequiredDescriptionExample
Connection Name✅Friendly labelDatabricks Production
Connection URL✅Workspace URLhttps://adb-123456789012.3.azuredatabricks.net
Authentication✅Access Token or Service PrincipalSee below
HTTP Path✅SQL warehouse path/sql/1.0/warehouses/abc123def456
Catalog✅Unity Catalog namemain, prod
SchemaOptionalDefault schemaanalytics
Credentials are encrypted locally using AES-256.

​
Authentication

For: Individual use, developmentHow to create:
  1. Settings → Developer → Access tokens
  2. Generate new token → Set description and lifetime
  3. Copy token immediately
Paste in: Databricks Access Token field

​
Key Capabilities

Unity Catalog

Three-level namespace: catalog.schema.table

Delta Lake

Query Delta tables with time travel support

Schema Discovery

Auto-introspect catalogs, schemas, tables, columns

Optimized SQL

Generate Databricks SQL with proper syntax

​
Security Best Practices

-- Grant catalog access
GRANT USE CATALOG ON CATALOG main TO `sp-id`;
GRANT USE SCHEMA ON SCHEMA main.analytics TO `sp-id`;
GRANT SELECT ON SCHEMA main.analytics TO `sp-id`;
  • Set expiration when generating tokens
  • Rotate OAuth secrets every 6-12 months
  • Revoke old credentials after rotation

​
Troubleshooting

Solutions:
  • Verify workspace URL format: https://adb-<id>.<region>.azuredatabricks.net
  • Ensure SQL Warehouse is running
  • Check network/firewall settings
Solutions:
  • Generate new access token
  • Verify service principal has catalog permissions
  • Check token hasn’t expired
Solutions:
  • List catalogs: SHOW CATALOGS
  • Use hive_metastore if Unity Catalog isn’t enabled
  • Grant access: GRANT USE CATALOG ON CATALOG main TO user

​
Next Steps

Query Your Lakehouse

Start analyzing Databricks data

dbt + Databricks

Connect dbt for Unity Catalog lineage

Snowflake

Add data warehouse connection

MCP Overview

External context connections