Databricks Connector

Index Databricks Unity Catalog (or Hive) table metadata in Hymalaia

How it works

The Databricks connector indexes table metadata for a given Unity Catalog (or Hive) database and schema: it reads column types from information_schema and builds a synthetic DDL document per table (including primary/foreign key metadata when available). It does not index row-level table data.

Tables whose metadata has changed (based on last_altered in information_schema.tables) are picked up on incremental runs. Like other connectors, indexing typically runs on a daily schedule.

Setting up

Authorization

Databricks uses a SQL warehouse connection over HTTP, with a personal access token (PAT). Create a credential with:

• Server hostname — the hostname from your warehouse’s JDBC/ODBC connection details (e.g. adb-1234567890.4.azuredatabricks.net), without https://
• HTTP path — the warehouse HTTP Path from the same connection dialog (starts with /sql/1.0/warehouses/...)
• Access token — a Databricks PAT for a user that can run metadata queries on the target catalog/database/schema

Create or rotate tokens under User settings → Developer → Access tokens in the Databricks workspace. The SQL warehouse must be running or auto-resume when the connector runs.

Indexing

1. Open Databricks connector

   In the Hymalaia Admin Panel, open the Databricks connector.

2. Configure authorization

   In Step 1, set up credentials:

   • Select an existing Databricks credential, or click Create New
   • Enter Server hostname, HTTP path, and Access token
3. Save configuration

   Click Create (or equivalent) to save the credential configuration.

4. Continue with selected credential

   Ensure the correct credential is selected, then click Continue.

5. Specify database and schema

   In Step 2, specify:

   • Connector Name — a display name for this connector (e.g. Lakehouse prod)
   • Database — the Databricks catalog.database name as used in SQL (the first level is often your catalog; use the value that matches information_schema for your environment)
   • Schema — the schema name within that database
   • Access Type — whether indexed content is Public or Private in Hymalaia
6. Create connector

   Click Create Connector to start indexing.

The connector will index table DDL for the configured database and schema. Repeat for other databases/schemas if needed.

For connection details and tokens, see Databricks SQL warehouses and personal access tokens.