Skip to main content

​
Prerequisites

  • Databricks workspace with Unity Catalog enabled
  • Metastore created and attached to your workspace
  • Appropriate admin privileges to create service principals and grant permissions

​
Integration Methods

​
Option 1: Direct Read Access via Service Principal

This method allows Aampe to directly access your Unity Catalog tables using a dedicated service principal with OAuth authentication.

​
Required Permissions

The Aampe service principal needs the following Unity Catalog privileges:
  • USE CATALOG - Access to view metadata in the catalog
  • USE SCHEMA - Access to view metadata in schemas
  • SELECT - Read access to tables and views
  • BROWSE - Optional: View object metadata in Catalog Explorer

​
Step-by-step Procedure

  1. Create the Service Principal
Option A: Databricks-managed Service Principal
  1. Go to your Databricks account console
  2. Navigate to User management > Service principals
  3. Click Add service principal
  4. Provide a name (e.g., aampe-integration-sp)
  5. Click Add
Option B: Azure Entra ID Service Principal (Azure Databricks only)
  1. Create an app registration in Azure Portal
  2. Note the Application (client) ID
  3. Add the service principal to your Databricks account
  4. Generate OAuth Credentials
  5. In the service principal’s details page, go to the Secrets tab
  6. Under OAuth secrets, click Generate secret
  7. Set the secret’s lifetime (maximum 730 days)
  8. Copy and securely store:
    • Client ID (same as Application ID)
    • OAuth Secret (shown only once)
  9. Click Done
  10. Grant Unity Catalog Permissions
Run the following SQL commands in a Databricks notebook or SQL editor: 
-- Grant catalog-level permissions
GRANT USE CATALOG ON CATALOG <catalog_name> TO <service_principal_id>;

-- Grant schema-level permissions
GRANT USE SCHEMA ON SCHEMA <catalog_name>.<schema_name> TO <service_principal_id>;

-- Grant table-level read permissions
GRANT SELECT ON TABLE <catalog_name>.<schema_name>.<table_name> TO <service_principal_id>;

-- Alternatively, grant SELECT on all tables in a schema
GRANT SELECT ON SCHEMA <catalog_name>.<schema_name> TO <service_principal_id>;
Replace:
  • <catalog_name>: Your Unity Catalog name
  • <schema_name>: Your schema/database name
  • <table_name>: Specific table names (or use schema-level grant)
  • <service_principal_id>: The Application ID of your service principal
  1. Provide Connection Details to Aampe
Share the following information with the Aampe team:
  • Workspace URL: https://<workspace-name>.cloud.databricks.com
  • Client ID: Your service principal’s Application ID
  • OAuth Secret: The generated secret (transfer securely)
  • Catalog Name: Your Unity Catalog name
  • Schema Names: List of schemas containing relevant tables
  • Table Names: Specific tables to sync (if not syncing entire schemas)
  • Workspace ID: Your Databricks workspace ID
  • Account ID: Your Databricks account ID (for account-level access)

​
Option 2: Push to Cloud Storage

For large-scale data transfers, you can export data to cloud storage that Aampe can access.

​
AWS S3 Setup

  1. Create External Location 
    -- Create storage credential
CREATE STORAGE CREDENTIAL IF NOT EXISTS aampe_storage_cred
WITH (
  AWS_IAM_ROLE = 'arn:aws:iam::<account-id>:role/<role-name>'
);

-- Create external location
CREATE EXTERNAL LOCATION IF NOT EXISTS aampe_export_location
URL 's3://<bucket-name>/aampe-export/'
WITH (STORAGE CREDENTIAL aampe_storage_cred);
  2. Grant Permissions 
    -- Grant write access to the service principal
GRANT CREATE EXTERNAL TABLE ON EXTERNAL LOCATION aampe_export_location 
TO <service_principal_id>;

GRANT WRITE FILES ON EXTERNAL LOCATION aampe_export_location 
TO <service_principal_id>;
  3. Set up Automated Export
Create a Databricks Job or Delta Live Tables pipeline to export data

​
Data Models

Ensure your exported data conforms to the Aampe Data Models specification.