Skip to content

feat(backend): add Databricks CLI auth for local dev#49

Open
sambryfczynski wants to merge 6 commits into
databrickslabs:developfrom
sambryfczynski:feat/auth-localcli
Open

feat(backend): add Databricks CLI auth for local dev#49
sambryfczynski wants to merge 6 commits into
databrickslabs:developfrom
sambryfczynski:feat/auth-localcli

Conversation

@sambryfczynski

Copy link
Copy Markdown

What

Adds a third auth mode to DatabricksAuth for local development:
Databricks CLI profile (~/.databrickscfg populated by databricks auth login).
Selected via DATABRICKS_CONFIG_PROFILE or implicitly via the default profile.

Resolution order is unchanged for the existing modes:

  1. App mode (DATABRICKS_APP_PORT) — M2M OAuth (unchanged)
  2. DATABRICKS_TOKEN set — PAT (unchanged)
  3. DATABRICKS_CONFIG_PROFILE set — named CLI profile (new)
  4. ~/.databrickscfg default profile (new)
  5. None — same failure path as before

Why

Some Databricks workspace policies block PAT generation, which prevented
local development entirely. CLI auth uses the SDK's
databricks.sdk.core.Config to read profiles — the credential is short-lived
and refreshes transparently, no token to copy-paste into .env.

How to test

  1. Empty DATABRICKS_TOKEN in .env and run databricks auth login.
  2. uv run python run.py — startup log should read
    DatabricksAuth init — host=..., mode=cli, profile=default, warehouse=....
  3. GET /health reports Databricks CLI profile 'default' configured.
  4. Browse a Unity Catalog table to exercise the SQL warehouse path.
  5. Set DATABRICKS_CONFIG_PROFILE=<name> to use a non-default profile;
    set it to a bogus value to confirm the WARNING surfaces.
  6. Existing PAT and Apps deployments are unaffected — covered by the
    regression tests in tests/units/auth/test_auth.py.

Tests

uv run pytest tests/units/ -q\ → 2110 passed in ~15s.

@sambryfczynski sambryfczynski requested a review from a team as a code owner June 10, 2026 18:52
@CLAassistant

CLAassistant commented Jun 10, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@benoitcayladbx benoitcayladbx changed the base branch from master to develop June 11, 2026 06:11
@benoitcayladbx

Copy link
Copy Markdown
Collaborator

Thanks for this — the CLI auth path is a clean addition and the precedence ordering (app → PAT → CLI) is well tested. A few things I'd like addressed before merge:

1. (blocking) credentials_provider contract for the SQL connector
The lambda returns the uncalled bound method, and the only test asserts identity rather than driving a real connection. I'm not confident the SQL / CloudFetch path actually works under CLI auth — the only validation today is the manual "browse a UC table" step. Could you add a test that drives sql.connect(**params) (connector mocked is fine) through the credentials_provider, or confirm the lambda matches the databricks-sql-connector contract?

2. (should-fix) thread-safety of the CloudFetch recursion guard
_resolving_cloud_fetch is a bare class-level flag mutated without a lock. Under concurrent DatabricksAuth() construction (FastAPI is multi-threaded), one thread can short-circuit on another's flag, or a finally can clear it
mid-probe. A threading.Lock or contextvars would make this safe.

3. (nit) scope
The recursion-guard fix is unrelated to the auth feature — would be cleaner as its own commit/PR for a reviewable history, but not a blocker.

Everything else looks good: no secret leakage in logs, no token persisted, CLI mode is correctly unreachable in App mode, and the health check fails eagerly. 👍

@sambryfczynski

Copy link
Copy Markdown
Author

Thanks for the feedback! I'll try to address over the next week or so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants