rpk: add OAUTHBEARER SASL mechanism support#30169
Conversation
Allow rpk to authenticate with Kafka brokers using the OAUTHBEARER SASL mechanism. This enables OIDC-based authentication workflows where users pass an OAuth token via --sasl-password (with optional "token:" prefix). The token is used as an OAuth bearer token for Kafka connections, admin API requests, and schema registry access. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
|
Test the oauthBearerToken helper (token: prefix stripping, raw tokens, empty inputs) and GetAuth with OAUTHBEARER profiles (bearer token construction, case-insensitivity, error paths). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Key finding: sasl_mechanisms_overrides works through the CRD bootstrap
config passthrough when using the correct Redpanda list-of-objects format:
- listener: oidc
sasl_mechanisms:
- OAUTHBEARER
The previous map format ({oidc: [OAUTHBEARER]}) was the wrong format
for this Redpanda property. No chart/operator code change needed.
Updated:
- manifests/redpanda-cr.yaml: correct list-of-objects format, removed
the "must set post-deploy via rpk" comment
- README: CRD listener config snippets, rpk OAUTHBEARER tracking PR
(redpanda-data/redpanda#30169), corrected test results showing
overrides work via CRD, removed workaround instructions
- scripts/oidc-test-pod.yaml: cleaned up test script
Validated end-to-end on Kind:
- sasl_mechanisms_overrides correctly applied via CRD bootstrap config
- OIDC token acquisition from Dex: PASS
- SASL/OAUTHBEARER authentication: PASS
- mTLS create/produce/consume: PASS
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sasl.mechanism help in both -X help and -X list was missing PLAIN and OAUTHBEARER. Update to list all four supported mechanisms and document the token: password prefix for OAUTHBEARER. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test that NewFranzClient returns the correct errors for OAUTHBEARER with empty token, token:-only prefix, and unknown SASL mechanisms. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous build failed in ci:pandatriage:parse-results due to a --fetch-pt-analysis argument parsing bug in the CI tool, unrelated to this PR's changes.
|
Not exactly sure why the tests keep failing does not seem related to this PR. |
c-julin
left a comment
There was a problem hiding this comment.
Very nice, got codex to run a review and it found the debug bundle drops the auth error silently. Just a few nits otherwise
|
Expanding on the "debug bundle drops auth silently" note from the review body, plus a couple of items that didn't make it in: Remote debug bundle — details
if p.HasSASLCredentials() {
s := p.KafkaAPI.SASL
opts = append(opts, rpadmin.WithSCRAMAuthentication(s.User, s.Password, s.Mechanism))
}
Options:
Missing schema registry tests
Ordering dependency in
|
Move oauthBearerToken to adminapi.OAuthBearerToken and share it across the kafka and schemaregistry clients (was previously duplicated in all three packages). Fix the "requires a token" error to reference the actual --password flag (not --sasl-password, which does not exist) and the profile field. Use errors.New where no format verbs were used. Drop the duplicate oauthBearerToken test from client_franz_test.go; the require-based version in adminapi/auth_test.go is canonical and gains the case-sensitivity case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@c-julin Thanks for the review! All 5 comments addressed in 7dd8eb0:
|
The remote debug bundle forwards SASL credentials to the broker so rpk running on the broker can authenticate to Kafka. The broker's admin API currently only accepts a SCRAM-shaped payload (username, password, mechanism), and rpk previously gated credential forwarding behind HasSASLCredentials(), which returns false for OAUTHBEARER profiles because they have no username. That meant a remote debug bundle started with an OAUTHBEARER profile would silently go out with no auth and fail in a confusing way once it reached Kafka. Until the broker/rpadmin gain a bearer payload, reject up front with a clear "not yet supported" error pointing users at SCRAM. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add a table test covering OAUTHBEARER in schemaregistry.NewClient: the token-prefix path, raw token, case-insensitive mechanism match, and the two empty-token error paths. The adminapi and kafka clients already had this coverage; the schema registry client did not. Also document the ordering dependency in the switch: the OAUTHBEARER case must match before the HasSASLCredentials() case. OAUTHBEARER profiles have no username, so HasSASLCredentials() returns false today and the ordering is fine, but a future loosening of that predicate or a re-ordering of the cases would route the bearer token into a BasicAuth password. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Point future readers at redpanda#30222 which tracks the broker-side and rpadmin-side work needed to lift the OAUTHBEARER rejection in rpk debug remote-bundle start. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Fixed a few build errors and should now pass tests. Waiting on #30225 to see if it merges to enable the oidc remote debug bundle sub command. |
|
/backport v26.1.x |
|
/backport v25.3.x |
|
Failed to create a backport PR to v26.1.x branch. I tried: |
|
Failed to create a backport PR to v25.3.x branch. I tried: |
|
Will try to backport just this PR back to 26.1.x, 25.3.x, and 25.2.x |
|
/backport v26.1.x |
|
Failed to create a backport PR to v26.1.x branch. I tried: |
Add OAUTHBEARER as a supported SASL mechanism in rpk, alongside the existing SCRAM-SHA-256 and SCRAM-SHA-512 mechanisms. - toSASLConfig and NewFranzClient now dispatch on OAUTHBEARER to set up kgo.SASL with the bearer token from the profile's sasl.password - toRpadminOptions similarly calls WithOAuthBearerAuthentication for the admin client - OAUTHBEARER is rejected early in remote debug bundle (follow-up issue referenced in the guard comment) - Update -X help text and profile docs to list all SASL mechanisms including PLAIN and OAUTHBEARER - Add unit tests for the SASL dispatch paths in adminapi, franz client, and schema registry client; fix $HOME-unset failures in those tests - Regenerate BUILD files for new test files (cherry picked from commits in PR #30169)
|
/backport v25.3.x |
|
/backport v25.2.x |
|
Failed to create a backport PR to v25.2.x branch. I tried: |
|
Failed to create a backport PR to v25.3.x branch. I tried: |
Add OAUTHBEARER as a supported SASL mechanism in rpk, alongside the existing SCRAM-SHA-256 and SCRAM-SHA-512 mechanisms. - toSASLConfig and NewFranzClient now dispatch on OAUTHBEARER to set up kgo.SASL with the bearer token from the profile's sasl.password - toRpadminOptions similarly calls WithOAuthBearerAuthentication for the admin client - OAUTHBEARER is rejected early in remote debug bundle (follow-up issue referenced in the guard comment) - Update -X help text and profile docs to list all SASL mechanisms including PLAIN and OAUTHBEARER - Add unit tests for the SASL dispatch paths in adminapi, franz client, and schema registry client; fix $HOME-unset failures in those tests - Regenerate BUILD files for new test files (cherry picked from commits in PR #30169)
Add OAUTHBEARER as a supported SASL mechanism in rpk, alongside the existing SCRAM-SHA-256 and SCRAM-SHA-512 mechanisms. - toSASLConfig and NewFranzClient now dispatch on OAUTHBEARER to set up kgo.SASL with the bearer token from the profile's sasl.password - toRpadminOptions similarly calls WithOAuthBearerAuthentication for the admin client - OAUTHBEARER is rejected early in remote debug bundle (follow-up issue referenced in the guard comment) - Update -X help text and profile docs to list all SASL mechanisms including PLAIN and OAUTHBEARER - Add unit tests for the SASL dispatch paths in adminapi, franz client, and schema registry client; fix $HOME-unset failures in those tests - Regenerate BUILD files for new test files (cherry picked from commits in PR #30169)
Summary
--password(supports both raw token andtoken:<TOKEN>format)-X help/-X listand profile docs to document all four SASL mechanismsRef: https://docs.redpanda.com/current/manage/security/authentication/#oidc
Ref: https://github.com/david-yu/redpanda-operator-mtls-oidc-listeners
Usage Examples
Obtain an OIDC token
Topic operations
Consumer group operations
Cluster operations
ACL operations
Using a profile (avoids repeating flags)
Test plan
Unit tests (all passing)
oauthBearerTokenhelper — token prefix stripping, raw tokens, empty inputs, case sensitivityGetAuthwith OAUTHBEARER — bearer token construction, case-insensitivity, empty token error, token:-only errorGetAuthregression — SCRAM-SHA-256 still returns BasicAuth, no-SASL still returns NopAuthNewFranzClienterror paths — OAUTHBEARER with empty token, token:-only prefix, unknown mechanismE2E tested locally on Kind + Dex OIDC provider
Tested with a local rpk build against a Kind cluster running Redpanda with Dex as the OIDC provider
(setup).
Redpanda configured with
oidc_discovery_url,oidc_token_audience,oidc_principal_mapping,and an OIDC listener with
sasl_mechanisms_overrides: [OAUTHBEARER]on NodePort 31094.rpk topic list_schemas,mtls-test)rpk topic create oidc-rpk-testrpk topic produce oidc-rpk-testrpk topic consume oidc-rpk-test --num 1rpk topic describe oidc-rpk-testrpk group listrpk cluster inforpk topic delete oidc-rpk-testtoken:prefix)token:prefixBackports Required
Release Notes
Features
--password(raw value ortoken:<TOKEN>format) with--sasl-mechanism OAUTHBEARER.Generated with Claude Code