[v25.3.x] rpk: add OAUTHBEARER SASL mechanism support

[david-yu]

Summary

Backport of #30169 to v25.3.x.

• Add OAUTHBEARER as a supported SASL mechanism in rpk for OIDC-based authentication
• toSASLConfig and NewFranzClient dispatch on OAUTHBEARER to set up bearer token auth
• toRpadminOptions calls WithOAuthBearerAuthentication for the admin client
• OAUTHBEARER rejected early in remote debug bundle (follow-up tracked in issue)
• Updated -X help text and profile docs to list all four SASL mechanisms
• Unit tests for SASL dispatch paths in adminapi, franz client, and schema registry client

Conflicts resolved from original backport

• profile_doc.go: deleted — file does not exist in v25.3.x; equivalent params.go doc update auto-merged cleanly
• schemaregistry/BUILD: dropped context_test.go from go_test srcs (does not exist in v25.3.x)

Release Notes

Features

• Add OAUTHBEARER SASL mechanism support to rpk, enabling OIDC-based authentication for the Kafka client, admin API, and schema registry. Pass the token via --password (raw value or token: format) with --sasl-mechanism OAUTHBEARER.

🤖 Generated with Claude Code

[vbotbuildovich]

Retry command for Build#83730

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/data_transforms_test.py::DataTransformsLoggingTest.test_log_topic_integrity

[vbotbuildovich]

CI test results

test results on build#83730

test_status	test_class	test_method	test_arguments	test_kind	job_url	passed	reason	test_history
FLAKY(FAIL)	DataTransformsLoggingTest	test_log_topic_integrity	null	integration	https://buildkite.com/redpanda/redpanda/builds/83730#019dd215-2cdd-4cb2-9a52-ae349fd9093e	9/11	Test FAILS after retries.Significant increase in flaky rate(baseline=0.0000, p0=0.0000, reject_threshold=0.0100)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DataTransformsLoggingTest&test_method=test_log_topic_integrity

[r-vasquez]

All looks the same with the original branch except the profile_doc file which doesn't exists in this branch 👍

[david-yu]

End-to-end test results

Tested the rpk binary built from this PR (b4e4d78) against a 25.3 operator + 25.3 Redpanda image using the dual-listener OIDC+mTLS demo at https://github.com/david-yu/redpanda-operator-mtls-oidc-listeners. The Kafka client path (rpk topic list/create/produce/consume) was exercised against both an mTLS listener and a SASL/OAUTHBEARER listener.

Environment

Component	Version
rpk	built from PR HEAD b4e4d781c148115005926aeebdb46c31e26f0d1a (go build ./cmd/rpk)
Redpanda image	docker.redpanda.com/redpandadata/redpanda:v25.3.14
Operator chart	redpanda/operator 25.3.4 (appVersion v25.3.4)
Kubernetes	kind v0.31.0, kindest/node:v1.35.0
OIDC IdP	Dex (in-cluster), aud=redpanda, principal claim $.email → user@example.com
Per-listener SASL	sasl_mechanisms_overrides: [{listener: oidc, sasl_mechanisms: [OAUTHBEARER]}]

Method

1. Cloned this PR branch and built rpk on macOS arm64:

   git clone --branch=manual-backport-30169-v25.3.x https://github.com/redpanda-data/redpanda
   cd redpanda/src/go/rpk && go build -o rpk ./cmd/rpk
   

2. Followed the README in david-yu/redpanda-operator-mtls-oidc-listeners to stand up the cluster (Steps 1–5):
   • kind create cluster --name redpanda-demo --config manifests/kind-config.yaml
   • helm install cert-manager jetstack/cert-manager --set crds.enabled=true …
   • kubectl apply -f manifests/dex.yaml
   • helm install redpanda-operator redpanda/operator --version 25.3.4 --set crds.enabled=true …
   • kubectl apply -f manifests/redpanda-cr.yaml with two small adjustments:
     • pinned image.tag: v25.3.14
     • moved the demo's superusers (incl. CN=mtls-client, user@example.com) into spec.clusterSpec.config.cluster.superusers so the operator doesn't reconcile away the runtime rpk cluster config set from Step 7.
3. Extracted CA + mTLS client cert via cert-manager (Step 6).
4. Obtained an OIDC access token from Dex via the password grant (scripts/get-oidc-token.sh); decoded JWT confirms aud=redpanda, email=user@example.com, iss=http://dex.dex.svc.cluster.local:5556/dex.
5. Drove every listener through the locally built rpk and through the kafka client path that this PR touches (pkg/kafka/client_franz.go for produce/consume/metadata, pkg/adminapi for the admin API token plumbing — admin endpoints aren't exposed in this demo, so admin API was exercised indirectly by the metadata+SASL handshake).

Results

mTLS listener (localhost:30095, mtls_identity)

Operation	Status
rpk topic list	PASS
rpk topic create mtls-test	PASS
rpk topic produce mtls-test	PASS (offset 0)
rpk topic consume mtls-test --num 1	PASS — round-trip OK

OIDC listener (localhost:31094, SASL/OAUTHBEARER)

Invocation:

rpk topic <op> --brokers localhost:31094 \
  -X tls.enabled=true -X tls.insecure_skip_verify=true -X tls.ca=certs/ca.crt \
  -X sasl.mechanism=OAUTHBEARER -X user= -X "pass=token:${OIDC_TOKEN}"

Operation	Status
rpk topic list	PASS — returns _schemas, mtls-test, oidc-test
rpk topic create oidc-test	PASS
rpk topic produce oidc-test	PASS (offset 0)
rpk topic consume oidc-test --num 1	PASS — round-trip OK

Negative / variant cases

Case	Result
OIDC listener + --sasl-mechanism SCRAM-SHA-256	UNSUPPORTED_SASL_MECHANISM: The broker does not support the requested SASL mechanism. ✅ (sasl_mechanisms_overrides fence holds)
OIDC listener + invalid token (pass=token:not-a-real-jwt)	SASL_AUTHENTICATION_FAILED: … security: Invalid credentials ✅
OIDC listener + raw-token form (no token: prefix)	PASS ✅ — both pass=<jwt> and pass=token:<jwt> accepted, matching the updated params.go doc

rpk topic list -X help reflects the new mechanism

sasl.mechanism=SCRAM-SHA-256
  The SASL mechanism to use for authentication. This can be SCRAM-SHA-256,
  SCRAM-SHA-512, PLAIN, or OAUTHBEARER. For OAUTHBEARER, pass the token via
  the pass field (optionally prefixed with "token:"). …

Summary

OAUTHBEARER works end-to-end against a 25.3 operator + Redpanda v25.3.14 cluster. The franz-go client correctly dispatches on the OAUTHBEARER mechanism, the broker accepts the Dex JWT, the principal is extracted from the $.email claim, and per-listener sasl_mechanisms_overrides correctly rejects SCRAM on the OIDC listener. Both pass=<jwt> and pass=token:<jwt> forms are accepted. mTLS on the second listener still works alongside.