sr/context: implement /contexts/{context}/... prefixed handlers#30189
Open
nguyen-andrew wants to merge 10 commits intoredpanda-data:devfrom
Open
sr/context: implement /contexts/{context}/... prefixed handlers#30189nguyen-andrew wants to merge 10 commits intoredpanda-data:devfrom
nguyen-andrew wants to merge 10 commits intoredpanda-data:devfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds Confluent-compatible context-prefixed Schema Registry routes (/contexts/{context}/...) so clients can target non-default contexts by setting a context-prefixed base URL. It does this by introducing request-rewriting helpers to scope subjects/query params and by updating authorization to evaluate ACLs against the context-qualified subject derived from the URL prefix.
Changes:
- Add
/contexts/{context}/...route registrations that rewrite requests and delegate to existing handlers. - Introduce context parsing/normalization and subject scoping utilities, plus unit/integration tests for routing and ACL isolation.
- Extend Schema Registry auth resource handling to support authorization against context-qualified subjects for prefixed routes.
Reviewed changes
Copilot reviewed 17 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/rptest/tests/schema_registry_test.py | Adds end-to-end tests for context-prefixed routing, delete-context normalization, serde client base-URL behavior, and ACL isolation. |
| tests/go/go-kafka-serde/go.mod | Updates Go toolchain/deps for the Go serde test client (used by the new serde acceptance test). |
| tests/go/go-kafka-serde/go.sum | Updates dependency lockfile for the Go serde test client. |
| src/v/utils/variant.h | Adds extend_variant_t helper to extend std::variant types. |
| src/v/utils/tests/variant_test.cc | Compile-only test coverage for extend_variant_t. |
| src/v/utils/tests/BUILD | Registers the new variant_test target. |
| src/v/pandaproxy/schema_registry/test/context_router.cc | Adds unit tests for context normalization and request rewrite helpers. |
| src/v/pandaproxy/schema_registry/test/BUILD | Registers the new context_router_test target. |
| src/v/pandaproxy/schema_registry/service.cc | Registers context-prefixed routes and delegates via rewrite wrappers; updates route resource typing. |
| src/v/pandaproxy/schema_registry/handlers.cc | Normalizes context path parameter when deleting contexts (bug fix). |
| src/v/pandaproxy/schema_registry/errors.h | Adds context_invalid error info helper. |
| src/v/pandaproxy/schema_registry/error.h | Adds context_invalid to the Schema Registry error_code enum. |
| src/v/pandaproxy/schema_registry/error.cc | Maps context_invalid to an HTTP 400 response. |
| src/v/pandaproxy/schema_registry/context_router.h | Implements context normalization and rewrite helpers for subject/path/query scoping. |
| src/v/pandaproxy/schema_registry/authorization.cc | Updates auth resource extraction to support context-prefixed subject authorization. |
| src/v/pandaproxy/schema_registry/auth.h | Introduces context_prefix_subject and route_resource to resolve route-time resources into auth-time resources. |
| src/v/pandaproxy/schema_registry/BUILD | Adds context_router.h and links the variant utility into schema registry. |
| src/v/pandaproxy/api/api-doc/schema_registry.json | Documents the new /contexts/{context}/... endpoints for compatibility. |
nguyen-andrew
force-pushed
the
sr/context-prefix-handlers
branch
from
5eae7af to
866cdb4
Compare
Member
Author
|
Force push to address copilot comment |
Collaborator
Retry command for Build#83227please wait until all jobs are finished before running the slash command |
Collaborator
nguyen-andrew
force-pushed
the
sr/context-prefix-handlers
branch
from
866cdb4 to
1bb9ae6
Compare
Member
Author
|
Force push to change confluent-kafka-go upgrade to v2.3.0. The only thing we need from the upgrade is the URL path fix (confluent-kafka-go#943, landed in v2.1.0). Jumping to v2.14.0 was overkill and pulled in unnecessary transitive deps (Azure SDK, JWT, oauth2). v2.3.0 is the first stable release past the fix. |
nguyen-andrew
force-pushed
the
sr/context-prefix-handlers
branch
from
1bb9ae6 to
f3b2f7a
Compare
Member
Author
|
Force push to fold in a security fix. Snyk flagged 4 transitive vulns in tests/go/go-kafka-serde |
Introduces context_router.h with inline helpers that will be used by
context-prefixed route wrappers:
- normalize_context(): canonicalize a URL path context parameter
by stripping outer ':' delimiters, adding '.' prefix, and
rejecting embedded colons (400 Bad Request via new
context_invalid error code)
- starts_with_context(): detect subjects already qualified with
a context prefix
Aliases like "staging", ":.staging:", and ".staging" all resolve to
the canonical form ".staging".
Includes gtest coverage for both helpers.
Includes gtest coverage for all helpers.
Adds a type-level utility for extending a std::variant with additional alternative types without repeating the original type list. This enables composing variant types where a superset variant needs all alternatives from a base variant plus extras.
Splits the auth resource type into `route_resource` (route registration
time) and `resource` (authorization time). The new
`context_prefix_subject` variant in `route_resource` qualifies the
subject with the {context} path param before the ACL check runs, then
resolves to `context_subject`.
This ensures a user with ACLs on "foo" (default context) cannot access
:.staging:foo via the /contexts/.staging/subjects/foo/... URL.
Registers 15 context-prefixed routes that have a {subject} path
parameter. Each route extracts the context from the URL prefix,
scopes the subject with it via scope_subject_param(), and
delegates to the existing handler.
Covers: subject CRUD, versions, compatibility, config/{subject}, and
mode/{subject} — all via /contexts/{context}/... URLs.
Includes unit tests for scope_subject_param() and ducktape coverage
for all 15 endpoints and ACL isolation.
Registers 4 context-prefixed routes for /schemas/ids/{id} and its
sub-resources (/schema, /versions, /subjects). Adds a
scope_subject_query helper that injects the context as a "subject"
query parameter, scoping schema lookups to the specified context.
Includes unit tests for scope_subject_query and ducktape coverage
for schema lookup and sub-resource queries.
Adds scope_subject_prefix_query(), which injects or prepends the
normalized context into the subjectPrefix query parameter. The
context-prefixed GET /contexts/{context}/subjects route uses this
helper to scope subject listings to the specified context.
Includes unit tests for scope_subject_prefix_query and ducktape
coverage verifying subject isolation across contexts.
Registers context-prefixed routes for context-level config/mode and
schema types:
- GET/PUT/DELETE /contexts/{context}/config
- GET/PUT/DELETE /contexts/{context}/mode
- GET /contexts/{context}/schemas/types (pass-through)
Adds inject_context_as_subject(), which sets the subject path
parameter to a context-only qualified subject (e.g., ":.staging:").
The config and mode wrappers use this to delegate to the existing
config/mode subject handlers. Schema types are global so the context
is accepted for compatibility but ignored.
Includes unit tests for inject_context_as_subject and ducktape
coverage for all operations.
While implementing context-prefixed route handlers, noticed that
DELETE /contexts/{context} was not normalizing the context path
parameter before the default-context check.
Apply normalize_context() to the context path parameter before the
default-context check. This ensures alias forms like "staging",
":.staging:", and ".staging" all resolve to the same canonical context
for deletion.
Includes ducktape coverage cycling through alias forms.
nguyen-andrew
force-pushed
the
sr/context-prefix-handlers
branch
from
f3b2f7a to
179c951
Compare
Member
Author
|
Force push to rebase on latest dev. |
The v2.0.2 Schema Registry client stripped the path component from URLs (confluent-kafka-go#943), preventing context-prefixed URLs like /contexts/.serde from working. Fixed in v2.1.0 via PR redpanda-data#950.
A serde client configured with schema.registry.url pointing to /contexts/.serde performs a full produce/consume round-trip. Verifies schemas are registered in the target context and isolated from the default context. Parametrized across Python, Go, and Java clients to cover all language ecosystems.
nguyen-andrew
force-pushed
the
sr/context-prefix-handlers
branch
from
179c951 to
a921cd1
Compare
Member
Author
|
Force push to bump golang/protobuf from v1.5.3 to v1.5.4 to try to fix issues with the CI docker image. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implement
/contexts/{context}/...prefixed routes for the Schema Registry,allowing clients to target a non-default context by prefixing any endpoint URL
with
/contexts/{context}. This is a Schema Registry compatibilityfeature that lets serde clients point their
schema.registry.urlat acontext-prefixed base URL (e.g.
http://host:8081/contexts/.serde) andtransparently register/lookup schemas in that context.
Each context-prefixed route extracts the
{context}path parameter, rewritesthe request (scoping the subject, query param, or injecting a synthetic subject),
and delegates to the existing handler. Four URL-rewriting strategies cover all
endpoint shapes:
scope_subject_param— qualifies the{subject}path param with the contextscope_subject_query— injects/qualifies the?subjectquery paramscope_subject_prefix_query— injects/prepends context into?subjectPrefixinject_context_as_subject— sets subject to context-only form for config/modeAuthorization uses a new
context_prefix_subjectresource type that qualifiesthe subject with the context before the ACL check, so users cannot bypass
context isolation via the URL prefix.
Also fixes a bug where
DELETE /contexts/{context}was not normalizing thecontext path parameter before the default-context check.
Fixes CORE-15191
Backports Required
Release Notes
Features
/contexts/{context}/...prefixed URLs on allendpoints, allowing serde clients to target a non-default context by
configuring their base URL (e.g.
schema.registry.url=http://host:8081/contexts/.myctx).