Skip to content

OSAC-4187: proxy CSI attach/detach directly to vendor controllers (0.2 temporary) - #393

Merged
omer-vishlitzky merged 1 commit into
osac-project:mainfrom
rgolangh:feat/OSAC-4187
Aug 20, 2026
Merged

OSAC-4187: proxy CSI attach/detach directly to vendor controllers (0.2 temporary)#393
omer-vishlitzky merged 1 commit into
osac-project:mainfrom
rgolangh:feat/OSAC-4187

Conversation

@rgolangh

Copy link
Copy Markdown
Contributor

Summary

  • Implement ControllerPublishVolume/ControllerUnpublishVolume in osac-csi-driver by proxying directly to the vendor CSI controller for the volume's backend, mirroring the existing node-plugin vendor-socket proxy. Routing key is osac.backend (the StorageBackend name resolved from the volume's StorageTier); the fulfillment volume id is translated to the vendor-side id and the vendor's PublishContext is returned to the CO unchanged.
  • Add a none sentinel endpoint so node-local backends (lvms/topolvm, attachRequired=false, no network CSI controller) make attach/detach a no-op instead of dialing a vendor. AlreadyExists/NotFound/Unimplemented vendor responses are treated as success.
  • Remove the unused ControlPlaneClient/ControlPlaneStub indirection (never backed by a real fulfillment attach API); add the --vendor-controllers flag and wire controller.vendorControllers into the csi-driver chart with a StorageBackend-name-keyed default (local=none,vast=…,pure=…,ontap=…:50051).

Temporary for milestone 0.2 — the controller talks to the vendor CSI controller directly instead of going through a fulfillment control-plane attach API. Expected to be reworked in 0.3 (marked provisional in code).

Jira

OSAC-4187

Test plan

  • New/updated unit tests in pkg/driver/controller_test.go exercise publish/unpublish against an in-process fake vendor CSI controller: vendor-id translation, PublishContext forwarding, context-vs-GetVolume resolution, unknown-backend NotFound, AlreadyExists/NotFound/Unimplemented idempotency, and the none sentinel no-op.
  • CSI sanity suite (test/sanity/) green with the new NewDriver signature.
  • Unit tests pass (make test, pkg/driver 79.9% coverage)
  • Lint/format/build pass (make fmt, make lint 0 issues, make build)
  • helm lint charts/csi-driver clean; --vendor-controllers renders on the controller container

Notes for reviewers

  • Chart change included (charts/csi-driver/values.yaml default + controller-deployment.yaml flag) — review-sensitive per AGENTS.md. The default endpoints target the osac-csi-backends Services on gRPC port 50051; keys are the expected StorageBackend naming convention.
  • Two ADVISORY security notes (non-blocking, both flagged for the 0.3 rework): publish trusts osac.backend/osac.volume-id volume-context over authoritative GetVolume (matches node-plugin trust model); secrets are forwarded to vendor controllers over the pre-existing plaintext in-cluster gRPC proxy.

This PR description was drafted with AI assistance (create-pr v0.1.3). Review for accuracy

@openshift-ci-robot

openshift-ci-robot commented Aug 19, 2026

Copy link
Copy Markdown

@rgolangh: This pull request references OSAC-4187 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.1.0" version, but no target version was set.

Details

In response to this:

Summary

  • Implement ControllerPublishVolume/ControllerUnpublishVolume in osac-csi-driver by proxying directly to the vendor CSI controller for the volume's backend, mirroring the existing node-plugin vendor-socket proxy. Routing key is osac.backend (the StorageBackend name resolved from the volume's StorageTier); the fulfillment volume id is translated to the vendor-side id and the vendor's PublishContext is returned to the CO unchanged.
  • Add a none sentinel endpoint so node-local backends (lvms/topolvm, attachRequired=false, no network CSI controller) make attach/detach a no-op instead of dialing a vendor. AlreadyExists/NotFound/Unimplemented vendor responses are treated as success.
  • Remove the unused ControlPlaneClient/ControlPlaneStub indirection (never backed by a real fulfillment attach API); add the --vendor-controllers flag and wire controller.vendorControllers into the csi-driver chart with a StorageBackend-name-keyed default (local=none,vast=…,pure=…,ontap=…:50051).

Temporary for milestone 0.2 — the controller talks to the vendor CSI controller directly instead of going through a fulfillment control-plane attach API. Expected to be reworked in 0.3 (marked provisional in code).

Jira

OSAC-4187

Test plan

  • New/updated unit tests in pkg/driver/controller_test.go exercise publish/unpublish against an in-process fake vendor CSI controller: vendor-id translation, PublishContext forwarding, context-vs-GetVolume resolution, unknown-backend NotFound, AlreadyExists/NotFound/Unimplemented idempotency, and the none sentinel no-op.
  • CSI sanity suite (test/sanity/) green with the new NewDriver signature.
  • Unit tests pass (make test, pkg/driver 79.9% coverage)
  • Lint/format/build pass (make fmt, make lint 0 issues, make build)
  • helm lint charts/csi-driver clean; --vendor-controllers renders on the controller container

Notes for reviewers

  • Chart change included (charts/csi-driver/values.yaml default + controller-deployment.yaml flag) — review-sensitive per AGENTS.md. The default endpoints target the osac-csi-backends Services on gRPC port 50051; keys are the expected StorageBackend naming convention.
  • Two ADVISORY security notes (non-blocking, both flagged for the 0.3 rework): publish trusts osac.backend/osac.volume-id volume-context over authoritative GetVolume (matches node-plugin trust model); secrets are forwarded to vendor controllers over the pre-existing plaintext in-cluster gRPC proxy.

This PR description was drafted with AI assistance (create-pr v0.1.3). Review for accuracy

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
openshift-ci Bot requested review from avishayt and zszabo-rh August 19, 2026 17:08
@coderabbitai

coderabbitai Bot commented Aug 19, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 53d63d30-e974-44c7-a377-ebfac3dabd94


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 19, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:09 PM UTC · Completed 5:25 PM UTC

Commit: 86c1e44 · View workflow run →

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 19, 2026

Copy link
Copy Markdown

Review

Findings

High

  • [logic-error] osac-csi-driver/cmd/osac-csi-driver/main.go:87 — In stub mode (no --fulfillment-endpoint), VolumeStub is created with backend name "default-backend", but the chart's default vendorControllers value maps local=none,vast=...,pure=...,ontap=... with no entry for "default-backend". When the CSI attacher calls ControllerPublishVolume, resolveVendorController("default-backend") returns codes.NotFound, breaking attach/detach in stub/dev mode. The AGENTS.md update claims the stub reports the local backend, but the code contradicts this.
    Remediation: Change fulfillment.NewVolumeStub("default-backend", "nfs") to fulfillment.NewVolumeStub("local", "nfs").

Medium

  • [test-inadequate] osac-csi-driver/test/sanity/sanity_test.go:51 — Neither the sanity test (which uses backendName="fake") nor unit tests (which use "test-backend") exercise the real stub-mode path where the stub backend name must match the chart's vendorControllers default. The backend name mismatch bug is invisible to the test suite.
    Remediation: Add a test that constructs the driver as main.go does in stub mode and verifies ControllerPublishVolume succeeds.

  • [stale-doc] osac-csi-driver/AGENTS.md — The Stub Mode section claims the driver "reports the local backend so attach/detach no-op against the chart's default local=none mapping," but main.go still passes "default-backend" to NewVolumeStub.
    Remediation: Fix the code (change to "local") or update the documentation to match reality.

  • [stale-doc] osac-csi-driver/README.md:44 — The Configuration table lists --vendor-sockets but not the new --vendor-controllers flag.
    Remediation: Add a row for --vendor-controllers.

  • [stale-doc] osac-csi-driver/charts/csi-backends/README.md:99 — States "the OSAC CSI meta-driver's controller plugin never talks to vendor drivers directly," which is now incorrect — the controller proxies ControllerPublish/UnpublishVolume to vendor CSI controllers.
    Remediation: Update the paragraph to reflect the 0.2 temporary direct proxying.

Low

  • [secrets-in-transit] osac-csi-driver/pkg/driver/controller.go:785req.GetSecrets() is forwarded to vendor CSI controllers over plaintext gRPC (proxy.Manager uses insecure.NewCredentials()). This extends the existing node-side pattern to controller-side attach/detach. The author flagged this as ADVISORY; tracked for 0.3 rework.

  • [trust-boundary-bypass] osac-csi-driver/pkg/driver/controller.go:866resolvePublishTarget trusts volume-context keys (osac.backend, osac.volume-id) without cross-checking the fulfillment service when both values are present. In practice, the CO/kubelet faithfully forwards context per the CSI spec, and a compromised CO is already a cluster-admin compromise. The author flagged this as ADVISORY; tracked for 0.3 rework.

  • [naming-convention] osac-csi-driver/pkg/driver/controller.go:340resolveVendorController lacks a doc comment, breaking the pattern where every private helper in this file and node.go has one.

  • [volume-context-forwarding] osac-csi-driver/pkg/driver/controller.go:786 — The full VolumeContext (including OSAC-internal keys osac.backend, osac.volume-id, osac.protocol) is forwarded to the vendor CSI controller. Consider stripping OSAC-internal keys before forwarding.


Labels: PR modifies CSI storage driver component


Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run

Review

Findings

High

  • [stale-doc] osac-csi-driver/charts/csi-backends/README.md:99 — The README states "Nothing calls them yet: the OSAC CSI meta-driver's controller plugin never talks to vendor drivers directly." This PR introduces direct ControllerPublish/UnpublishVolume proxying to the vendor CSI controllers exposed by this chart, making the statement factually wrong. Operators relying on this README could incorrectly skip deploying the csi-backends chart.
    Remediation: Update the "Provisioning volumes through a vendor controller" section to reflect that attach/detach is now proxied to vendor CSI controller Services (OSAC-4187, 0.2 temporary). Volume creation still goes through the fulfillment-service.

Medium

  • [technical-accuracy] osac-csi-driver/AGENTS.md:114 — The Stub Mode documentation says the VolumeStub "reports the local backend," but main.go creates it with NewVolumeStub("default-backend", "nfs"). Volumes in stub mode get osac.backend="default-backend", which has no entry in the chart's default vendorControllers value (local=none,...). ControllerPublishVolume will fail with codes.NotFound in dev mode.
    Remediation: Change main.go to fulfillment.NewVolumeStub("local", "nfs") to match the chart default.

  • [missing-doc] osac-csi-driver/README.md:44 — The Configuration table lists --vendor-sockets but does not include the new --vendor-controllers flag added by this PR.
    Remediation: Add a --vendor-controllers row to the Configuration table.

Low

  • [plaintext-grpc] osac-csi-driver/pkg/proxy/proxy.go:49 — The proxy.Manager uses insecure.NewCredentials() for gRPC connections, pre-existing but now extended to TCP vendor controller endpoints. Acknowledged in PR as a known gap for 0.3.
  • [secrets-forwarding] osac-csi-driver/pkg/driver/controller.go:785 — CSI secrets forwarded unchanged to vendor controllers via req.GetSecrets(). Standard CSI proxy pattern, but combined with plaintext transport.
  • [volume-context-trust] osac-csi-driver/pkg/driver/controller.go:865resolvePublishTarget trusts osac.backend and osac.volume-id from volume context without fulfillment-service verification. Mirrors the node-plugin trust model; hardening planned for 0.3.
  • [edge-case] osac-csi-driver/pkg/driver/controller.go:786VolumeContext forwarded to vendor includes OSAC-internal keys (osac.backend, osac.volume-id, osac.protocol) that are meaningless to vendors.
  • [fail-open] osac-csi-driver/pkg/driver/controller.go:220 — The "none" sentinel makes publish/unpublish a no-op. Only exploitable if an attacker controls the osac.backend value AND a backend is mapped to "none".
  • [endpoint-injection] osac-csi-driver/cmd/osac-csi-driver/main.go:436--vendor-controllers accepts arbitrary endpoints with no validation. Requires cluster-admin privileges to modify.
  • [naming-convention] osac-csi-driver/pkg/driver/controller.go:906resolveVendorController returns a string endpoint; resolveVendorControllerEndpoint would be clearer, though it parallels node.go's resolveVendorSocket.
  • [architectural-coherence] osac-csi-driver/charts/csi-driver/values.yaml:13vendorControllers ships with a non-empty default unlike vendorSockets. The inline comment explains unused entries are harmless, but the asymmetry could confuse chart consumers.
  • [error-handling-idiom] osac-csi-driver/pkg/driver/controller.go:898 — Error message includes backend name while node.go counterpart does not. The controller version is arguably better for debugging.

Next steps:

  • /fs-fix — agent addresses review findings automatically
  • /fs-fix <your instruction> — agent fixes with your specific guidance
  • Push commits directly — review re-runs automatically on push
  • /fs-fix-stop — disable automatic fix runs for this PR
Previous run (2)

Review

Findings

Medium

  • [logic-error] osac-csi-driver/pkg/driver/controller.go — In ControllerUnpublishVolume, when GetVolume returns codes.NotFound, the code treats unpublish as a no-op and returns immediately without forwarding to the vendor controller. Per the CSI spec, ControllerUnpublishVolume can be called after the volume is deleted (the normal Kubernetes detach flow calls unpublish before delete, but crash recovery or manual deletion can reverse this order). If the OSAC volume record is deleted before the volume is detached, the vendor-side detach is skipped, potentially leaving the volume attached on the vendor side. This is an accepted limitation of the 0.2 temporary approach — without the volume record, the controller cannot determine which backend or vendor volume ID to forward the request to.
    Remediation: Consider storing the backend and vendor volume ID in the PublishContext returned from ControllerPublishVolume, so ControllerUnpublishVolume can extract them from the request even when the OSAC volume record is gone.

  • [data-exposure] osac-csi-driver/pkg/driver/controller.go — CSI secrets from the CO's ControllerPublish/UnpublishVolumeRequest are forwarded unmodified to vendor controllers via the plaintext gRPC proxy (proxy.Manager uses insecure.NewCredentials()). The controller-to-vendor-controller path crosses Kubernetes namespaces (osac-csi → osac-csi-backends) over plaintext TCP, unlike the node plugin's local unix sockets. No NetworkPolicy restricts access to the vendor controller endpoints. The PR body explicitly flags both the secrets forwarding and the plaintext transport as advisory, deferred to the 0.3 rework.
    Remediation: In 0.3, enable mTLS on gRPC connections between the OSAC controller and vendor CSI controllers. In the interim, consider adding NetworkPolicy resources to the csi-backends chart to restrict ingress on vendor controller gRPC ports to the OSAC controller's namespace.

Low

  • [auth-bypass] osac-csi-driver/pkg/driver/controller.goresolvePublishTarget trusts osac.backend and osac.volume-id from volume context to route requests without consulting the fulfillment service when both keys are present. This matches the existing node-plugin trust model — volume context originates from CreateVolume (not user-fabricated), and exploiting this requires a compromised CO (cluster-admin level). Flagged for the 0.3 rework per PR body.

  • [test-inadequate] osac-csi-driver/pkg/driver/controller_test.goresolvePublishTarget has three distinct code paths: (1) both context keys present — short-circuit return, (2) fallback to GetVolume with partial merge (one key present, one missing), (3) GetVolume error. Path (1) is indirectly tested via TestControllerPublishVolume_Success and path (3) via TestControllerPublishVolume_VolumeNotFound. Path (2) — partial volume context — has no test coverage.
    Remediation: Add unit tests for partial volume context fallback (e.g., osac.backend present but osac.volume-id missing, and vice versa).

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Aug 19, 2026
@akshaynadkarni
akshaynadkarni self-requested a review August 19, 2026 23:19
if err != nil {
if st, ok := status.FromError(err); ok && st.Code() == codes.AlreadyExists {
klog.Infof("Volume %s already published to node %s", req.GetVolumeId(), req.GetNodeId())
return &csi.ControllerPublishVolumeResponse{}, nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AlreadyExists returns an empty response here, so we drop the vendor's PublishContext. The attacher persists that on the VolumeAttachment and hands it to NodeStageVolume, so if this branch fires on a re-publish (e.g. controller restart) we overwrite a good context with an empty one and node staging loses the device/mount attrs.

A compliant vendor returns OK with the same context on re-publish rather than AlreadyExists, so the normal path is fine, but that's not guaranteed across Trident/VAST/Pure. Fine to leave for 0.2 with a comment; worth fixing in the 0.3 rework.

@rgolangh

Copy link
Copy Markdown
Contributor Author

waiting for #405 to be merged, then rebase and push

…2 temporary)

Milestone-0.2 temporary arrangement (reworked in 0.3): the CSI controller proxies ControllerPublish/UnpublishVolume directly to the vendor CSI controller selected by the 'osac.backend' volume-context key (the StorageBackend name resolved from the volume's StorageTier). The sentinel 'none' marks node-local backends that need no controller-side attach, making publish/unpublish a no-op.

- controller.go: route attach/detach to the vendor controller via the proxy manager, keyed by osac.backend; drop the ControlPlane attach path.

- driver.go/main.go: NewDriver now takes vendorControllers; add the --vendor-controllers flag; generalize parseVendorSockets into parseBackendMap; remove the unused ControlPlaneClient/ControlPlaneStub.

- chart: add controller.vendorControllers to values.yaml and render --vendor-controllers in the controller Deployment.

Assisted-by: Claude Code <noreply@anthropic.com>
Signed-off-by: Roy Golan <rgolan@redhat.com>
@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 3:30 PM UTC · Ended 3:46 PM UTC

Commit: 5103986 · View workflow run →

@akshaynadkarni

Copy link
Copy Markdown
Contributor

/lgtm

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot removed the requires-manual-review Review requires human judgment label Aug 20, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 3:30 PM UTC · Completed 3:46 PM UTC

Commit: 5103986 · View workflow run →

@omer-vishlitzky
omer-vishlitzky dismissed fullsend-ai-review[bot]’s stale review August 20, 2026 15:46

Auto-dismissed: only Prow labels gate merging

@omer-vishlitzky
omer-vishlitzky added this pull request to the merge queue Aug 20, 2026
@openshift-ci openshift-ci Bot removed the lgtm label Aug 20, 2026
@openshift-ci

openshift-ci Bot commented Aug 20, 2026

Copy link
Copy Markdown

New changes are detected. LGTM label has been removed.

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

🤖 Review · ⚠️ Cancelled · Started 5:01 PM UTC · Ended 5:04 PM UTC

Commit: 7e6dc43 · View workflow run →

@openshift-ci

openshift-ci Bot commented Aug 20, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rgolangh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fullsend-ai-review

fullsend-ai-review Bot commented Aug 20, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 5:06 PM UTC · Completed 5:22 PM UTC

Commit: 5103986 · View workflow run →

@fullsend-ai-review fullsend-ai-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the review comment for full details.


Note: The following inline comments could not be posted on the diff (GitHub returned 422) and are included here instead:

  • osac-csi-driver/cmd/osac-csi-driver/main.go:87: [high] logic-error

In stub mode, VolumeStub is created with backend name "default-backend", but the chart default vendorControllers has no entry for "default-backend" (only local=none,vast=...,pure=...,ontap=...). ControllerPublishVolume fails with codes.NotFound in stub/dev mode. The AGENTS.md claims the stub reports the "local" backend, but the code contradicts this.

Suggested fix: Change fulfillment.NewVolumeStub("default-backend", "nfs") to fulfillment.NewVolumeStub("local", "nfs") so stub-created volumes route to the "local" backend mapped to the "none" sentinel.

  • osac-csi-driver/test/sanity/sanity_test.go:51: [medium] test-inadequate

Neither the sanity test (backendName="fake") nor unit tests ("test-backend") exercise the real stub-mode path where the stub backend name must match the chart vendorControllers default. The backend name mismatch bug is invisible to the test suite.

Suggested fix: Add a test that constructs the driver as main.go does in stub mode and verifies ControllerPublishVolume succeeds.

  • osac-csi-driver/pkg/driver/controller.go (file-level): Line 785 · [low] secrets-in-transit

CSI secrets (req.GetSecrets()) are forwarded to vendor CSI controllers over plaintext gRPC. Extends the existing node-side pattern to controller-side attach/detach. Author flagged as ADVISORY; tracked for 0.3 rework.

  • osac-csi-driver/pkg/driver/controller.go (file-level): Line 866 · [low] trust-boundary-bypass

resolvePublishTarget trusts volume-context keys without cross-checking the fulfillment service when both values are present. A compromised CO is already cluster-admin; the CSI spec relies on faithful context forwarding. Author flagged as ADVISORY; tracked for 0.3 rework.

  • osac-csi-driver/pkg/driver/controller.go:340: [low] naming-convention

resolveVendorController lacks a doc comment, breaking the pattern where every private helper in this file and node.go has one.

  • osac-csi-driver/pkg/driver/controller.go (file-level): Line 786 · [low] volume-context-forwarding

The full VolumeContext including OSAC-internal keys (osac.backend, osac.volume-id, osac.protocol) is forwarded to the vendor CSI controller. Consider stripping OSAC-internal keys before forwarding.

Suggested fix: Strip OSAC-internal keys from the volume context before forwarding to the vendor.

@omer-vishlitzky
omer-vishlitzky dismissed fullsend-ai-review[bot]’s stale review August 20, 2026 17:23

Auto-dismissed: only Prow labels gate merging

Merged via the queue into osac-project:main with commit 6604965 Aug 20, 2026
143 of 150 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants