Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions osac-csi-driver/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ osac-csi-driver/
│ │ └── node.go # NodeServer (Stage/Unstage, Publish/Unpublish, vendor proxy)
│ ├── fulfillment/
│ │ ├── volume.go # VolumeClient interface, VolumeInfo, CreateVolumeParams
│ │ ├── controlplane.go # ControlPlaneClient interface (Publish/Unpublish)
│ │ └── stubs.go # In-memory stubs for development (no fulfillment-service needed)
│ └── proxy/
│ └── proxy.go # gRPC connection manager for vendor CSI sockets (lazy, cached)
Expand All @@ -73,41 +72,44 @@ Kubernetes PVC
ControllerServer ──→ fulfillment-service Volume API
(policy check, tier resolution, vendor dispatch)
↓ (CSI ControllerPublishVolume / ControllerUnpublishVolume)
ControllerServer ──→ fulfillment-service ControlPlane API
ControllerServer ──→ proxy Manager ──→ vendor CSI controller
(routed by "osac.backend"; OSAC-4187, 0.2 temporary)

↓ (CSI NodeStageVolume / NodePublishVolume)
NodeServer ──→ proxy Manager ──→ vendor CSI node plugin
(routed by "osac.backend" volume context key)
```

The controller plugin **never talks to vendor CSI drivers directly** — all volume lifecycle and publish/unpublish operations go through the fulfillment-service, which handles policy checks, storage tier resolution, and vendor dispatch. Only the node plugin communicates directly with vendor CSI sockets for mount operations.
The controller plugin delegates **volume lifecycle** (CreateVolume, DeleteVolume) to the fulfillment-service, which handles policy checks, storage tier resolution, and vendor dispatch. For **attach/detach** (ControllerPublish/UnpublishVolume) the controller proxies directly to the vendor CSI controller selected by the `osac.backend` volume-context key — a temporary arrangement for milestone 0.2 (OSAC-4187), expected to be reworked in 0.3. The node plugin communicates directly with vendor CSI sockets for mount operations.

`osac.backend` carries the **StorageBackend name** resolved from the volume's StorageTier, so the controller's `--vendor-controllers` map (and the node's `--vendor-sockets` map) are keyed by StorageBackend name. A backend whose vendor-controller endpoint is the sentinel `none` (e.g. node-local `lvms`/`topolvm`, which sets `attachRequired=false` and exposes no network CSI controller) makes attach/detach a no-op instead of dialing a vendor.

### Key Subsystems

| Package | Purpose |
|---------|---------|
| `pkg/driver/` | CSI gRPC server (Identity, Controller, Node) implementing the meta-driver pattern |
| `pkg/fulfillment/` | Interfaces and stubs for fulfillment-service volume and control plane operations |
| `pkg/fulfillment/` | VolumeClient interface and in-memory stubs for fulfillment-service volume operations |
| `pkg/proxy/` | Lazy gRPC connection manager for vendor CSI sockets (unix + TCP) |

### Volume Context Keys

The controller sets these keys in volume context at creation time, consumed by the node plugin:
The controller sets these keys in volume context at creation time, consumed by the node plugin (and, for attach/detach, by the controller itself):

| Key | Purpose |
|-----|---------|
| `osac.backend` | Routing key — identifies which vendor CSI socket to proxy to |
| `osac.backend` | Routing key (StorageBackend name) — selects the vendor CSI socket (node) and vendor CSI controller (controller attach/detach) to proxy to |
| `osac.volume-id` | Vendor-side volume ID |
| `osac.protocol` | Storage protocol (e.g., `nfs`) |

### Dual-Plugin Topology

- **Controller plugin** (Deployment): `CreateVolume`, `DeleteVolume`, `ControllerPublishVolume`, `ControllerUnpublishVolume` — delegates all operations to the fulfillment-service volume and control plane APIs
- **Controller plugin** (Deployment): `CreateVolume`, `DeleteVolume` via the fulfillment-service volume API; `ControllerPublishVolume`, `ControllerUnpublishVolume` proxied to the vendor CSI controller routed by `osac.backend` (OSAC-4187, 0.2 temporary)
- **Node plugin** (DaemonSet): `NodeStageVolume`, `NodePublishVolume`, `NodeUnstageVolume`, `NodeUnpublishVolume` — routes to vendor node sockets via `osac.backend`; maintains in-memory `volumeBackends` map to track which vendor handled each volume's stage

### Stub Mode

The real gRPC fulfillment client is not yet implemented. If `--fulfillment-endpoint` is not set, the driver uses in-memory stubs (`VolumeStub`, `ControlPlaneStub`) for development. Setting `--fulfillment-endpoint` currently exits with an error.
The real gRPC fulfillment client is not yet implemented. If `--fulfillment-endpoint` is not set, the driver uses an in-memory `VolumeStub` for development, which reports the `local` backend so attach/detach no-op against the chart's default `local=none` mapping. Setting `--fulfillment-endpoint` currently exits with an error.

## Configuration

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ spec:
{{- if .Values.controller.vendorSockets }}
- "--vendor-sockets={{ .Values.controller.vendorSockets }}"
{{- end }}
{{- if .Values.controller.vendorControllers }}
- "--vendor-controllers={{ .Values.controller.vendorControllers }}"
{{- end }}
- "--v=2"
volumeMounts:
- name: osac-socket-dir
Expand Down
14 changes: 14 additions & 0 deletions osac-csi-driver/charts/csi-driver/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,20 @@ controller:
replicas: 1
clusterID: ""
vendorSockets: ""
# Comma-separated backend=endpoint pairs for vendor CSI controllers, used to
# proxy attach/detach (ControllerPublish/UnpublishVolume). The routing key is
# the "osac.backend" volume-context value, which is the StorageBackend name
# resolved from the volume's StorageTier — so each key MUST match a
# StorageBackend's metadata.name that a Cloud Provider Admin created. The names
# below are the expected convention; align them with your actual backends.
# Endpoints for network-attached vendors are the Services created by the
# csi-backends chart (namespace osac-csi-backends, gRPC port 50051). Use the
# special value "none" for node-local backends (e.g. lvms/topolvm) that need no
# controller-side attach — publish/unpublish become a no-op for those.
# An entry for a backend that is not deployed is harmless — it is only dialed
# when a volume names it.
# OSAC-4187 (0.2, temporary): direct vendor-controller proxying; reworked in 0.3.
vendorControllers: "local=none,vast=vast-csi-controller.osac-csi-backends.svc:50051,pure=pure-csi-controller.osac-csi-backends.svc:50051,ontap=trident-csi-controller.osac-csi-backends.svc:50051"
# Connection to the OSAC fulfillment-service (runs on the hub / control plane cluster).
# The CSI driver runs on tenant clusters and needs credentials that the
# fulfillment-service recognises. AAP provisions a Secret with a bearer token
Expand Down
36 changes: 22 additions & 14 deletions osac-csi-driver/cmd/osac-csi-driver/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,11 @@ func main() {
"Path to a file containing the bearer token for fulfillment-service authentication")
grpcInsecure := flag.Bool("grpc-insecure", false, "Skip TLS server certificate verification")
vendorSocketsFlag := flag.String("vendor-sockets", "",
"Comma-separated backend=socketpath pairs (e.g. ontap=/csi/trident/csi.sock)")
"Comma-separated backend=socketpath pairs for vendor node CSI sockets (e.g. ontap=/csi/trident/csi.sock)")
vendorControllersFlag := flag.String("vendor-controllers", "",
"Comma-separated backend=endpoint pairs for vendor CSI controllers, keyed "+
"by StorageBackend name (e.g. ontap=trident-csi-controller.osac-csi-backends.svc:50051). "+
"Use the value 'none' for node-local backends that need no attach (e.g. local=none)")
driverName := flag.String("driver-name", "csi.osac.openshift.io", "CSI driver name")

flag.Parse()
Expand All @@ -43,19 +47,25 @@ func main() {
os.Exit(1)
}

vendorSockets, err := parseVendorSockets(*vendorSocketsFlag)
vendorSockets, err := parseBackendMap(*vendorSocketsFlag)
if err != nil {
fmt.Fprintf(os.Stderr, "Error parsing --vendor-sockets: %v\n", err)
os.Exit(1)
}

vendorControllers, err := parseBackendMap(*vendorControllersFlag)
if err != nil {
fmt.Fprintf(os.Stderr, "Error parsing --vendor-controllers: %v\n", err)
os.Exit(1)
}

klog.Infof("Starting OSAC CSI driver %s version %s (commit %s)", *driverName, version, gitCommit)
klog.Infof("CSI endpoint: %s", *csiEndpoint)
klog.Infof("Node ID: %s", *nodeID)
klog.Infof("Vendor sockets: %v", vendorSockets)
klog.Infof("Vendor controllers: %v", vendorControllers)

var volumeClient fulfillment.VolumeClient
var controlPlaneClient fulfillment.ControlPlaneClient

if *fulfillmentEndpoint != "" {
// Establish the gRPC connection to the fulfillment-service and back the
Expand All @@ -77,13 +87,9 @@ func main() {
volumeClient = fulfillment.NewVolumeStub("default-backend", "nfs")
}

// Attach/publish still goes through the stub; the control-plane attach API
// is out of scope for OSAC-4109 (tracked separately in OSAC-3278/OSAC-4187).
controlPlaneClient = &fulfillment.ControlPlaneStub{}

d, err := driver.NewDriver(
*driverName, version, *csiEndpoint, *nodeID, *clusterID,
volumeClient, controlPlaneClient, vendorSockets,
volumeClient, vendorSockets, vendorControllers,
)
if err != nil {
klog.Fatalf("Failed to create driver: %v", err)
Expand Down Expand Up @@ -130,7 +136,9 @@ func (f *fileTokenSource) Token() (*oauth2.Token, error) {
}, nil
}

func parseVendorSockets(s string) (map[string]string, error) {
// parseBackendMap parses a comma-separated list of backend=value pairs into a
// map. It is used for both --vendor-sockets and --vendor-controllers.
func parseBackendMap(s string) (map[string]string, error) {
result := make(map[string]string)
if s == "" {
return result, nil
Expand All @@ -145,17 +153,17 @@ func parseVendorSockets(s string) (map[string]string, error) {

parts := strings.SplitN(pair, "=", 2)
if len(parts) != 2 {
return nil, fmt.Errorf("invalid vendor socket pair %q: expected format backend=socketpath", pair)
return nil, fmt.Errorf("invalid pair %q: expected format backend=value", pair)
}

backend := strings.TrimSpace(parts[0])
socketPath := strings.TrimSpace(parts[1])
value := strings.TrimSpace(parts[1])

if backend == "" || socketPath == "" {
return nil, fmt.Errorf("invalid vendor socket pair %q: backend and socketpath must not be empty", pair)
if backend == "" || value == "" {
return nil, fmt.Errorf("invalid pair %q: backend and value must not be empty", pair)
}

result[backend] = socketPath
result[backend] = value
}

return result, nil
Expand Down
Loading
Loading