[UX-889] rpk: fill in --format json/yaml support#30194
Open
graham-rp wants to merge 23 commits intoredpanda-data:devfrom
Open
[UX-889] rpk: fill in --format json/yaml support#30194graham-rp wants to merge 23 commits intoredpanda-data:devfrom
graham-rp wants to merge 23 commits intoredpanda-data:devfrom
Conversation
d37fe4a to
6e1dffd
Compare
- Add `--format` flag to `rpk cluster maintenance status` - Introduce `brokerMaintenanceStatus` struct with `json`/`yaml` tags; field names match the existing table headers (`node_id`, `enabled`, `finished`, etc.) - Nullable progress fields (`finished`, `errors`, `partitions`, `eligible`, `transferring`, `failed`) use `omitempty` — non-draining nodes produce compact output rather than a row of nulls - Add unit tests covering text, JSON, and YAML output paths
- Introduces `nodeConfigStatus` response struct with `json`/`yaml` tags matching the table columns (`node`, `config_version`, `needs_restart`, `invalid`, `unknown`) - Extracts `buildNodeStatuses` to map the admin API response to the output struct - Extracts `printNodeStatus` using `out.NewTableTo` for the text path and `f.Format` for JSON/YAML, consistent with how other commands in this package handle output - Adds `p.InstallFormatFlag(cmd)` and an `f.Help` check at the top of `Run`
- Add `--format` flag to `rpk profile list` - Introduce `profileListItem` struct with `json`/`yaml` tags matching the existing table columns (`name`, `description`) - `description` uses `omitempty` — profiles without a description produce compact output - Add unit tests covering text, JSON, and YAML output paths
- Add `--format` flag to `rpk plugin list` - Introduce `pluginRow` struct with `json`/`yaml` tags for the remote manifest listing; installed status is represented as a boolean `installed` field rather than the text asterisk prefix (text output preserves the `*` prefix) - Introduce `localPluginRow` struct for `--local` listing; shadowed paths are represented as `[]string` with `omitempty` - `NewCommand` now accepts `*config.Params` (consistent with all other command packages) - Add unit tests covering text and JSON output for both remote and local listings
- Add `--format` flag to `rpk cluster partitions move-cancel` (and its `alter-cancel` alias) - Introduce `movementCancelResult` struct with `json`/`yaml` tags matching the existing table columns (`namespace`, `topic`, `partition`, `result`) - All fields are required (non-nullable), so no `omitempty` is needed - Empty results now consistently route through the formatter — `--format json` with no movements returns `[]` rather than a prose message - Add unit tests covering `buildMovementCancelResult` and `printMovementsResult` for text, JSON, and empty-results paths
- Add `--format json|yaml|text|help` flag to `rpk redpanda admin brokers decommission-status` - Introduce `decommissionStatusResponse`, `decommissionPartition`, and `reallocationFailure` structs with json/yaml tags for structured output - `buildDecommissionStatus` transforms the admin API response into the typed struct; `printDecommissionStatus` handles both text and structured output paths - Text output is unchanged from before (section headers, optional detailed columns, human-readable sizes) - JSON/YAML output always uses raw integer sizes regardless of `--human-readable` (appropriate for machine consumers) - Fix a pre-existing bug where the `completion` variable was shared across loop iterations
- Introduces a `partitionResponse` struct with `json`/`yaml` tags for structured output - Extracts a `printAdminPartitionList(f, data, w)` helper that handles both text and structured (JSON/YAML) output - Adds `p.InstallFormatFlag(cmd)` to expose `--format` on the `list` subcommand - Adds `f.Help(...)` at the top of `Run` for `--format help` support - Text output is unchanged: columns remain `TOPIC`, `PARTITION`, `IS-LEADER`
- Added `--format` flag to `rpk cloud auth list` - Introduced `cloudAuthRow` response struct with `json`/`yaml` tags - In JSON/YAML output the current-auth indicator is a `current: true` boolean field (with `omitempty` so non-current entries omit the field entirely) - Text output continues to append `*` to the current auth name, preserving existing behaviour - Added table-driven unit tests covering text, JSON, and YAML output
- Introduces `balancerStatusResponse` struct with json/yaml tags that captures all fields from the command's two output sections (key-value status and broker replica distribution table) - `buildBalancerStatusResponse` maps `rpadmin.PartitionBalancerStatus` + cluster partitions into the response struct, sorting broker distribution by node ID - `printBalancerStatus` now accepts `config.OutFormatter` and `io.Writer`; non-text formats delegate to `f.Format`, while text preserves the existing section-based output exactly - `BrokerReplicaDistribution` uses `omitempty` so JSON/YAML output is compact when no partition data is available - `p.InstallFormatFlag(cmd)` and `f.Help(...)` wired in following the canonical pattern from `cluster maintenance status`
- New `logDirRow` struct with `json`/`yaml` tags used for both serialization and table output - `printLogDirs` helper centralizes both text and structured-format output paths - JSON/YAML always emit the full flat per-partition struct regardless of `--aggregate-into`; text preserves the existing per-mode column aggregation behavior - `--sort-by-size` now runs inside `printLogDirs`, after aggregation, so sizes are ranked on post-aggregation totals (same semantics as before) - `p.InstallFormatFlag(cmd)` and `f.Help(...)` added following the canonical pattern
- Introduces a `secretListItem` struct with `json`/`yaml` tags (`id`, `scopes`) to drive both structured and text output
- Adds `printSecretList(f config.OutFormatter, items []secretListItem, w io.Writer)` — a testable print function separate from the cloud API call
- Wires up `p.InstallFormatFlag(cmd)` and `f.Help([]secretListItem{})` at the top of `Run`
- Text output is unchanged: scopes are comma-joined as before
- JSON/YAML output represents scopes as a proper string array rather than a pre-joined string
- Moves the invalid-scope warning from stdout to stderr
- `rpk group list` and `rpk group delete` now accept `--format json|yaml|text|help` - Response structs `listedGroup` and `deletedGroup` with matching json/yaml tags - `state` field on `listedGroup` uses `omitempty` since it is absent on older brokers that return the v3 ListGroups response - Text path preserves existing behaviour: the STATE column is conditionally shown only when at least one group has a non-empty state - Unit tests cover text (with and without state), json, and yaml output for both commands
- Adds `p.InstallFormatFlag(cmd)` and `f.Help(...)` guard to the command - Introduces `partitionMoveStatus` struct with JSON/YAML tags for structured output; node lists are `[]int` and byte counts are `int` (raw values, letting consumers format as needed) - Extracts `buildMoveStatuses` helper to convert `[]rpadmin.ReconfigurationsResponse` → `[]partitionMoveStatus` - Extracts `printMoveStatus(f, statuses, human, w)` that handles both structured (json/yaml) and text output - Text output preserves the existing column names and formatting (human-readable bytes via `-H`, node ID lists as `[1 2 3]`) - Fixes a latent bug: the empty-result path now routes through the formatter so `--format json` returns `[]` rather than a prose message
- Installs the `--format` flag via `p.InstallFormatFlag(cmd)`
- Removes the stale `// always text for now` comment
- Adds `f.Help(&aclDeleteOutput{})` guard at the top of Run, matching the pattern in `acl list`
- Introduces `aclDeleteOutput` (with `filters,omitempty` and `deletions` fields) as the structured output type, mirroring `aclListOutput` from the same package
- Replaces the separate `printDeleteFilters` / `printDeleteResults` helpers with a single `printDeleteOutput(f, output, printDeletionsHeader, w io.Writer)` that handles both text and structured-format paths
- Removes the custom `cmd.Flags().StringVar(&format, "format", ...)` declaration and replaces it with `p.InstallFormatFlag(cmd)` - Introduces local `selfTestResult` / `selfTestNodeReport` types that mirror the `rpadmin` structs and add the `yaml:` tags required by `OutFormatter.Help()` - Routes all output through `cmd.OutOrStdout()` via a new `printSelfTestStatus(f, reports, w)` function, eliminating direct `fmt.Print` calls inside the run closure - `--format json` key names are unchanged (matching the original `rpadmin` JSON field names)
- Adds `topicDeleteResult` struct with `topic` and `status` JSON/YAML fields - Extracts `printTopicDeleteResults` for testability - Wires `p.InstallFormatFlag` to register the flag
- Adds `addPartitionsResult` struct with `topic` and `status` JSON/YAML fields - Extracts `printAddPartitionsResults` for testability - Wires `p.InstallFormatFlag` to register the flag - Preserves exit code 1 on any partition error
- Adds `alterConfigResult` struct with `topic` and `status` JSON/YAML fields - Extracts `printAlterConfigResults` for testability - Wires `p.InstallFormatFlag` to register the flag
- Replaces `seekCommit`/`seekCommitErr` dual-struct pattern with a single `seekCommitResult` struct with `error,omitempty` for clean JSON output - Extracts `printSeekResults` for testability - Wires `p.InstallFormatFlag` to register the flag
- Adds `offsetDeleteResult` struct with `topic`, `partition`, and `status` fields - Replaces raw `fmt.Fprintf(tw, ...)` with `printOffsetDeleteResults` for testability - Results are sorted deterministically (topic then partition) for stable output - Wires `p.InstallFormatFlag` to register the flag - Preserves exit code 1 when any partition deletion fails
- Use cmd.OutOrStdout() throughout all 20 --format commands instead of os.Stdout, so output can be redirected (e.g. cmd.SetOut in tests) - Add out.SectionTo(w, header) so section headers go to the same writer as table rows; Section() now delegates to it. Fixes acl delete and decommission-status where headers were going to stdout while rows went to the writer parameter - profile list: represent current profile with Current bool field in JSON/YAML instead of embedding '*' in the name; asterisk is still appended in text output only, consistent with cloud auth list - cancel: restore "there are no ongoing partition movements to cancel" message on the text path when results are empty Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
7782ae2 to
1e58bd5
Compare
Collaborator
Retry command for Build#83403please wait until all jobs are finished before running the slash command |
Collaborator
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1e58bd5 to
f3ad110
Compare
Collaborator
Retry command for Build#83408please wait until all jobs are finished before running the slash command |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
--format json|yaml|text|helpto 20 rpk commands that previously only supported plain text output. Each commit is independent and can be reviewed or cherry-picked individually.Commits
1cc5f78e76cluster maintenance statusf06dce7583cluster config status8687543885profile list497a023ff7plugin listaf27184ab7cluster partitions move-cancel72cdeca066redpanda admin brokers decommission-status456c22d77bredpanda admin partitions listac1e469643cloud auth listfe375cff53cluster partitions balancer-status0018aa98fecluster logdirs describec2d87e155csecurity secret list3098479f49group list/group delete2f424e2246cluster partitions move-status944dd72133security acl delete470e778fb0cluster selftest status(migration from custom flag)50fe5a3142topic deletedb46f11ea6topic add-partitions0cfab42a31topic alter-config851d6e6939group seek9160f1c3f5group offset-deletePattern
All changes follow the same pattern established by existing commands like
rpk redpanda admin brokers list:p.InstallFormatFlag(cmd)to register--formatf.Help(...)guard at the top of Run for--format helpf.Format(data)for JSON/YAML output; text path unchangedjson/yamltags replacing ad-hoc table printingprint*helpers extracted from command Run functionsTest results
All commands verified against a live 3-node
rpk containercluster.security secret listrequires cloud credentials and was unit-tested only.1cc5f78e76cluster maintenance statusSetup: Maintenance mode enabled on broker 1.
[ {"node_id":0,"enabled":false}, {"node_id":1,"enabled":true,"finished":true,"errors":false,"partitions":11,"eligible":0,"transferring":4,"failed":0}, {"node_id":2,"enabled":false} ]f06dce7583cluster config status[ {"node":0,"config_version":2,"needs_restart":false,"invalid":[],"unknown":[]}, {"node":1,"config_version":2,"needs_restart":false,"invalid":[],"unknown":[]}, {"node":2,"config_version":2,"needs_restart":false,"invalid":[],"unknown":[]} ]8687543885profile list[ {"name":"local-console","description":"","current":false}, {"name":"rpk-cloud","description":"Redpanda Production \"gcs-test/gcs-test\"","current":false}, {"name":"rpk-container","description":"Automatically generated profile from 'rpk container start'","current":true} ]The current profile is represented by
current: truein JSON/YAML. In textoutput, the current profile's
NAMEcolumn is suffixed with*(matchingcloud auth list).497a023ff7plugin list --localSetup: fake
.rpk-testplugininstalled in~/.local/bin; two managed plugins already present.[ {"name":"byoc","path":"/Users/graham.smith/.local/bin/.rpk.managed-byoc"}, {"name":"connect","path":"/Users/graham.smith/.local/bin/.rpk.managed-connect"}, {"name":"testplugin","path":"/Users/graham.smith/.local/bin/.rpk-testplugin"} ]af27184ab7cluster partitions move-cancelSetup:
raft_learner_recovery_ratethrottled to 1000 B/s; all 3format-addpartspartitions (RF=1) moved simultaneously then immediately cancelled.[ {"namespace":"kafka","topic":"format-addparts","partition":0,"result":"Success"}, {"namespace":"kafka","topic":"format-addparts","partition":1,"result":"Success"}, {"namespace":"kafka","topic":"format-addparts","partition":2,"result":"Success"} ]72cdeca066redpanda admin brokers decommission-statusSetup:
raft_learner_recovery_ratethrottled to 1000 B/s; broker 2 decommissioned then status captured immediately. All partitions fail reallocation — 3-node cluster with RF=3 topics leaves no eligible destination node.{ "reallocation_failures": [ {"partition":"kafka/format-test/0","reason":"No eligible node found to move replica"}, {"partition":"kafka/format-test/1","reason":"No eligible node found to move replica"}, {"partition":"kafka/format-test/2","reason":"No eligible node found to move replica"}, {"partition":"kafka/__consumer_offsets/0","reason":"No eligible node found to move replica"}, {"partition":"kafka/__consumer_offsets/1","reason":"No eligible node found to move replica"}, {"partition":"kafka/__consumer_offsets/2","reason":"No eligible node found to move replica"}, {"partition":"kafka/format-seek-topic/0","reason":"No eligible node found to move replica"}, {"partition":"kafka/format-seek-topic/1","reason":"No eligible node found to move replica"}, {"partition":"kafka/format-seek-topic/2","reason":"No eligible node found to move replica"}, {"partition":"kafka_internal/id_allocator/0","reason":"No eligible node found to move replica"} ], "partitions": [] }456c22d77bredpanda admin partitions list 0[ {"topic":"__consumer_offsets","partition":0,"is_leader":false}, {"topic":"__consumer_offsets","partition":1,"is_leader":false}, {"topic":"__consumer_offsets","partition":2,"is_leader":true}, {"topic":"format-seek-topic","partition":0,"is_leader":true}, {"topic":"format-seek-topic","partition":1,"is_leader":false}, {"topic":"format-seek-topic","partition":2,"is_leader":false}, {"topic":"format-test","partition":0,"is_leader":false}, {"topic":"format-test","partition":1,"is_leader":true}, {"topic":"format-test","partition":2,"is_leader":false} ]ac1e469643cloud auth list[ {"name":"...-sso Redpanda Integration","kind":"sso","organization":"Redpanda Integration","organization_id":"a845616f-0484-4506-9638-45fe28f34865"}, {"name":"...-sso Redpanda Production","kind":"sso","organization":"Redpanda Production","organization_id":"617d637f-4645-4627-8841-c24be02f8817","current":true} ]fe375cff53cluster partitions balancer-status{ "status": "ready", "seconds_since_last_tick": 14, "current_reassignments_count": 0, "partitions_pending_force_recovery_count": 0, "broker_replica_distribution": [ {"node_id":0,"count":11}, {"node_id":1,"count":10}, {"node_id":2,"count":11} ] }0018aa98fecluster logdirs describe[ {"broker":0,"dir":"/var/lib/redpanda/data","topic":"format-test","partition":0,"size":465}, {"broker":0,"dir":"/var/lib/redpanda/data","topic":"format-test","partition":1,"size":155}, {"broker":0,"dir":"/var/lib/redpanda/data","topic":"format-test","partition":2,"size":423}, "..." ]c2d87e155csecurity secret listUnit-tested only — requires Redpanda Cloud credentials.
3098479f49group list[{"broker":0,"group":"format-group","state":"Stable"}]2f424e2246cluster partitions move-statusNo movements in progress — returns
[]. Empty-result path verified to route through formatter.944dd72133security acl delete{ "deletions": [ {"principal":"User:alice","host":"*","resource_type":"TOPIC","resource_name":"format-test","resource_pattern_type":"LITERAL","operation":"READ","permission":"ALLOW","message":""} ] }470e778fb0cluster selftest statusFull self-test run (~6 min, disk + network). Three nodes, all
"status":"idle"after completion.[ {"node_id":2,"status":"idle","stage":"cloud","results":[{"name":"512KB sequential r/w","test_type":"disk","p50":191,"p90":271,"p99":415,"rps":17660,"bps":9258961032},{"name":"8Kb Network Throughput Test","test_type":"network","p50":67,"rps":68925,"bps":564635511},"..."]}, {"node_id":0,"status":"idle","stage":"cloud","results":["..."]}, {"node_id":1,"status":"idle","stage":"cloud","results":["..."]} ]50fe5a3142topic delete[{"topic":"format-delete-me","status":"OK"}]db46f11ea6topic add-partitions[{"topic":"format-addparts","status":"OK"}]0cfab42a31topic alter-config[{"topic":"format-test","status":"OK"}]851d6e6939group seekGroup was active; seek rejected with
INVALID_OPERATIONfor all partitions.[ {"topic":"format-seek-topic","partition":0,"prior_offset":-1,"current_offset":0,"error":"INVALID_OPERATION: seeking a non-empty group is not allowed."}, {"topic":"format-seek-topic","partition":1,"prior_offset":1,"current_offset":0,"error":"INVALID_OPERATION: seeking a non-empty group is not allowed."}, {"topic":"format-seek-topic","partition":2,"prior_offset":-1,"current_offset":0,"error":"INVALID_OPERATION: seeking a non-empty group is not allowed."} ]9160f1c3f5group offset-deleteGroup put into Empty state first.
[ {"topic":"format-seek-topic","partition":0,"status":"OK"}, {"topic":"format-seek-topic","partition":1,"status":"OK"}, {"topic":"format-seek-topic","partition":2,"status":"OK"} ]Release Notes
Improvements
--format json|yaml|text|helpto 20 rpk commands:cluster maintenance status,cluster config status,profile list,plugin list,cluster partitions move-cancel,redpanda admin brokers decommission-status,redpanda admin partitions list,cloud auth list,cluster partitions balancer-status,cluster logdirs describe,security secret list,group list,group delete,cluster partitions move-status,security acl delete,cluster selftest status,topic delete,topic add-partitions,topic alter-config,group seek,group offset-delete