Skip to content

feat(storage): add move command#838

Open
natalie-o-perret wants to merge 2 commits into
masterfrom
feat/storage-move
Open

feat(storage): add move command#838
natalie-o-perret wants to merge 2 commits into
masterfrom
feat/storage-move

Conversation

@natalie-o-perret

@natalie-o-perret natalie-o-perret commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

What

Adds exo storage move to move objects within a bucket or across buckets, without downloading them locally.

How

Server-side copy + delete. Works for single objects and prefixes. Large objects (over 5 GiB) use multipart copy for efficiency.

Metadata, headers, and ACLs are preserved.

Not atomic

If the delete step fails after a successful copy, the object stays in both locations. No automatic rollback. This is documented in the command help.

Flags

  • -n dry-run mode
  • -f skip confirmation prompt
  • -r recursive (for prefix moves)
  • -v verbose output (shows copy/delete steps, size, URL)
  • --multipart-concurrency control parallel part uploads

Testing

Unit tests for single object and multipart paths. Abort on part copy failure covered. Parts are verified to be sorted before completion. Metadata preservation is verified on multipart uploads.

Unit tests

=== RUN   TestMoveObject_SingleObject
    --- PASS: TestMoveObject_SingleObject/successful_move_within_same_bucket
    --- PASS: TestMoveObject_SingleObject/move_fails_when_copy_fails
    --- PASS: TestMoveObject_SingleObject/delete_fails_after_copy
    --- PASS: TestMoveObject_SingleObject/head_object_fails
=== RUN   TestMoveObject_Multipart
    --- PASS: TestMoveObject_Multipart/successful_multipart_move
    --- PASS: TestMoveObject_Multipart/multipart_abort_on_part_copy_failure
PASS

Local smoke tests

Validation rejects missing destination key when source is a single object:

$ exo storage move -n sos://meow/smoke.txt sos://other-bucket
error: destination must include an object key when source is a single object: sos://other-bucket

Dry-run with prefix (no spurious "no objects exist" message):

$ exo storage move -n -r sos://meow/ sos://meow/dest/
[DRY-RUN]
move sos://meow/smoke.txt -> sos://meow/dest/smoke.txt

Non-dry-run with empty prefix (correctly reports no objects):

$ exo storage move -f -r sos://meow/missing-prefix/ sos://meow/dest/
no objects exist at "missing-prefix/"

Verbose prefix move:

$ exo storage move -v -f -r sos://meow/ sos://meow/dest/
copying: sos://meow/smoke.txt -> sos://meow/dest/smoke.txt
deleting: sos://meow/smoke.txt
moved: sos://meow/smoke.txt -> sos://meow/dest/smoke.txt
moved 1 objects

Single-object verbose move:

$ exo storage move -v -f sos://meow/file.txt sos://meow/folder/file.txt
copying: sos://meow/file.txt -> sos://meow/folder/file.txt
deleting: sos://meow/file.txt
moved: file.txt -> https://sos-ch-gva-2.exo.io/meow/folder/file.txt (12 bytes, 2026-06-04 13:31:54 UTC)

Note

Includes Go 1.26.4 update to fix stdlib vulnerabilities (govulncheck was failing on earlier Go versions).

Note

GitHub Copilot with Sonnet 4.6 has been used for respectively:

  • Refactoring purposes
  • Drafting comments
  • Drafting this description

@natalie-o-perret natalie-o-perret added the enhancement New feature or request label Jun 3, 2026
@natalie-o-perret natalie-o-perret self-assigned this Jun 3, 2026
@natalie-o-perret

Copy link
Copy Markdown
Contributor Author

[SC-182796]

@natalie-o-perret natalie-o-perret marked this pull request as ready for review June 4, 2026 13:18
@natalie-o-perret natalie-o-perret marked this pull request as draft June 4, 2026 13:18
Adds `exo storage move` to move objects within a bucket or across
buckets, without downloading them locally.

Implements server-side copy + delete, preserving metadata, headers,
and ACLs. Large objects (over 5 GiB) use multipart copy. Prefix mode
selects all objects sharing a prefix, controlled by a trailing slash
on the source or the -r flag.

Includes a Go 1.26.4 update to fix stdlib vulnerabilities
(GO-2026-5039, GO-2026-5037) that were failing govulncheck.
@natalie-o-perret natalie-o-perret marked this pull request as ready for review June 4, 2026 14:00
@natalie-o-perret natalie-o-perret requested a review from a team June 4, 2026 14:02
@natalie-o-perret natalie-o-perret marked this pull request as draft June 4, 2026 14:43
- Merge storage_move_utils.go into storage_move.go; no other storage
  command has a _utils.go split.
- Inline the confirmPrefixMove wrapper; the call is a single AskQuestion
  expression.
- Handle flag-lookup errors in RunE instead of discarding them with _,
  matching the pattern used by every other storage command.
- Replace return cmd.Usage() in PreRunE with
  exocmd.CmdExitOnUsageError, matching storage_delete and other commands
  that exit with a 'invalid arguments' reason on bad arity.
@natalie-o-perret natalie-o-perret marked this pull request as ready for review June 4, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants