Feature: enable faster server-server copies by enabling ComposeObject with --part-size and --parallel

[afkbluey]

Overview

I have a fast k8s on prem cluster and have proposed adding --part-size and --parallel flags for supporting fast streaming (copy) of large files >= 5TB in PR 5257 on mc cli repo

From that PR I discovered the server-server copy methods don't have configurable --part-size or --parallel. This means I can configure mc cp for very large files but ironically have less control for smaller ones < 5TB.

This PR enables ComposeObject to be configurable by equivalent args PartSize and NumThreads (--parallel).

[allanrogerr]

A server-side copy moves no object bytes through the client, so there is no memory-usage effect. Values above 100 are capped rather than rejected, and PartSize steers an even split rather than setting an exact part size. Both fields are ignored when the compose collapses to a single copy.

Suggested change

	// NumThreads sets the number of concurrent part uploads. If not set,
	// defaults to 4. Maximum allowed is 100.
	NumThreads int
	// PartSize sets the part size for multipart uploads. If not set,
	// uses the automatic calculation. Minimum is 5MiB (absMinPartSize).
	// This is useful for controlling memory usage and optimizing for
	// different network conditions.
	PartSize int64
	// NumThreads sets the number of concurrent upload-part-copy requests.
	// Defaults to 4, and values above 100 are capped at 100. Ignored when
	// the compose collapses to a single server-side copy.
	NumThreads int
	// PartSize sets the target part size for the multipart copy, in bytes.
	// Minimum is 5 MiB; if unset, the library computes it. Sources are
	// split evenly, so actual part sizes may be smaller or larger. Ignored
	// when the compose collapses to a single server-side copy.
	PartSize int64

[allanrogerr]

A negative PartSize passes both checks (-1 > 0 is false) and is then silently treated as unset. != 0 rejects it with the same message.

Suggested change

	if opts.PartSize > 0 && opts.PartSize < absMinPartSize {
	if opts.PartSize != 0 && opts.PartSize < absMinPartSize {

[allanrogerr]

calculateEvenSplits divides each source evenly across the part count, so any PartSize under 10 MiB can produce parts below the server's 5 MiB minimum: a 12 MiB source with PartSize: 5<<20 becomes three 4 MiB parts, which the server rejects with EntityTooSmall. The cap keeps splits legal, and that compose then succeeds as two 6 MiB parts.

Rejecting sub-10 MiB values in validate() would also close it. Consider and test whichever contract you want.

Suggested change

	r := size / partSize
	if size%partSize > 0 {
	r++
	}
	return r
	r := size / partSize
	if size%partSize > 0 {
	r++
	}
	// Even splits across this count must stay at or above absMinPartSize,
	// or the server rejects every part but the last.
	if r > 1 && size/r < absMinPartSize {
	r = max(size/absMinPartSize, 1)
	}
	return r

[allanrogerr]

Three linked problems, each reproduced against a live server:

• dst.Progress is read from every part goroutine at once, so a reader that was safe with the serial ComposeObject now trips the race detector — and can still be read after the call returns.
• A failed part does not stop the others: collection starts only once every part is launched, so a fault injected at part 3 of 16 still ran 15 copies.
• The failed upload is left incomplete on the server.

The replacement below covers the whole section, because the pieces interlock across two hunks. It launches from its own goroutine so the collector can cancel the rest on the first error, reads progress from one goroutine, and aborts the upload. Consider and test this.

	// 3. Perform copy part uploads
	numThreads := dst.NumThreads
	if numThreads <= 0 {
		numThreads = totalWorkers
	}
	if numThreads > 100 {
		numThreads = 100
	}

	// Cancels queued and in-flight part copies once any part fails.
	partCtx, cancelParts := context.WithCancel(ctx)
	defer cancelParts()

	type partResult struct {
		part CompletePart
		size int64
		err  error
	}
	results := make(chan partResult, totalParts)
	var wg sync.WaitGroup
	sem := make(chan struct{}, numThreads)

	// Launched from its own goroutine so results are collected while parts
	// are still queued behind the semaphore, letting the first failure
	// cancel the rest.
	go func() {
		partIndex := 1
		for i, src := range srcs {
			h := make(http.Header)
			src.Marshal(h)
			if dst.Encryption != nil && dst.Encryption.Type() == encrypt.SSEC {
				dst.Encryption.Marshal(h)
			}

			// calculate start/end indices of parts after
			// splitting.
			startIdx, endIdx := calculateEvenSplits(srcObjectSizes[i], src, dst.PartSize)
			for j, start := range startIdx {
				end := endIdx[j]

				wg.Add(1)
				sem <- struct{}{}

				go func(idx int, s, e int64, hdr http.Header) {
					defer wg.Done()
					defer func() { <-sem }()

					// Add (or reset) source range header for
					// upload part copy request.
					hdr.Set("x-amz-copy-source-range",
						fmt.Sprintf("bytes=%d-%d", s, e))

					// make upload-part-copy request
					complPart, err := c.uploadPartCopy(partCtx, dst.Bucket,
						dst.Object, uploadID, idx, hdr)

					results <- partResult{part: complPart, size: e - s + 1, err: err}
				}(partIndex, start, end, h.Clone())

				partIndex++
			}
		}
		wg.Wait()
		close(results)
	}()

	// Progress is read only here, so a caller's reader is never accessed
	// concurrently. Draining to the close keeps any goroutine from
	// outliving this call.
	objParts := make([]CompletePart, 0, totalParts)
	var partErr error
	for res := range results {
		if res.err != nil {
			if partErr == nil {
				partErr = res.err
				cancelParts()
			}
			continue
		}
		if partErr == nil {
			objParts = append(objParts, res.part)
			if dst.Progress != nil {
				io.CopyN(io.Discard, dst.Progress, res.size)
			}
		}
	}
	if partErr != nil {
		c.abortMultipartUpload(ctx, dst.Bucket, dst.Object, uploadID)
		return UploadInfo{}, partErr
	}

	// Results arrive out of order; complete-multipart requires
	// ascending part numbers.
	sort.Sort(completedParts(objParts))

[allanrogerr]

The existing custom-size rows are exact multiples of the part size, the one shape even splits always handle. These sizes are not multiples, so they fail without the cap in partsRequired and pin it in place.

Suggested change

	{50 * 1024 * 1024, 5 * 1024 * 1024, 10},
	{50 * 1024 * 1024, 5 * 1024 * 1024, 10},
	// Sizes that are not multiples of the custom part size: the count
	// is capped so even splits stay at or above absMinPartSize.
	{12 * 1024 * 1024, 5 * 1024 * 1024, 2},
	{10*1024*1024 + 1, 5 * 1024 * 1024, 2},
	{6 * 1024 * 1024, 5 * 1024 * 1024, 1},

[allanrogerr]

Every other example in this directory builds with the example tag, apart from healthcheck.go.

Suggested change

	//go:build ignore
	// +build ignore
	//go:build example
	// +build example

[allanrogerr]

New file — the year should be the current one.

Suggested change

	* Copyright 2024 MinIO, Inc.
	* Copyright 2026 MinIO, Inc.

[allanrogerr]

A server-side copy moves no bytes through the client, so the memory-usage claim does not hold, and these are copies rather than uploads.

Suggested change

	// Configure parallel uploads with 10 concurrent threads
	NumThreads: 10,
	// Configure custom part size (10 MiB)
	// This is useful for controlling memory usage and optimizing for
	// different network conditions. If not specified, uses automatic calculation.
	PartSize: 10 * 1024 * 1024, // 10 MiB
	}
	// Perform the compose operation with parallel uploads
	// Run 10 concurrent server-side copy requests.
	NumThreads: 10,
	// Target part size; the library computes one if unset.
	PartSize: 10 * 1024 * 1024, // 10 MiB
	}
	// Perform the compose operation with parallel server-side copies

[allanrogerr]

log.Println already starts a fresh line; the leading \n puts a blank line after the timestamp prefix instead of before the message.

Suggested change

	log.Println("\nParallel compose completed with:")
	log.Println("Parallel compose completed with:")

[allanrogerr]

The title works as-is; the description could say more about what the PR changes. A proposed replacement, then why:

# Overview

I have a fast on-prem k8s cluster and proposed `--part-size` and `--parallel` flags in [mc PR #5257](https://github.com/minio/mc/pull/5257) for fast streaming copies of large files. Working on that PR I found the server-side copy path has no equivalent controls: `mc cp` can be tuned for very large files, but server-side copies of smaller ones cannot.

This PR adds two fields to `CopyDestOptions`, used by `ComposeObject`'s multipart copy path:

- `NumThreads` — number of concurrent upload-part-copy requests (defaults to 4, capped at 100).
- `PartSize` — target part size in bytes (minimum 5 MiB). Each source is split evenly, so the actual part size is derived from this value rather than used exactly.

Also adds `examples/s3/compose-object-parallel.go` demonstrating both options.

`examples/s3/go.sum` gains the module-zip (`h1:`) hash for `gopkg.in/check.v1` — TODO: confirm it is needed, or drop it.

## Tests

- `TestPartsRequired` covers the custom part-size arithmetic.
- TODO: describe how the parallel path was verified (for example, a compose against a live server with `NumThreads`/`PartSize` set, content compared byte-for-byte).
- TODO: add both new fields to the ComposeObject section of `docs/API.md` (it documents `PutObjectOptions.PartSize` but nothing for `CopyDestOptions`).

• "configurable by equivalent args" overstates the PartSize half: ComposeObject splits each source evenly, so PartSize steers the split rather than setting the exact size the mc flag implies.
• The new example file and the examples/s3/go.sum change go unmentioned. That change adds the module-zip (h1:) hash for gopkg.in/check.v1, which the examples do not import; the file already carried that module's /go.mod hash, as examples/minio/go.sum still does.
• Nothing describes how this was tested, which matters because the compose path now behaves differently under failure and concurrency.
• docs/API.md's ComposeObject section does not mention the new fields.