Skip to content

blob: add Bucket.SetDefaultReaderOptions - #3820

Merged
vangent merged 2 commits into
google:masterfrom
stanhu:sh-blob-set-default-reader-options
Nov 2, 2026
Merged

vangent merged 2 commits into
google:masterfrom
stanhu:sh-blob-set-default-reader-options

Conversation

@stanhu

@stanhu stanhu commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Implements the approach @vangent proposed in #3818, as an alternative to the URL-parameter PR #3819 (now closed).

What

Bucket.SetDefaultReaderOptions(ReaderOptions) error lets an application set bucket-wide default ReaderOptions once, instead of passing them on every Download/NewReader/NewRangeReader call. When a read method is called with nil opts, a copy of the default is used in place of an empty ReaderOptions.

b, _ := blob.OpenBucket(ctx, "s3://mybucket?region=us-west-2")
_ = b.SetDefaultReaderOptions(blob.ReaderOptions{AllowParallelDownload: true, BufferSize: 16 << 20, MaxConcurrency: 8})
// Now Download with nil opts uses parallel download by default.
_ = b.Download(ctx, key, f, nil)

How

  • Adds a defaultReaderOptions *ReaderOptions field to Bucket (guarded by the existing b.mu).
  • SetDefaultReaderOptions stores a copy. Only performance-related fields may be defaulted; setting BeforeRead returns an InvalidArgument error, since it's a per-call concern.
  • A small readerOptionsOrDefault helper substitutes the default (a copy) when opts == nil, applied in Download and newRangeReader (the shared helper behind NewReader/NewRangeReader/ReadAll).

This lives entirely in the portable blob package and needs no driver changes — it just substitutes the ReaderOptions before the existing per-call plumbing (the AllowParallelDownload gate and the BufferSize/MaxConcurrency fields drivers already honor).

Semantics

A nil opts gets a copy of the default; a non-nil opts is used as-is (the default is not merged in), which avoids any unset-vs-zero ambiguity. Explicit per-call options therefore always win.

Tests

TestSetDefaultReaderOptions covers: the BeforeRead allowlist error; a default enabling the parallel path for a nil-opts Download; part-size/concurrency defaults flowing to the driver; fallback to the reader with no default; and that an explicit opts does not inherit the default. go vet/gofmt clean; full blob + s3blob/gcsblob/azureblob suites pass (no driver changes).

Follow-up

SetDefaultWriterOptions (with a smaller allowlist) is a natural follow-up.

Let applications set bucket-wide default ReaderOptions once instead of
passing them on every Download/NewReader call. When a read method is
called with nil opts, a copy of the default is used in place of an empty
ReaderOptions.

Only performance-related fields may be defaulted (AllowParallelDownload,
BufferSize, MaxConcurrency); setting BeforeRead returns an error, since
it is a per-call concern. This lives entirely in the portable blob
package and needs no driver changes.

Alternative to the URL-parameter approach, per discussion on
google#3818.
Comment thread blob/blob.go Outdated
}
b.mu.Lock()
defer b.mu.Unlock()
cp := opts

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it should be OK to set b.defaultReaderOptions = &optshere, right?opts` is already a copy.

@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.14%. Comparing base (18e4aa5) to head (8b6b2cc).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3820      +/-   ##
==========================================
- Coverage   80.15%   80.14%   -0.01%     
==========================================
  Files         104      104              
  Lines       12454    12464      +10     
==========================================
+ Hits         9982     9989       +7     
- Misses       2471     2474       +3     
  Partials        1        1              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

opts is a value parameter, so it is already a per-call copy; take its
address directly instead of copying again.
@vangent
vangent merged commit 9f12690 into google:master Nov 2, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants