Skip to content

test: reproduce opendal 0.58's S3/ADLS credential-ignoring regressions - #112

Merged
gbrgr merged 3 commits into
mainfrom
gb/test-s3-allow-anonymous-credential-priority
Sep 2, 2026
Merged

test: reproduce opendal 0.58's S3/ADLS credential-ignoring regressions#112
gbrgr merged 3 commits into
mainfrom
gb/test-s3-allow-anonymous-credential-priority

Conversation

@gbrgr

@gbrgr gbrgr commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Regression coverage for two related production incidents found while
validating the RustyIceberg 0.9.2 bump in raicode
(raicode#28144).

S3: s3.allow-anonymous ignores configured credentials

raicode's Iceberg data-loading integration always sends
"s3.allow-anonymous" => "true" alongside any explicit credentials,
intending it purely as a fallback for when no credentials are configured.

Under the fork's old opendal (0.55, pre RelationalAI/iceberg-rust#78),
that was safe: allow_anonymous was only consulted after trying to load
a real credential, and only skipped signing if none was found
(opendal-0.55.0/src/services/s3/core.rs, S3Core::load_credential).

Upstream's opendal 0.58 renamed the flag to skip_signature and changed
its semantics: it's now checked first, unconditionally, before any
credential is even loaded (opendal-service-s3-0.58.2/src/core.rs:
if self.skip_signature { return ...unsigned... }). So configured
credentials are silently ignored whenever allow-anonymous is also set —
every request now goes out unsigned, which a private bucket correctly
rejects with 403 AccessDenied / AUTH_FAILED.

Verified locally against this repo's own docker-compose MinIO: 14/14
pass, including both new assertions (allow-anonymous + valid credentials
AUTH_FAILED; allow-anonymous omitted with the same credentials →
success).

ADLS: no anonymous-access mode existed at all

Unlike S3's opendal service, opendal's Azdls service had no anonymous/
unsigned mode: every request went through a generic credential-provider
chain that hard errors ("failed to load signing credential") if no
credential can be found or validated. An empty adls.sas-token doesn't
help either — it's treated as a found-but-invalid credential, which
fails the same way. So reading a genuinely public, unauthenticated Azure
container wasn't possible through this stack at all.

RelationalAI/iceberg-rust#79
adds an explicit "adls.allow-anonymous" opt-in (mirroring
s3.allow-anonymous), routing to opendal's generic Http service
(plain unauthenticated GET/HEAD) instead of the signed Azdls one.

Verified against the real public Azure test container
(publicbenchmarks.dfs.core.windows.net/integration-tests, the same
one raicode's own integration tests read from): without the flag,
table_open correctly fails with AUTH_FAILED; with it, the read
succeeds. Local run: 17/17 pass in the same testset.

Status

  • iceberg/iceberg-catalog-rest currently point at
    RelationalAI/iceberg-rust#79's branch tip for this verification —
    needs a follow-up bump to the actual merge commit once that PR
    lands
    (flagged with a TODO in Cargo.toml).
  • Version bumped 0.9.2 → 0.9.3 to match.

🤖 Generated with Claude Code

…ndal 0.58

Regression coverage for a real production incident: raicode's Iceberg
data-loading integration always sends "s3.allow-anonymous" => "true"
alongside any explicit credentials, intending it purely as a fallback
for when no credentials are configured.

Under the fork's old opendal (0.55), that was safe -- allow_anonymous
was only consulted after trying to load a real credential, and only
skipped signing if none was found. Upstream's opendal 0.58 (pulled in
by the Aug 2026 iceberg-rust merge, RelationalAI/iceberg-rust#78)
renamed the flag to skip_signature and changed its semantics: it's now
checked first, unconditionally, before any credential is even loaded.
Configured credentials are silently ignored whenever allow-anonymous
is also set, so every request goes out unsigned -- which a private
bucket correctly rejects with 403/AUTH_FAILED.

Verified locally against this repo's own docker-compose MinIO/Polaris
setup: both assertions pass against the current (0.9.2) FFI build.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@gbrgr
gbrgr requested review from hall-alex and robertbuessow and removed request for robertbuessow September 2, 2026 07:37
Regression coverage for the ADLS counterpart of the s3.allow-anonymous
incident already covered in this PR: unlike S3's opendal service (which
exposes an explicit skip_signature bypass), opendal's Azdls service has
no anonymous/unsigned mode at all -- every request goes through a
generic credential-provider chain that hard errors ("failed to load
signing credential") if no credential can be found or validated. An
empty adls.sas-token doesn't help either: it's treated as a found-but-
invalid credential, which fails the same way. So reading a genuinely
public, unauthenticated Azure container was not possible through this
stack at all.

RelationalAI/iceberg-rust#79 adds an explicit "adls.allow-anonymous"
opt-in (mirroring s3.allow-anonymous), routing to opendal's generic Http
service instead of the signed Azdls one. Verified locally against this
repo's own docker-compose infra, plus the real public Azure test
container (publicbenchmarks.dfs.core.windows.net/integration-tests,
the same one RelationalAI/raicode's own integration tests read from):
without the flag, table_open correctly fails with AUTH_FAILED; with it,
the read succeeds.

Bumps the git rev to iceberg-rust#79's branch tip for this verification;
needs a follow-up bump to the actual merge commit once that PR lands
(flagged with a TODO in Cargo.toml). Version bumped to 0.9.3 to match.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@gbrgr gbrgr changed the title test: pin s3.allow-anonymous's credential-ignoring behavior (opendal 0.58) test: reproduce opendal 0.58's S3/ADLS credential-ignoring regressions Sep 2, 2026
PR #79 (RelationalAI/iceberg-rust) merged as 9eb8921f9; switch from the
temporary WIP branch-tip rev to the real main commit.
@gbrgr
gbrgr enabled auto-merge (squash) September 2, 2026 12:03
@gbrgr
gbrgr disabled auto-merge September 2, 2026 12:08
@gbrgr
gbrgr merged commit 5c8767f into main Sep 2, 2026
6 checks passed
@gbrgr
gbrgr deleted the gb/test-s3-allow-anonymous-credential-priority branch September 2, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants