feat(io): support opt-in anonymous access for ADLS/Azure Blob storage - #79
Merged
Conversation
opendal's Azdls service (via reqsign-azure-storage) has no anonymous/
unsigned mode: every request goes through a generic credential-provider
chain whose Credential variants (SharedKey/SasToken/BearerToken) all
require a non-empty secret to be considered valid, and the signer hard
errors ("failed to load signing credential") if none can be found or
validated -- unlike its S3 service, which exposes an explicit
skip_signature bypass. An empty SAS token doesn't help either: it's
treated as a found-but-invalid credential, which fails the same way.
So there was previously no way to read a genuinely public/anonymous
Azure container through this crate at all.
Add "adls.allow-anonymous" (ADLS_ALLOW_ANONYMOUS), mirroring
s3.allow-anonymous. When set, azdls_config_build routes to a new
azdls_anonymous_operator_build, which uses opendal's generic Http
service (plain unauthenticated HTTPS: GET with a Range header for
reads, HEAD for stat) instead of the signed Azdls service, scoped to
the same account/container endpoint the Azdls builder would have used.
This is deliberately opt-in rather than inferred from "no credentials
configured": the absence of an explicit adls.* credential can also
legitimately mean "rely on ambient Azure credentials" (managed
identity, Azure CLI, environment variables, all tried by the Azdls
service's own default credential chain), which must not be silently
broken for callers who never asked for anonymous access. Whenever any
credential is configured, azdls_config_build is unaffected and goes
through the real, signed Azdls service exactly as before.
Only read and stat are supported via the anonymous path, which matches
Iceberg's own read-only file access pattern for tables it doesn't own.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Picks up the 4 new symbols from the previous commit: the ADLS_ALLOW_ANONYMOUS constant (re-exported at two module paths) and the new OpenDalStorage::Azdls::allow_anonymous field (also at two module paths).
gbrgr
added a commit
to RelationalAI/RustyIceberg.jl
that referenced
this pull request
Sep 2, 2026
Regression coverage for the ADLS counterpart of the s3.allow-anonymous
incident already covered in this PR: unlike S3's opendal service (which
exposes an explicit skip_signature bypass), opendal's Azdls service has
no anonymous/unsigned mode at all -- every request goes through a
generic credential-provider chain that hard errors ("failed to load
signing credential") if no credential can be found or validated. An
empty adls.sas-token doesn't help either: it's treated as a found-but-
invalid credential, which fails the same way. So reading a genuinely
public, unauthenticated Azure container was not possible through this
stack at all.
RelationalAI/iceberg-rust#79 adds an explicit "adls.allow-anonymous"
opt-in (mirroring s3.allow-anonymous), routing to opendal's generic Http
service instead of the signed Azdls one. Verified locally against this
repo's own docker-compose infra, plus the real public Azure test
container (publicbenchmarks.dfs.core.windows.net/integration-tests,
the same one RelationalAI/raicode's own integration tests read from):
without the flag, table_open correctly fails with AUTH_FAILED; with it,
the read succeeds.
Bumps the git rev to iceberg-rust#79's branch tip for this verification;
needs a follow-up bump to the actual merge commit once that PR lands
(flagged with a TODO in Cargo.toml). Version bumped to 0.9.3 to match.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
gbrgr
added a commit
to RelationalAI/RustyIceberg.jl
that referenced
this pull request
Sep 2, 2026
#112) Regression coverage for two related production incidents found while validating the RustyIceberg 0.9.2 bump in raicode ([raicode#28144](RelationalAI/raicode#28144)). ## S3: `s3.allow-anonymous` ignores configured credentials raicode's Iceberg data-loading integration always sends `"s3.allow-anonymous" => "true"` alongside any explicit credentials, intending it purely as a fallback for when no credentials are configured. Under the fork's old opendal (0.55, pre `RelationalAI/iceberg-rust#78`), that was safe: `allow_anonymous` was only consulted *after* trying to load a real credential, and only skipped signing if none was found (`opendal-0.55.0/src/services/s3/core.rs`, `S3Core::load_credential`). Upstream's opendal 0.58 renamed the flag to `skip_signature` and changed its semantics: it's now checked *first*, unconditionally, before any credential is even loaded (`opendal-service-s3-0.58.2/src/core.rs`: `if self.skip_signature { return ...unsigned... }`). So configured credentials are silently ignored whenever `allow-anonymous` is also set — every request now goes out unsigned, which a private bucket correctly rejects with `403 AccessDenied` / `AUTH_FAILED`. Verified locally against this repo's own docker-compose MinIO: 14/14 pass, including both new assertions (allow-anonymous + valid credentials → `AUTH_FAILED`; allow-anonymous omitted with the same credentials → success). ## ADLS: no anonymous-access mode existed at all Unlike S3's opendal service, opendal's Azdls service had **no** anonymous/ unsigned mode: every request went through a generic credential-provider chain that hard errors (`"failed to load signing credential"`) if no credential can be found or validated. An empty `adls.sas-token` doesn't help either — it's treated as a *found-but-invalid* credential, which fails the same way. So reading a genuinely public, unauthenticated Azure container wasn't possible through this stack at all. [RelationalAI/iceberg-rust#79](RelationalAI/iceberg-rust#79) adds an explicit `"adls.allow-anonymous"` opt-in (mirroring `s3.allow-anonymous`), routing to opendal's generic `Http` service (plain unauthenticated GET/HEAD) instead of the signed Azdls one. Verified against the real public Azure test container (`publicbenchmarks.dfs.core.windows.net`/`integration-tests`, the same one raicode's own integration tests read from): without the flag, `table_open` correctly fails with `AUTH_FAILED`; with it, the read succeeds. Local run: 17/17 pass in the same testset. ## Status - `iceberg`/`iceberg-catalog-rest` currently point at `RelationalAI/iceberg-rust#79`'s branch tip for this verification — **needs a follow-up bump to the actual merge commit once that PR lands** (flagged with a `TODO` in `Cargo.toml`). - Version bumped 0.9.2 → 0.9.3 to match. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
"adls.allow-anonymous"(mirrorings3.allow-anonymous) to enablegenuinely anonymous/unsigned reads from public Azure Blob/ADLS containers.
Why
opendal's Azdls service (via
reqsign-azure-storage) has no anonymous/unsigned mode: every request goes through a generic credential-provider
chain whose
Credentialvariants (SharedKey/SasToken/BearerToken) allrequire a non-empty secret to be considered valid, and the signer hard
errors (
"failed to load signing credential") if none can be found orvalidated — unlike its S3 service, which exposes an explicit
skip_signaturebypass. An empty SAS token doesn't help either: it'streated as a found-but-invalid credential, which fails the same way. So
there was previously no way to read a genuinely public/anonymous Azure
container through this crate at all.
This surfaced as 4 failing
test/DataLoader/iceberg_tests.jl"load fromazure" (public access) cases in
RelationalAI/raicode, caught whilevalidating the RustyIceberg 0.9.2 bump
(raicode#28144) —
opendal was bumped 0.55→0.58 as part of that merge
(#78), and this
regression came along for the ride (the old fork's opendal 0.55 had the
exact same gap; it just happened not to be exercised until now).
How
azdls_config_buildnow takes anallow_anonymous: bool. When true, itroutes to a new
azdls_anonymous_operator_build, which uses opendal'sgeneric
Httpservice (plain unauthenticated HTTPS —GETwith aRangeheader for reads,
HEADfor stat) instead of the signed Azdls service,scoped to the same account/container endpoint the Azdls builder would have
used.
This is deliberately opt-in via the new property, not inferred from "no
credentials configured": the absence of an explicit
adls.*credential canalso legitimately mean "rely on ambient Azure credentials" (managed
identity, Azure CLI, environment variables — all tried by the Azdls
service's own default credential chain), which must not be silently broken
for callers who never asked for anonymous access. Whenever any credential
is configured,
azdls_config_buildis unaffected and goes through thereal, signed Azdls service exactly as before — even if
allow-anonymousisalso (mistakenly) set, matching how S3's
skip_signatureis checkedunconditionally, before any credential is loaded.
Only
readandstatare supported via the anonymous path, matchingIceberg's own read-only file access pattern for tables it doesn't own.
Testing
azdls.rscovering: anonymous fallback buildssuccessfully and resolves paths correctly; the signed path is still used
by default; a configured credential doesn't change which path
allow_anonymouspicks.cargo test -p iceberg --features storage-azdls,cargo clippy -p iceberg --features storage-azdls -- -D warnings,cargo fmt --checkall clean.(
publicbenchmarks.dfs.core.windows.net/.blob.core.windows.net) fromthis environment; will be verified via RustyIceberg.jl and the raicode PR.
🤖 Generated with Claude Code