Support v4 Multi-Bucket Application Keys - #30
Conversation
mlech-reef
left a comment
mlech-reef
left a comment
There was a problem hiding this comment.
v4 b2_authorize_account response is parsed from the wrong location - restricted keys lose their scope
The v4 auth response nests key restrictions under apiInfo.storageApi.allowed:
storageApi.allowed.buckets[]→{ id, name }storageApi.allowed.namePrefixstorageApi.allowed.capabilities
But StorageAPIInfo reads bucketIds/bucketNames/namePrefix at the storageApi top level, where v4 emits nothing. So for any bucket- or prefix-restricted key, B2.buckets and B2.pfx come back empty. The client never learns its own restriction, lists outside the allowed scope, and B2 rejects with 401:
=== RUN TestListBucketContentsWithKey
integration_test.go:1079: b2_list_file_names: 401:
--- FAIL: TestListBucketContentsWithKey (6.24s)
This also breaks the headline feature: a multi-bucket key authorizes but the client forgets which buckets it's scoped to.
Master (v3) returned bucketId/bucketName at the top level, so this only surfaced with the v4 migration. Unrestricted/master keys are unaffected (their allowed is empty), which is why most tests pass.
Adds a live integration test that mints a Multi-Bucket Application Key and confirms it can list within each allowed bucket.
|
@mlech-reef Thanks for taking a look, and apologies on the miss - added an integration test for a multi-bucket key this time around and ran the suite. |
|
Hi @johnmaguire. Sorry for the delay, but my capacity for this project is very low right now. I can confirm that you fix has solved my previous claim. For the rest, I did some LLM-assisted review and I found few things which we need to address. Please check them |
There was a problem hiding this comment.
Verified against the live B2 v3/v4 APIs (master key, single-bucket key, multi-bucket key over two buckets, prefix-only key), the full live integration suite, and go vet/gofmt — details inline.
First, the good news: the v4 struct shapes match the real wire format exactly (apiInfo.storageApi.allowed.buckets[].{id,name}, allowed.namePrefix, bucketIds in create-key request/response); keeping CreateKey on v3 is correct and necessary (v4 b2_create_key rejects singular bucketId: "The bucketId parameter has been deprecated in favor of bucketIds list"); v4 b2_delete_key/b2_list_keys interop correctly with v3-created keys; and the "multi-bucket keys are unusable by v3-only clients" claim is accurate (v3 auth rejects them with 400).
Blocking
-
ListBuckets()401s for multi-bucket keys —b2_list_bucketsdoes no server-side scope enforcement; without abucketIdfilter it rejects any bucket-restricted key. A multi-bucket-key client can look up buckets by name but cannot enumerate them. Needs a per-bucket fan-out or an explicit error. -
Dispatch threshold contradicts the PR body — body says
len(bucketIDs) > 1, code does> 0, soBucketIDs("one-id")mints a v4-only key that v3 clients can't use at all. That drops v3 compat in exactly the single-bucket case this PR's v3-compat design is meant to preserve. Fixing it also incidentally fixes#1for single-ID keys, but ≥2-ID keys still hit#1, so both changes are needed.
Nits
- Live v4 returns
allowedas an object with null fields for master keys ({"buckets": null, ..., "namePrefix": null}), not an absent field — the code handles it, but the comments and thev4AuthJSONfixture encode the wrong shape, so nothing pins the real master-key format. AllowedBucket.Nameis returned even withoutlistBuckets(verified live); it's only empty for deleted buckets.DefaultUserAgentis stillblazer/0.7.2.Key.BucketID/Key.BucketIDsare parsed but never surfaced.- Pre-existing, slightly more relevant now:
B2.Update()doesn't copybuckets/pfxon re-auth. - Pre-existing:
Client.CreateKeyrejects account-wide prefix-only keys even though B2 accepts them and the new parsing handles them; the new error message reads like a B2 constraint rather than a library one.
A v4-minted key cannot authorize against v3, so minting single-bucket keys via v4 silently broke them for v3-only clients. Dispatch to createKeyMultiBucket only for two or more bucket IDs, matching the documented behavior that CreateKey produces legacy single-bucket keys.
b2_list_buckets does no scope narrowing for restricted keys: it rejects any unfiltered request with 401 and accepts at most one bucketId filter. A key restricted to two or more buckets therefore could never enumerate them. Issue one filtered request per allowed bucket and merge, so ListBuckets returns the key's visible buckets as it does for single-bucket keys.
Live v4 returns allowed as an object with null buckets and namePrefix for master keys, not an absent field. The parsing already handled both, but the fixture and comments encoded the absent-field shape, so nothing verified the format B2 actually emits.
The name is returned regardless of the listBuckets capability; it is empty only when the bucket has been deleted.
Update dropped buckets and pfx on re-auth, silently widening the client's view of its own restrictions. Benign while re-auth reuses the same credentials, but a latent trap now that scope drives ListBuckets filtering.
The v4 migration and Multi-Bucket Application Key support are a minor version under semver.
TestListBucketsWithKey and TestListBucketContentsWithKey left their application key behind on every live run; every other key-minting live test already cleans up after itself.
|
Hi @mlech-reef - no worries, I completely understand. I believe I have addressed all the review feedback, and ran a passing live test against my account. |
2 participants
Fixes #27 with the ultimate goal of solving restic/restic#5741
A new b2.CreateKeyMultiBucket was added to avoid breaking the type signature of b2.CreateKey, with support for multi-bucket application keys.
b2.CreateKey continues to use the old API (similar to the terraform provider's usage of
bucket_idvsbucket_ids) in order to support clients which do not support multi-bucket application keys.Client.CreateKey accepts a new BucketIDs option and selects CreateKeyMultiBucket when len(bucketIDs) > 1.
References:
b2-sdk-pythonPR #538 (the porting reference):<Add support for multi-bucket application keys b2-sdk-python#538terraform-provider-b2v0.12.1 (per-call v3/v4 dispatch): https://github.com/Backblaze/terraform-provider-b2/releases/tag/v0.12.1