Skip to content

Fix create_partitions sending empty assignments array instead of null (#3146) - #3147

Closed
ebrard wants to merge 1 commit into
dpkp:masterfrom
ebrard:fix/create-partitions-null-assignments
Closed

Fix create_partitions sending empty assignments array instead of null (#3146)#3147
ebrard wants to merge 1 commit into
dpkp:masterfrom
ebrard:fix/create-partitions-null-assignments

Conversation

@ebrard

@ebrard ebrard commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #3146.

Problem

create_partitions({topic: N}) with a plain int total count — the form the docstring recommends — fails against a normal broker with InvalidReplicationAssignmentError (error 39):

kafka.errors.InvalidReplicationAssignmentError: [Error 39]
  CreatePartitionsTopic(name='demo-topic', count=6, assignments=[])
  error_message='Attempted to add 3 additional partition(s), but only 0 assignment(s) were specified.'

create_partitions({topic: NewPartitions(N)}) (auto-assign, no manual assignments) is also broken — it raises TypeError: 'NoneType' object is not iterable, because the branch iterates count.new_assignments unconditionally even though it defaults to None.

Root cause

_process_create_partitions_input always built the per-topic assignments field as a list. For the int form (and a bare NewPartitions) that yields an empty list [], which serialises to a present-but-empty array rather than null. The broker reads a present array as "manual replica assignment" and rejects it because the assignment count doesn't match the number of new partitions.

Verified at the byte level (assignments field only, after count=00000006):

# v3 (compact array length):
[]   -> ...00000006 01 ...   # length 1 => 0 elements, present array
None -> ...00000006 00 ...   # 0 => null

# v0 (classic 4-byte array length):
[]   -> ...00000006 00000000 ...   # length 0, present
None -> ...00000006 ffffffff ...   # -1 => null

null means "auto-assign replicas"; a present array means "manual assignment". The empty list is what triggers error 39.

Fix

Pass assignments=None when the caller supplies no manual assignments, so it serialises to null and the broker auto-assigns. Manual assignments (dict form and the deprecated NewPartitions form) are preserved unchanged.

Minor related change: the dict branch now uses count.get('assignments') instead of count['assignments'], so {'count': N} with no assignments means auto-assign rather than raising KeyError. This is backward compatible (any dict that previously worked still works).

Tests

Added unit coverage in test/admin/test_admin_topics.py asserting _process_create_partitions_input produces assignments is None for the int and bare-NewPartitions inputs, and preserves explicit manual assignments. The auto-assign test fails on the current code and passes with this change.

Verified:

  • test/admin/test_admin_topics.py — 20 passed (incl. the 2 new tests)
  • test/protocol/admin/test_protocol_admin.py — 121 passed
  • End-to-end against a live confluentinc/cp-kafka:latest broker: create_partitions({topic: N}) (int) now increases the partition count instead of raising error 39, confirmed with rpk topic describe.

_process_create_partitions_input always built the per-topic assignments
field as a list. For the int total-count form (and a bare NewPartitions
with no manual assignments) this produced an empty list [], which
serialises to a present-but-empty array. The broker reads that as a
manual replica assignment and rejects the request with
InvalidReplicationAssignmentError.

Pass assignments=None when the caller supplies no manual assignments so
it serialises to null and the broker auto-assigns replicas. Add unit
coverage for the auto-assign and manual-assignment paths.

Fixes dpkp#3146
@dpkp

dpkp commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Thanks; I think the fix is much simpler than this -- we can just update the default to 'null'. However, the tests are much appreciated!

@dpkp

dpkp commented Aug 4, 2026

Copy link
Copy Markdown
Owner

Updated in #3148

@dpkp dpkp closed this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

KafkaAdminClient.create_partitions with int total count fails with InvalidReplicationAssignmentError (sends empty assignments array instead of null)

2 participants