Skip to content

Fix JsonSerializer returning None and raising TypeError on deserialize - #3150

Open
davexat wants to merge 2 commits into
dpkp:masterfrom
davexat:fix-json-serializer
Open

Fix JsonSerializer returning None and raising TypeError on deserialize#3150
davexat wants to merge 2 commits into
dpkp:masterfrom
davexat:fix-json-serializer

Conversation

@davexat

@davexat davexat commented Aug 11, 2026

Copy link
Copy Markdown

Problem

JsonSerializer is broken since the 3.0 serializer refactor (#3046):

  • JsonSerializer().serialize(topic, headers, data) always returns None
  • JsonSerializer().deserialize(topic, headers, data) raises
    TypeError: the JSON object must be str, bytes or bytearray, not NoneType

This silently corrupts messages produced with value_serializer=JsonSerializer() and crashes consumers using value_deserializer=JsonSerializer().

Root cause

kafka/serializer/json.py inherits from Serializer, Deserializer, whose abstract methods in kafka/serializer/abstract.py are no-ops (pass). The super().serialize(...) / super().deserialize(...) calls in JsonSerializer therefore return None. Before the 3.0 refactor, JsonSerializer inherited from DefaultSerializer, which actually performs the UTF-8 encode/decode.

Fix

Restore delegation to DefaultSerializer by inheriting from it:

class JsonSerializer(DefaultSerializer):

serialize now returns UTF-8 bytes and deserialize decodes bytes before json.loads.

Tests

Added test/test_serializer.py (no serializer unit tests existed):

  • DefaultSerializer: roundtrip (utf-8 / utf-16), bytes-like passthrough (bytes / bytearray / memoryview / None), deserialize(None) -> None, rejection of non-bytes input
  • JsonSerializer: roundtrip dict -> bytes -> dict, None handling, deserializing plain JSON bytes

All tests pass with the fix; the JsonSerializer tests fail against the previous code (reproducing the bug).

Verification

python -m pytest test/test_serializer.py -v
# 11 passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant