Skip to content

[mypyc] Fix crashes and compile errors from the result of native __len__ methods - #22485

Merged
p-sawicki merged 2 commits into
python:masterfrom
Jahongir-Qurbonov:mypyc-len-native-int
Nov 20, 2026
Merged

p-sawicki merged 2 commits into
python:masterfrom
Jahongir-Qurbonov:mypyc-len-native-int

Conversation

@Jahongir-Qurbonov

Copy link
Copy Markdown
Contributor

The sq_length/mp_length slot of a native class didn't check the value returned by __len__ like CPython's slot_sq_length does, and the compiled len() of a native class instance only partly did:

  • A negative result from __len__ declared to return int or Any was returned from the slot as is, without an exception set. len(x) then raised SystemError: <built-in function len> returned NULL without setting an exception when called from interpreted code, and crashed the interpreter (segfault) in some other cases, such as len(x) in compiled code where x has type Any.
  • __len__ declared to return bool or a native int type (i64, i32, i16, i8, u8, ...) generated PyLong_AsSsize_t(retval) with a C integer retval, which is a C compile error (-Wint-conversion) with current GCC and clang.
  • With -> Any, a result that isn't an int raised TypeError: an integer is required, while Python accepts any object with __index__.
  • The compiled len(x), where x is a native class instance, checked that the length isn't negative, but returned a value that doesn't fit in Py_ssize_t (such as 2**63) instead of raising OverflowError.
class C:
    def __init__(self, n: int) -> None:
        self.n = n

    def __len__(self) -> int:
        return self.n

class B:
    def __len__(self) -> bool:  # C compile error
        return True
>>> len(C(-1))
SystemError: <built-in function len> returned NULL without setting an exception

CPython's slot_sq_length converts the result with __index__, raises ValueError: __len__() should return >= 0 if it's negative, and OverflowError: cannot fit 'int' into an index-sized integer if it doesn't fit in Py_ssize_t.

Now:

  • generate_len_wrapper() handles a short int inline. Anything else goes through the new CPy_LenFromResult() in generic_ops.c, which does what slot_sq_length does, after boxing the value if needed (like generate_hash_wrapper() after [mypyc] Fix OverflowError when a native __hash__ returns a large int #22453).
  • In the compiled len() of a native class instance, the length check is the new CPyTagged_CheckLen() in CPy.h. It's inline for short ints, and for long ints it also raises OverflowError if the value doesn't fit in Py_ssize_t. If __len__ returns something other than an int type (for example Any), the result is converted by CPyObject_LenFromResult() like in the slot.

Tests: the new run test testDundersLenResult covers -> int, -> Any, -> bool, -> i64, -> i8 and -> u8, through both the compiled len() and the slot (len() of an Any value), with negative values, values that don't fit in Py_ssize_t, True, an object with __index__ and results that aren't ints. I updated testDundersLen in irbuild-dunders.test and added testDundersLenAny.

The sq_length/mp_length slot of a native class returned a negative
__len__ result without setting an exception, so len() raised
SystemError or crashed. With a bool or native int return type, the
slot passed the C value to PyLong_AsSsize_t(), which doesn't compile.
With an Any return type, objects with __index__ were rejected.

The slot now converts the result like slot_sq_length in CPython, and
the compiled len() of a native class instance also raises
OverflowError for a length that doesn't fit in Py_ssize_t.
Comment thread mypyc/codegen/emitwrapper.py Resolved
…pper

Move the conversion into CPyTagged_AsLen, which returns the length as a
Py_ssize_t (-1 on error). The sq_length/mp_length wrapper calls it
instead of duplicating the short int fast path, and CPyTagged_CheckLen
(used by compiled len()) is now a thin wrapper around it, so the IR is
unchanged.

@p-sawicki p-sawicki left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@p-sawicki
p-sawicki merged commit 4ad3a3c into python:master Nov 20, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants