Repository navigation
[mypyc] Fix OverflowError when a native __hash__ returns a large int - #22453
Merged
Merged
Conversation
JukkaL
approved these changes
Nov 13, 2026
JukkaL
left a comment
Collaborator
There was a problem hiding this comment.
Thanks, looks good! This fixes a few different bugs.
p-sawicki
pushed a commit
that referenced
this pull request
Nov 20, 2026
…n__ methods (#22485) The `sq_length`/`mp_length` slot of a native class didn't check the value returned by `__len__` like CPython's `slot_sq_length` does, and the compiled `len()` of a native class instance only partly did: - A negative result from `__len__` declared to return `int` or `Any` was returned from the slot as is, without an exception set. `len(x)` then raised `SystemError: <built-in function len> returned NULL without setting an exception` when called from interpreted code, and crashed the interpreter (segfault) in some other cases, such as `len(x)` in compiled code where `x` has type `Any`. - `__len__` declared to return `bool` or a native int type (`i64`, `i32`, `i16`, `i8`, `u8`, ...) generated `PyLong_AsSsize_t(retval)` with a C integer `retval`, which is a C compile error (`-Wint-conversion`) with current GCC and clang. - With `-> Any`, a result that isn't an int raised `TypeError: an integer is required`, while Python accepts any object with `__index__`. - The compiled `len(x)`, where `x` is a native class instance, checked that the length isn't negative, but returned a value that doesn't fit in `Py_ssize_t` (such as `2**63`) instead of raising `OverflowError`. ```python class C: def __init__(self, n: int) -> None: self.n = n def __len__(self) -> int: return self.n class B: def __len__(self) -> bool: # C compile error return True ``` ```python >>> len(C(-1)) SystemError: <built-in function len> returned NULL without setting an exception ``` CPython's `slot_sq_length` converts the result with `__index__`, raises `ValueError: __len__() should return >= 0` if it's negative, and `OverflowError: cannot fit 'int' into an index-sized integer` if it doesn't fit in `Py_ssize_t`. Now: - `generate_len_wrapper()` handles a short int inline. Anything else goes through the new `CPy_LenFromResult()` in `generic_ops.c`, which does what `slot_sq_length` does, after boxing the value if needed (like `generate_hash_wrapper()` after #22453). - In the compiled `len()` of a native class instance, the length check is the new `CPyTagged_CheckLen()` in `CPy.h`. It's inline for short ints, and for long ints it also raises `OverflowError` if the value doesn't fit in `Py_ssize_t`. If `__len__` returns something other than an int type (for example `Any`), the result is converted by `CPyObject_LenFromResult()` like in the slot. Tests: the new run test `testDundersLenResult` covers `-> int`, `-> Any`, `-> bool`, `-> i64`, `-> i8` and `-> u8`, through both the compiled `len()` and the slot (`len()` of an `Any` value), with negative values, values that don't fit in `Py_ssize_t`, `True`, an object with `__index__` and results that aren't ints. I updated `testDundersLen` in `irbuild-dunders.test` and added `testDundersLenAny`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes mypyc/mypyc#1253
The
tp_hashslot of a native class converted the value returned by__hash__withCPyTagged_AsSsize_t(), so an int that doesn't fit inPy_hash_tmadehash()raiseOverflowError. CPython'sslot_tp_hash, which a class defined in Python uses, accepts any int and hashes one that doesn't fit again withint.__hash__.generate_hash_wrapper()now does the same:CPy_HashFromResult()ingeneric_ops.c, which does whatslot_tp_hashdoes with the result: an int that fits in
Py_hash_tis used as is, a larger one is hashed again, andanything that isn't an int raises
TypeError: __hash__ method should return an integer.CPy_HashFromResult(). With-> Any, a result thatisn't an int used to raise
TypeError: an integer is required. A__hash__declared to return a native inttype such as
i64segfaulted (or raisedSystemErrorif it returned 0), because the slot passed the C valueto
PyLong_AsSsize_t()as an object pointer.The new run test
testDundersHashResultcovers-> intand-> Anywith values that fit inPy_hash_t, -1, values that don't fit, a long int that fits only on 64-bit platforms (2**62),Trueand results that aren't ints, plus an-> i64method that returns -1. The expected results are the same on 32-bit platforms.