Skip to content

[mypyc] Fix attribute offsets when only a subclass defines __call__ - #22483

Merged
p-sawicki merged 1 commit into
python:masterfrom
Jahongir-Qurbonov:mypyc-call-subclass-layout
Nov 20, 2026
Merged

p-sawicki merged 1 commit into
python:masterfrom
Jahongir-Qurbonov:mypyc-call-subclass-layout

Conversation

@Jahongir-Qurbonov

Copy link
Copy Markdown
Contributor

The object struct of a native class that has a __call__ method has a vectorcall field (used through tp_vectorcall_offset). It was placed right after vtable, before the attributes. A class inherits __call__ from its bases, so if only a subclass defines it, the subclass struct has the extra field while the base class struct doesn't. All attributes inherited from the base class were then at different offsets in the subclass, and code compiled for the base class (methods, attribute access through the base type) read and wrote the wrong fields. This is a segfault or memory corruption:

class A:
    def __init__(self, x: str) -> None:
        self.x = x

    def get_x(self) -> str:
        return self.x

class B(A):
    def __call__(self) -> str:
        return self.x

B("b")()  # Segfault

Generated struct of B:

typedef struct {
    PyObject_HEAD
    CPyVTableItem *vtable;
    vectorcallfunc vectorcall;  // not in the struct of A
    PyObject *_x;               // at a different offset than in A
} B;

A.__init__ (compiled for the struct of A) stores x in the vectorcall field of the B instance, so calling it jumps to the address of a str object. With more attributes, base class methods and subclass code also see each other's attributes in the wrong fields.

Now the vectorcall field goes after the attributes, so the attributes of a base class are at the same offsets in all subclasses. The field is only accessed through the struct of the class itself (in its setup function, which isn't shared with native subclasses) and through tp_vectorcall_offset, which is set for each class, so it's fine for its offset to differ between a class and its subclasses. This also works with separate compilation, since each module computes the layout from the class IR in the same way.

Classes where the base class also defines __call__ were not affected, and neither were classes without attributes.

Tests: a run test with __call__ defined only in a subclass, accessing inherited attributes through base class methods, directly and through Any; __call__ overridden in a sub-subclass; an intermediate class with attributes; a base class with weak reference support; and interpreted subclasses (one inheriting __call__, one defining it). A multi-module test has the base class in another module (another group in separate compilation mode). The new run test fails on master.

The object struct of a native class with a __call__ method has a
vectorcall field. It was placed before the attributes, and since
__call__ is inherited, a subclass that defines __call__ when its base
class doesn't had all inherited attributes at different offsets than
the base class. Code compiled for the base class then read and wrote
the wrong fields, e.g. the base class __init__ stored an attribute in
the vectorcall field of the subclass instance, and calling the
instance crashed.

Put the vectorcall field after the attributes instead. It's only
accessed through the struct of the class itself and through
tp_vectorcall_offset, which is set for each class.

@p-sawicki p-sawicki left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@p-sawicki
p-sawicki merged commit 98295b7 into python:master Nov 20, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants