Skip to content

Add mypy to CI: fix real type errors and generate typed protobuf stubs #30

Description

@craigmcchesney

Summary

Make mypy src/ clean enough to enforce as a blocking CI check. Split out of #29, where a baseline scan found that enforcing it today is not viable.

Current state

mypy src/ reports 175 errors across 19 files. Almost none are real defects:

Error code Count What it actually is
name-defined 112 Name "query_pb2.ColumnTable" is not defined and similar
attr-defined 31 Module has no attribute "PvSelector" and similar
import-untyped 30 missing stubs for grpc and yaml
assignment 2 genuine

The 143 name-defined / attr-defined errors are all the same root cause: the generated _pb2.py files in src/dp_python_lib/grpc/ carry no type information, so mypy cannot see any protobuf message class. The runtime is fine — this is purely a type-checking blind spot.

By file, the noise concentrates in the clients that touch protobuf most heavily: machine_config_client.py (70), query_client.py (31), pv_metadata_client.py (29), query_conversions.py (13).

The 2 real errors

Both in src/dp_python_lib/client/mldp_client.py:

mldp_client.py:101: error: Incompatible types in assignment (expression has type "None", variable has type "AnnotationClient")
mldp_client.py:111: error: Incompatible types in assignment (expression has type "None", variable has type "QueryClient")

self.annotation and self.query are each assigned a client in one branch and None in the other, so mypy infers the non-optional type from the first assignment and rejects the None. CLAUDE.md already documents that client.query is None when no query channel is configured, so the behavior is intended — the annotations just need to say so explicitly (Optional[AnnotationClient] / Optional[QueryClient], or the X | None spelling once the ruff modernization in #29 lands).

Proposed work

Near term — make CI enforcement possible

  • Fix the 2 real errors with explicit optional annotations
  • Add types-PyYAML to the dev extra (clears the yaml import errors)
  • Add mypy config ignoring missing imports for grpc (no official stub package exists)
  • Add mypy config suppressing the generated src/dp_python_lib/grpc/ module
  • Add mypy src/ to ci.yml once the above brings it to zero

This is a blunt instrument — suppressing the protobuf module means client code touching protobuf messages is effectively unchecked — but it converts mypy from "175 errors, unusable" into a check that catches regressions in hand-written logic.

Real fix — type the protobuf stubs

  • Add mypy-protobuf to the stub-generation pipeline so .pyi files are emitted alongside each _pb2.py

This resolves all 143 errors properly rather than hiding them, and gives genuine type checking against the wire format — catching wrong field names and wrong message types at check time instead of at runtime. It touches the generation process described in CLAUDE.md ("Import Fix Process"), including whatever post-processing fixes relative imports, so it needs coordinating with how stubs are produced from the upstream dp-grpc project.

Worth noting this would have real payoff beyond silencing errors: the client code builds protobuf requests by hand throughout, and that is exactly the code the current setup cannot check at all.

Context

Split out of #29 (GHA workflows). That ticket ships CI with ruff, the unit test suite, and the cookbook snippet checker; mypy stays scoped to the cookbook checker (.dev/tools/check-cookbook-snippets.py), where it already works today because it type-checks usage of the library rather than the library's internals.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions