Skip to content

Add SafeDisk.with_types to allow chosen classes and module prefixes while keeping reads and writes restricted - #23

Merged
avgvi merged 1 commit into
masterfrom
feat/safedisk-with-types
Oct 18, 2026
Merged

avgvi merged 1 commit into
masterfrom
feat/safedisk-with-types

Conversation

@avgvi

@avgvi avgvi commented Oct 18, 2026

Copy link
Copy Markdown
Member

Adds SafeDisk.with_types(*types, modules=()), so caches that hold application objects (dataclasses, named tuples, enums, pydantic models) can use the restricted reader instead of falling back to the default Disk. Downstream libraries such as dspy currently maintain their own restricted unpickler for this.

  • with_types returns a SafeDisk subclass whose allowlist adds the given classes, matched by identity, and every class defined in the named modules or below them. Calling it on the returned class adds to its allowlist.
  • SafeUnpickler takes names, classes and modules. A class is looked up attribute by attribute, every step must be a class, and the result must be one of the given classes or defined in a trusted module. A dotted name such as os.system reached through a module that imports os, a class a trusted module only imports, a module, and dunder attributes are refused. Classes are looked up only in modules already in sys.modules, so reading never imports a module.
  • Writes stay exact: a value outside the standard-library types is pickled and the names its pickle looks up (GLOBAL, INST, STACK_GLOBAL with the memo tracked) are checked against the same allowlist, so a value the reader would refuse raises TypeError naming the class before anything is written. Pickles the scan cannot follow, such as copyreg extension codes, are checked with a trial restricted load.
  • numpy classes and numpy module prefixes are refused (UNSAFE_MODULES): numpy restores a dtype's internal flags from the pickle without checking them, so a crafted array pickle can make an array read raw bytes as object pointers. The docs suggest storing arrays as bytes.
  • Derived classes use a metaclass registered with copyreg, so a Cache, FanoutCache, Deque, Index or DjangoCache using one pickles as before; a class defined in a module on top of a derived class stays its own root.
  • The plain SafeDisk behaves exactly as in 5.7.0.
  • Tests in tests/test_safedisk_types.py with model classes in tests/safedisk_models/: round trips for protocols 0 to 5, refusals at write and read, a poisoned row, the dotted-name and module-import cases, the prefix boundary, pickling, every container, pydantic models, and the restricted-disk pattern dspy uses running on the fork. The CI test job installs numpy and pydantic for them.

Stacked on #22.

Verified: 452 tests pass on Python 3.12; blue, isort, flake8, mypy, doc8 and the Sphinx build with warnings as errors pass.

@read-the-docs-community

read-the-docs-community Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 diskcache-community | 🛠️ Build #35226540 | 📁 Comparing a12c6ab against latest (823b63a)

  🔍 Preview build  

7 files changed · ± 7 modified

± Modified

@avgvi
avgvi force-pushed the feat/safedisk-with-types branch from 56cf9b9 to a12c6ab Compare October 18, 2026 19:14
@avgvi
avgvi force-pushed the feat/separate-import-name branch from b3f1248 to 6637c73 Compare October 18, 2026 19:14
Base automatically changed from feat/separate-import-name to master October 18, 2026 19:30
@avgvi
avgvi merged commit 662798d into master Oct 18, 2026
45 of 67 checks passed
@avgvi
avgvi deleted the feat/safedisk-with-types branch October 18, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant