Skip to content

Design layered memory and persistent prefetch cache #8

Description

@RenanGBarreto

Why

Critical metadata should remain in memory when practical, while larger file-region data may eventually need persistent disk storage. Current implementation has no RemotePFS-owned disk cache.

Plan

  • Define cache-layer requirements without adding an unimplemented cache protocol.
  • Keep current executor reading through NBD/mapper and nbdkit cache.
  • Measure metadata memory use and nbdkit temporary-file behavior first.
  • Design memory-to-disk promotion only after a real RemotePFS-owned backend exists.
  • Define ownership, generation identity, eviction, restart recovery, and invalidation semantics before implementation.

Acceptance criteria

  • No parallel cache coherence or invalidation path is introduced prematurely.
  • Design documents memory, future disk, and NAS fallback layers.
  • Cache ownership and generation boundaries are explicit.
  • Prototype includes measurements and restart/reload behavior before production use.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions