Skip to content

Store an edge row address in each CSR slot #73

Closed

Description

@prrao87

Problem

A CSR slot holds only the neighbour key, so it can't point back to the edge row it came from. As a result:

Idea

Store each edge's Lance row address next to its neighbour, and fetch the requested relationship properties with take_rows, the way the fused target lookup already fetches node properties. The review of lance-format/lance-graph#162 makes the same point: adjacency entries need an edge row id for relationship properties and parallel-edge semantics.

This is Stage 1 of #72, which also relies on it to update a stored CSR after the old dataset version has been cleaned up.

To watch

  • About 8 more bytes per edge per direction.
  • A large, filtered hop can be slower as scattered take_rows than as one filtered scan and a hash join, especially in lazy mode. IC5's joinDate filter on hasMember is the benchmark query to watch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions