Skip to content

docs: teach the data app skill to page through stored DataFrames - #38

Draft
tkislan wants to merge 1 commit into
mainfrom
tk/data-app-stored-dataframes
Draft

tkislan wants to merge 1 commit into
mainfrom
tk/data-app-stored-dataframes

Conversation

@tkislan

@tkislan tkislan commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

What

Data apps can now page through DataFrames stored in the project, without running the notebook. A notebook block stores a frame with deepnote-toolkit's store_dataframe(df, '<name>'), and the page reads it a page at a time with Deepnote.connect().dataframe('<name>', {pageIndex, pageSize}). This PR teaches the deepnote-data-apps skill to use them.

  • SKILL.md, The Notebook:
    • When to store a frame instead of putting it in the JSON output.
    • Store it with a writable create_run, then check the reference ID the block returned. Read-only runs, viewers' runs included, store nothing.
    • Keep the storing block in the notebook, because running it again refreshes the frame.
    • Keep input-dependent data in the JSON output.
    • Store a SQL result from a SQL block in DataFrame output mode, never pd.DataFrame(df) of a 100-row query preview.
    • Running blocks in the editor leaves the JSON output alone but does refresh a stored frame.
  • SKILL.md, The Page:
    • One Deepnote.connect() client, used for both the notebook and dataframe().
    • The page defaults, the 1000-row cap, and the response fields.
    • int64 and decimal values arrive as strings: convert them, keep null, and keep ID columns as strings.
    • Start again from page 0 when a new frame replaces the stored one.
    • Load the frame without waiting for appData(). A page that shows only stored frames calls neither appData() nor run().
    • Reads are refused in projects with private outputs.
  • SKILL.md, Testing: the preview command takes --project-id.
  • scripts/app-dev.mjs: a required --project-id option, checked to be a UUID and sent as projectId in the token reply. dataframe() builds its URL from that ID, and without it the client rejects with "Reload the page to read dataframes."
  • Manifests: version 0.9.2 → 0.10.0, so installed plugins update.

Hold

Keep this as a draft until https://deepnote.com/static/app-client/v1.js contains Reload the page to read dataframes. Today it doesn't, so the hosted client has no dataframe() yet.

Testing

  • app-dev.mjs without --project-id exits 1 and prints the usage, which documents the option. A non-UUID value exits 1 with an error. With a valid ID, the stand-in page's token reply includes projectId.
  • Not yet run end to end: a preview of a page that reads a stored frame. Once the client ships, store a 2,500-row frame with a writable create_run, mint a token, run the preview with --project-id, and check that the bar reads "Connected to Deepnote. No app errors." and that the page shows rowCount 2500 and pageCount 3.

@coderabbitai

coderabbitai Bot commented Nov 20, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant