Skip to content

DEV-2201 Correct and reorganize Data Sources developer documentation - #316

Merged
hcarneiro merged 15 commits into
masterfrom
fix/DEV-2201
Nov 2, 2026
Merged

hcarneiro merged 15 commits into
masterfrom
fix/DEV-2201

Conversation

@hcarneiro

@hcarneiro hcarneiro commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Product areas affected

Developer documentation for the Data Sources JavaScript API and data-source security, including generated human/LLM discovery indexes. No CLI runtime, Studio or API implementation changes.

What does this PR do?

Corrects incomplete or misleading Data Sources API signatures, options, result shapes and security guarantees. Splits the reference into focused pages for reads, writes, attachments, subscriptions, native storage and source management, alongside security rules, examples and preview testing. Preserves existing entry URLs and legacy fragments, updates operators/views/joins and regenerates discovery with one SDK catalog owner.

Documents the joined-source boundary: a join evaluates only the primary source's rules, so the joined source's read permission and field restrictions are not applied. Explains that hashed fields are returned as **REDACTED**.

JIRA ticket

DEV-2201

Result

Readers can distinguish online, native-local and administrative calls, handle queued/pending writes separately from server persistence and choose method-specific security tests. Examples no longer advertise JSON-unsafe regex/type patterns, universal atomic commit behavior or unverified CSV import.

Validation

  • 191 documentation tests and strict discovery generation pass.
  • 33 preview route variants return HTTP 200; all 62 original SDK heading fragments are retained without duplicate IDs.
  • Security statements were checked against fliplet-api and by calling the SDK directly as signed-in app users. The commit() field-restriction gap found there is tracked in DEV-2269.

Branch preview: https://fix-dev-2201.fliplet-cli.pages.dev/API/fliplet-datasources

Checklist

  • Added automated test coverage as appropriate for this change.

Deployment instructions

Requires fliplet-api v1.69.32 or later (DEV-2269, released 22 Oct): the commit() pages describe that release. After publication, verify the generated discovery, raw Markdown, legacy fragments and the refreshed hosted search index.

Author concerns

The diff is large because long references were split and the full generated documentation index was regenerated.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Deploying fliplet-cli with  Cloudflare Pages  Cloudflare Pages

Latest commit: cc8dd1e
Status: ✅  Deploy successful!
Preview URL: https://b7877b12.fliplet-cli.pages.dev
Branch Preview URL: https://fix-dev-2201.fliplet-cli.pages.dev

View logs

@tonytlwu tonytlwu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (via /fliplet-pr-review)

Combined DEV-2201 review with Fliplet/fliplet-studio#9021. The focused SDK/security documentation and discovery changes substantially improve the contract, but joins need an explicit security limitation so builders do not infer direct-read protection for related private records. See inline comment.

Acceptance evidence: 191 documentation tests and strict discovery generation passed; rendered security documentation was checked at desktop and mobile sizes. These checks do not demonstrate real-actor authorization. Include joined-source access and field restrictions in the Studio allowed/denied test matrix alongside representative and held-out workflows. The API behavior described in the finding is existing behavior identified by source inspection, not a newly demonstrated production exploit or a request to expand this ticket into an API fix.

Review basis: subject=aad1baeb0bcd24a1a549356973910c2726effe75 standards=fliplet-engineering@0.18.0 revision=d8c727a0419f9d87a6e38a767a4b95b84380cc44+dirty source=fliplet-pr-review digest=sha256:40c49b60a8bd5384

Comment thread docs/API/datasources/joins.md Resolved
hcarneiro and others added 3 commits October 9, 2026 10:40
@hcarneiro
hcarneiro marked this pull request as ready for review October 14, 2026 13:25

@farhantariq12b farhantariq12b left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (via /fliplet-pr-review)

Review basis: subject=48c27fa8897c19eb6220ff37cf2e5950f870f8fc standards=fliplet-engineering@0.16.0 revision=f8a269a7f374041e55df4fa3d1d5890f3232d043 source=fliplet-pr-review digest=sha256:b54eff56f86bcdda

Combined DEV-2201 review with Fliplet/fliplet-studio#9021.

Read this first: the content is not what is being blocked

@tonytlwu's joined-source finding is fully resolved, and I found no defect in the documentation itself. The change request below is about the PR's state, not its content — please don't read it as the joins fix not landing.

The joined-source boundary is handled better than asked: a dedicated "Security boundary for online joins" section sits at docs/API/datasources/joins.md:13-19, ahead of the examples as requested, stating that the join evaluates only the primary source's select rules, that the source access lookup does not establish the joined source's read permission or column restrictions, and that primary include/exclude does not sanitize nested joined data. Both Users snippets carry the inline caveat, where/attributes are explicitly marked caller-controlled rather than a security boundary, and it cross-links security-rules.md#joined-source-reads and testing-security.md#check-joined-source-responses. The native-local execution difference is called out too.

Summary

Documentation only — no CLI runtime, Studio or API code. Splits the Data Sources SDK reference into focused pages (reads, writes, attachments, subscriptions, native storage, source management) plus security rules, examples and preview testing; corrects signatures, options, result shapes and security guarantees; updates operators, views and joins; and regenerates the human/LLM discovery indexes with one SDK catalog owner. Entry URLs and legacy fragments are preserved.

Regression risk

No runtime path. The risk is reader-facing: the new write-contract tables and the joins warning describe existing platform limits, so a wrong statement becomes a wrong guarantee someone builds on. That is exactly why the joined-source boundary mattered, and it is the right shape of fix.

Scope exceeds the guard (23 files, +5,175/-6,679) but reads far smaller than it measures: .well-known/llms-full.txt alone is +2,962/-3,995 of generated output, and most of the rest is two long references being split. Net deletion. Branch strategy correct — fix/DEV-2201 to master is right for a generic repo — and the branch is current with master with no foreign files; docs/ only.

Findings

0 Critical, 0 Warnings against the documentation, 1 process Warning. Being explicit so the block is not misread as a content defect.

The body says "Draft for reviewing the changes. Do not merge/publish until the remaining runtime checks and release decision are complete" — but the PR is marked ready for review (isDraft: false) with green CI. The stated state and the actual state disagree, and the only thing preventing a merge is prose a reviewer may not read to the end.

Two things would clear this:

  1. Convert to draft, or remove the "do not merge/publish" language so the PR's state matches its intent. This is the whole of the change request — it is one click.
  2. Close the runtime gates in your own Author concerns: authenticated disposable app-user operations, real upload/storage persistence, server write/readback, security-enabled preview enforcement/restoration, and native-device IO. Source inspection and mocked probes do not identify the deployed SDK revision. This is an evidence gap, not a demonstrated defect — and it is the substantive reason not to publish yet, since this PR's job is to state platform guarantees that readers will rely on.

Acceptance criteria

Not independently re-verified against Jira this round — stating that plainly rather than implying otherwise. @tonytlwu's earlier assessment stands.

CI status

Green — Frontmatter + unit tests and Cloudflare Pages both pass. Branch preview resolves.

Tooling notes

fe-update-check reported FAILED git-pull, so this ran on pack 0.16.0 while the prior review ran 0.18.0 — worth knowing if the standards moved between them. fliplet-kb needs authorisation, so the KB steps used the local checkout. No incident corpus on this machine. I did not re-run the Jekyll build or the 191 documentation tests locally; CI covers the frontmatter and unit suites.

Verdict

REQUEST_CHANGES — on the PR's state and the outstanding runtime evidence, not on its content. The documentation work is good and the joins boundary is resolved thoroughly; I am not asking for edits to the pages.

If you would rather carry the evidence gap as a release gate than a review block, convert this to draft and I will drop to a comment.

Automated review — a human still does the formal approve.

Keep the split security pages and carry over master's owner-key and broad component query guidance; regenerate the discovery indexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hcarneiro

Copy link
Copy Markdown
Contributor Author

Thanks @farhantariq12b. I have updated the description; this PR is no longer a draft. The live, signed-in SDK checks are now in Validation, and the one gap they found (commit()) is tracked in DEV-2269. Master is merged in (c11a161); the split security pages now also carry the owner-key and broad component query guidance from DEV-2203.

hcarneiro and others added 3 commits October 21, 2026 19:02
…ithholding advice

Commits by app users now apply the matching rule's exclude/include, merge
updates onto the stored row and return only the touched rows filtered by the
caller's read rule; append() and replaceWith() still do not filter. The
"leave writes unavailable until a trusted write path" instructions led the
builder to refuse approval workflows, so replace them with a requests and
approvals example built on per-actor exclusions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@farhantariq12b farhantariq12b left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review (via /fliplet-pr-review)

Review basis: subject=7212e8d82f50f99be1cc4e218c78deb655832f61 standards=fliplet-engineering@0.16.0 revision=f8a269a7f374041e55df4fa3d1d5890f3232d043 source=fliplet-pr-review digest=sha256:b54eff56f86bcdda

Combined DEV-2201 re-review with Fliplet/fliplet-studio#9021.

Read this first: the previous feedback is all fixed

Thanks, @hcarneiro. Both things I asked for last time are done: the description no longer says draft, and the live signed-in SDK checks are now under Validation. @tonytlwu's joined-source boundary is still in place at joins.md:13.

This block is on new grounds. Last time I blocked on the PR's state, not its content. This time it's the content: commits a91155a2 and 3330257a publish security guarantees the platform doesn't provide. Once merged, these pages go to developers.fliplet.com and the llms indexes, which the production AI builder reads.

Findings

2 Critical, 1 Warning, all inline:

  • Critical: the new "Requests and approvals" example promises "members can never set the decision fields", but a member can set them with append(), before and after DEV-2269.
  • Critical: commit() is described as it behaves after DEV-2269, which isn't released, and the warnings that covered the gap were deleted.
  • Warning: the testing guidance limits append() testing to apps that use it, which contradicts the same page.

DEV-2269 status (checked live)

  • DEV-2269 ("commit() bypasses access-rule field restrictions and returns every row to app users") is Ready for Release.
  • Its fix, Fliplet/fliplet-api#8834, merged on 2026-10-22 06:07 UTC into projects/DEV-2269-commit-rule-bypass, not master.
  • origin/master has no DEV-2269 commits. The latest API release, v1.69.24 (2026-10-21 16:27), was cut before the fix existed.

Merging this PR publishes it, per its own deployment note, so these pages would be live while production commit() still behaves the old way. Please list the DEV-2269 API release as a hard dependency in the description.

Regression risk

High, and reader-facing. Developers and the AI builder will set up access rules based on these guarantees.

CI status

Green: Frontmatter + unit tests, Cloudflare Pages.

Tooling note

I'm still on pack 0.16.0. The skill pack on my machine is pinned to a stale branch, so it can't update.

Verdict

REQUEST_CHANGES. The rest of the documentation work holds up. The blocker is the approvals example and the commit() pages, which should wait for the API release and describe append() honestly.

Automated review. A human still does the formal approve.

Comment thread docs/API/datasources/security-rules.md Resolved
hcarneiro and others added 3 commits October 22, 2026 11:37
… data sources

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecision

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
farhantariq12b
farhantariq12b previously approved these changes Oct 23, 2026

@farhantariq12b farhantariq12b left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Both earlier blockers are fixed and I checked them against the live code and Jira: the approvals example pins Status with require, so append() is refused, and the commit() pages now match production (DEV-2269 is in v1.69.32). The testing guidance covers every call the rules grant, and CI is green.

The list-value gap behind the approvals and sign-up guidance is DEV-2409, which now has an owner and is in QA.

Keeps the reorganized data source docs and carries DEV-1719's note that
server-executed App Actions bypass access rules into the SDK intro, the
security overview and the rule evaluation section. llms-full.txt is
regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hcarneiro
hcarneiro merged commit 8230b36 into master Nov 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants