Skip to content

skills: fix the download path (Files API place, securable vs bundle name) and prompt before fetching - #253

Open
xsh310 wants to merge 8 commits into
databricks:mainfrom
xsh310:skills-dedup-before-fetch-main
Open

skills: fix the download path (Files API place, securable vs bundle name) and prompt before fetching#253
xsh310 wants to merge 8 commits into
databricks:mainfrom
xsh310:skills-dedup-before-fetch-main

Conversation

@xsh310

@xsh310 xsh310 commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Problem

Three problems in the ucode configure skills download path.

1. Bundle reads went to the wrong Files API place, so downloads never worked.
The client read bundle bytes from /Volumes/<cat>/<sch>/<leaf>, but a skill has no backing UC Volume. Listing succeeded (that goes through the 2.1 skills API), then every file fetch failed:

list_schema_skills -> ['ai-ops-oncall-daily', 'ai-ops-investigate', 'task-triage', 'work-recap', 'humanize']
list_skill_files   -> [] reason: HTTP 404 "Volume 'xsh.bb-0806.humanize' does not exist."

The unit tests mocked the HTTP layer and asserted the /Volumes/... URLs, so they encoded the bug instead of catching it, and no test pinned the listing URL at all.

2. A skill's two names were treated as one, so any skill whose names differ 404'd.
A skill has a securable name (the leaf of skills/<cat>.<sch>.<leaf>) and a bundle_name that FinalizeSkill parses from the bundle's SKILL.md frontmatter. Only the securable name resolves through the Files API, while bundle_name is what an agent looks for on disk. _skill_bundle_name preferred bundle_name and used it for both jobs, so fixing problem 1 exposed this:

! Skipping `xsh.bb-0806.task-triage`: HTTP 404 Not Found:
  "Path '/Skills/xsh/bb-0806/task-triage' did not resolve to a Unity Catalog skill."

Nothing keeps the two in sync. FinalizeSkillValidator.verifyFrontmatter checks the frontmatter name for emptiness, length, and control characters, but never compares it to the securable, so the divergence is permitted by design rather than being a data anomaly.

3. The overwrite dedup prompt ran after the bytes were already downloaded.
download_skills fetched every skill's full bundle up front, then prompted per skill at write time. Declining a skill threw away a download that had already completed, wasted work that scales with how much the user already has on disk, which is the common re-download case.

Change

Files API place (0768e1a): point the directory walk and the file fetch at the Skills place through a single SKILL_FILES_API_PREFIX constant.

Two names (fe1295b, 410f21f, 94fd4af): replace _skill_bundle_name with a frozen SkillRef(securable_name, bundle_name), so the type carries the distinction instead of leaving it to a bare string two callers read differently.

  • Fetches take ref.securable_name, the only name the Files API resolves.
  • Directories and dedup use ref.bundle_name, so the directory matches the name: in the SKILL.md written inside it.
  • should_download_skill validates both names, since one reaches a URL and the other the filesystem.
  • --skill matches the securable name only, since that is what identifies a skill in UC.
  • A finalized skill missing either name is skipped with a warning naming the missing field, rather than substituting the other name. Guessing a directory that doesn't match the bundle's frontmatter would hide the skill from the agent meant to load it. An unfinalized skill is still skipped quietly, since having no bundle yet is a normal in-progress state.

Prompt before fetch (415e7c6, e056755): move the disk-only checks ahead of the fetch, so a declined or invalid skill is never downloaded. write_skill splits into should_download_skill (the decision, needs no bytes) and a pure writer. The per schema parallel fetch and the sequential for location in locations loop are unchanged, so cross location overwrite prompts still fire.

Tests

uv run pytest tests/test_skills_download.py gives 48 passed. Full suite excluding the live gateway e2e tests: 1274 passed, 6 skipped. uv run ruff check and the ty type check are clean.

Unit coverage added for each change:

  • test_lists_under_the_skills_place pins the listing URL, previously unasserted, and the mocks that asserted /Volumes/... are updated.
  • test_fetches_by_securable_and_writes_under_bundle_name asserts the fetch uses the securable name while the directory is the bundle name, and that no securable named directory is created.
  • test_skill_filter_matches_securable_name_only asserts both directions: the securable name selects, the bundle name does not.
  • test_either_unsafe_name_is_skipped covers an unsafe value in either position.
  • test_skips_and_warns_when_a_name_is_missing is parametrized over all four missing name combinations, plus test_unfinalized_skill_is_skipped_without_a_warning for the quiet case.
  • test_declined_skill_is_not_fetched asserts fetch_skill_bundle is never called for a declined skill and the existing copy is untouched.

End to end against a real workspace

Verified against xsh.bb-0806 on eng-ml-inference.staging, the workspace where both Skills flags are on. That schema contains a skill whose two names differ, which is what surfaced problem 2:

securable_name=ai-ops-oncall-daily    bundle_name=ai-ops-oncall-daily
securable_name=ai-ops-investigate     bundle_name=ai-ops-investigate
securable_name=task-prioritizer       bundle_name=task-triage        <-- differ
securable_name=work-recap             bundle_name=work-recap
securable_name=humanize               bundle_name=humanize

The two paths, confirmed directly against the Files API:

/Skills/xsh/bb-0806/task-triage       -> 404 "did not resolve to a Unity Catalog skill"
/Skills/xsh/bb-0806/task-prioritizer  -> 200, SKILL.md

Full schema download. download_skills(..., ["xsh.bb-0806"], path) reports Downloaded 5/5, up from 4/5 before the name fix, where task-triage was the one failure. The directories written are:

.claude/skills/  ai-ops-investigate  ai-ops-oncall-daily  humanize  task-triage  work-recap

The skill with differing names lands in task-triage/ (the bundle name), its bytes came from the task-prioritizer path (the securable name), and its SKILL.md frontmatter reads name: task-triage, so the directory and the frontmatter agree. No task-prioritizer/ directory is created, and both roots (.claude/skills and .agents/skills) receive it.

--skill on the mismatched skill. Requesting the securable name downloads it into the bundle named directory; requesting the bundle name is reported as not found:

--skill task-prioritizer  ->  Downloaded 1/1 skill(s)    (writes .claude/skills/task-triage/)
--skill task-triage       ->  Skipping requested skill(s) not found in `xsh.bb-0806`: task-triage.
                              No requested skills to download from `xsh.bb-0806`.

Bundle integrity. For a skill whose names match, the fetched bytes equal the sizes the Files API reports, and nested files are preserved:

files:  ['SKILL.md', 'references/ai-patterns.md']
bundle: {'SKILL.md': 9176, 'references/ai-patterns.md': 21848}

Note that the server side /Skills byte path is still gated by databricks.unitycatalog.aigov.enableSkillsFilesApi, which is default off and enabled only for that test workspace. This PR does not change that gating, it just stops ucode from asking the wrong place with the wrong name.

This pull request and its description were written by Isaac.

Comment thread src/ucode/skills_download.py Outdated
@xsh310
xsh310 marked this pull request as ready for review July 31, 2026 01:49
@xsh310
xsh310 force-pushed the skills-dedup-before-fetch-main branch from 03c30d7 to 829d83a Compare July 31, 2026 22:23
@xsh310 xsh310 changed the title skills: prompt before fetching so declined skills aren't downloaded skills: read bundles from the Files API /Skills place, and prompt before fetching Aug 6, 2026
xsh310 added 3 commits August 6, 2026 21:12
Download mode fetched every skill's bytes up front, then prompted to
overwrite existing dirs at write time — so declining a skill threw away
an already-completed download.

Move the overwrite prompt and invalid-name check ahead of the fetch:
split write_skill into should_download_skill (the disk-only decision,
extracting existing_skill_on_disk) and a pure write_skill, and filter
each schema's leaves through the decision before _fetch_bundles runs.
The per-schema parallel fetch and the sequential location loop are
unchanged, so cross-location same-leaf overwrite prompting still works.
Address review nit — restructure the download_skills docstring into
explicit list/decide/fetch stages. Run ruff format to wrap an
over-length line in tests (fixes the test_ruff_format CI check).
The download client read bundle bytes from `/Volumes/<cat>/<sch>/<leaf>`, but
skills have no backing UC Volume, so every fetch failed against a real
workspace: listing succeeded via the 2.1 skills API, then each file 404'd with
"Volume ... does not exist". Point both the directory walk and the file fetch at
the `Skills` place instead.

The old path survived because the tests mocked the HTTP layer and asserted the
`/Volumes/...` URLs, so they encoded the bug. Updated those and pinned the
listing URL with a test, which was previously unasserted.

Verified end to end against xsh.bb-0806 on eng-ml-inference.staging (the
workspace where both Skills flags are on): download_skills writes each bundle,
nested files included, into .claude/skills and .agents/skills.

Co-authored-by: Isaac
@xsh310
xsh310 force-pushed the skills-dedup-before-fetch-main branch from 6e9e2f4 to 0768e1a Compare August 6, 2026 21:14
xsh310 added 2 commits August 6, 2026 21:35
A skill has two names that are not interchangeable. The securable leaf of
`skills/<cat>.<sch>.<leaf>` is the only one the Files API resolves, while
`bundle_name` is set at finalize from the bundle's SKILL.md frontmatter and is
what an agent looks for on disk. They coincide only when a skill was created
under a securable matching its frontmatter.

`_skill_bundle_name` preferred `bundle_name` and used it for both jobs, so a
skill whose two names differ 404'd:

    Path '/Skills/xsh/bb-0806/task-triage' did not resolve to a Unity Catalog
    skill.

where the securable is `task-prioritizer` and the frontmatter says
`name: task-triage`.

Replace it with a frozen `SkillRef` carrying both, so the type makes the
distinction explicit rather than leaving it to a bare string: fetches take
`ref.securable`, directories and dedup use `ref.bundle`. `should_download_skill`
now validates both names, since each reaches a URL or the filesystem. `--skill`
matches either name, so whichever a user knows works.

Verified against xsh.bb-0806 on eng-ml-inference.staging: 5/5 skills download
(was 4/5), and the divergent one lands in `task-triage/` with a SKILL.md whose
frontmatter matches the directory.

Co-authored-by: Isaac
Rename `SkillRef.securable`/`.bundle` to `securable_name`/`bundle_name` so both
read as the API fields they come from, rather than leaving `bundle` to be
mistaken for the bundle itself.

Narrow `--skill` to match the securable name only. It selects which skills to
download, and the securable is what identifies a skill in UC, so accepting the
bundle name too gave one skill two selectors with no gain. Requesting a bundle
name now reports it as not found, alongside the existing unknown-name warning.

Verified against xsh.bb-0806 on eng-ml-inference.staging: the whole schema still
downloads 5/5, `--skill task-prioritizer` downloads it into `task-triage/`, and
`--skill task-triage` is reported as not found.

Co-authored-by: Isaac
Comment thread src/ucode/skills_download.py Outdated
Comment thread src/ucode/skills_download.py
xsh310 and others added 2 commits August 6, 2026 21:57
Addresses review on _skill_ref: drop the bundle_name fallback and warn instead.

A finalized skill is expected to carry both names -- `name` is immutable from
CreateSkill, and FinalizeSkill is the sole writer of `bundle_name` -- so either
one missing is an anomaly, not a case to paper over. Substituting the securable
name for a missing bundle_name guessed a directory name that may not match the
bundle's SKILL.md `name:`, which would silently hide the skill from the agent
meant to load it. Now both names are required and a skill missing either is
skipped with a warning naming the missing field(s).

An unfinalized skill is still skipped quietly, since having no bundle yet is a
normal in-progress state rather than an anomaly.

Also correct the SkillRef docstring: finalize validates the frontmatter name for
emptiness, length, and control characters, but never compares it to the
securable, which is why the two can legitimately differ.

Extract `_non_empty_str` so both names narrow from the untyped API payload
without repeating the isinstance dance (also keeps `ty` happy).

Co-authored-by: Isaac
@xsh310 xsh310 changed the title skills: read bundles from the Files API /Skills place, and prompt before fetching skills: fix the download path (Files API place, securable vs bundle name) and prompt before fetching Aug 6, 2026
Only the securable name is unique within a schema. `bundle_name` is parsed from
each bundle's SKILL.md frontmatter and never compared against its siblings, so
one schema can hold two finalized skills claiming the same directory.

The decide stage runs before any write, so neither sibling saw the other on disk
and both passed. Both then wrote to the same directory, whichever finished last
won, and `written` counted both:

    ✔ Downloaded 2/2 skill(s)      # one surviving directory, no prompt

Reduce each location's skills to the first claimant of a bundle name and warn
about the rest, naming the winner and how to resolve it. Runs before the decide
stage, so a dropped sibling is never fetched and the summary's denominator counts
only skills that can reach disk. Now:

    ! Skipping `main.default.skill-b`: its bundle name `foo` is already claimed
      by `main.default.skill-a`. Rename one skill's SKILL.md `name:` to download
      both.
    ✔ Downloaded 1/1 skill(s)

Keeping the first rather than prompting matches the existing treatment of
unusable skills, which warn and skip; a prompt here would ask the user to choose
between two skills they cannot tell apart from the directory name alone.

The guard is per location, so a same-named skill from a *later* location still
reaches the overwrite prompt as before -- covered by a test, since that is the
behavior most at risk of regressing here.

Co-authored-by: Isaac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant